5

I have a [python] AppEngine app which creates multiple tasks and adds them to a custom task queue. dev_appserver.py seems to ignore the rate/scheduling parameters I specify in queue.yaml and executes all the tasks immediately. This is a problem [as least for dev/testing purposes] as my tasks call a rate-throttled url; immediate execution of all tasks breaches the throttling limits and returns me a bunch of errors.

Does anyone know if task scheduling if dev_appserver.py is disabled ? I can't find anything that suggests this in the AppEngine docs. Can anyone suggest a workaround ?

Thank you.

Improve this question

3 Answers 3

Reset to default
4

When your app is running in the development server, tasks are automatically executed at the appropriate time just as in production.
You can examine and manipulate tasks from the developer console: http://localhost:8080/_ah/admin/taskqueue

Documentation here

Improve this answer
Sign up to request clarification or add additional context in comments.

4 Comments

Thanks for the documentation link. "automatically executed at the appropriate time" - that's not my experience. Eg if I specify queue.yaml rate as '6/m' and bucket_size as 1 then I'd expect to get one task executed every 10 secs; instead they all get executed immediately. So I'm asking if anyone else has had a similar experience; if I've configured queue.yaml incorrectly or if perhaps there is a problem with dev_appserver task scheduling.
I'm reading the queue.yaml docs now. I can see that 6/m means "tasks will be processed at a rate of 6 times per minute"; but there's no guarantee it's every 10 secs. But how does adding a rate of 10/s help ? Doesn't this mean you get 10 tasks per second [rather than one every 10 secs] ? Or do you mean there's a separate 'rate_limit' field ? [can't see one]. Thanks
Ok - thanks - then back to the original question. Firstly, I started with a rate of 6/m and 24 tasks, yet all 24 tasks were executed inside of a minute. So this suggests to me that task scheduling is not working on dev_appserver.py. Secondly, can anyone suggest how to configure queue.yaml so a task is processed every ten seconds ? [or at least the rate is no quicker than ten seconds]
@Just could you share your code [queue.yaml+task submission]?
1

The documentation lies: the development server doesn't appear to support rate limiting. (This is documented for the Java dev server, but not for Python). You can demonstrate this by pausing a queue by giving it a 0/s rate, but you'll find it executes tasks anyway. When such an app is uploaded to production, it behaves as expected.

I opened a defect.

Improve this answer

2 Comments

Can you share the link to the reported issue, please. i believe this hasn't been resolved.
It's linked from the word "defect".
0

Rate parameter is not used for setting absolute upper bounds of TaskQueue processing. In fact, if you use for example:

rate: 10/s
bucket_size: 20

the processing can burst up to 20/s. Something more useful would be:

max_concurrent_requests: 1

which sets the maximum number of execution to 1 at a time.

However, this will not stop tasks from executing. If you are adding multiple Tasks a time but know that they need to be executed at a later time, you should probably use countdown.

_countdown using deferred library
countdown using Task class

Improve this answer

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.