I love the new TaskQueue API. I have a question about the ETA/Countdown, if I set it a new Task to execute 10 minutes in the future and it is the only item in the queue - will it execute in roughly 10 minutes or will it execute straight away? ...
What does the TombstonedTaskError mean? It is being raised while trying to add a task to the queue, from a cron-job: Traceback (most recent call last): File "/base/python_lib/versions/1/google/appengine/ext/webapp/__init__.py", line 501, in __call__ handler.get(*groups) File "/base/data/home/apps/.../tasks.py", line 132, in get ...
In Task Queues code is executed to connect to the server side through URL Fetch. My file queue.yaml. queue: - Name: default rate: 10 / m bucket_size: 1 In such settings, Tusk performed all at once, simultaneously. Specificity is that between the requests should be delayed at least 5 sec. Task must perform on stage with a d...
I need to track data from another website. Since it's spread over 60+ pages, I intend to use a daily cron job to add a task to the queue. This task then should take care of one page and depending on some checks, put another instance of itself on the queue for the next page. Now a simple taskqueue.add(url='/path/to_self', params=control...
I'm trying to use compressed data with my Tasks in the Task Queue like so: t = taskqueue.Task(url='/tasks/queue', params={'param': zlib.compress(some_string)} However when I try to decompress it in the queue handler like so message = self.request.get('param') message = zlib.decompress(message) I get this error: ...
I have been testing the taskqueue with mixed success. Currently I am using the default queue, in default settings ect ect.... I have a test url setup which inserts about 8 tasks into the queue. With short order, all 8 are completed properly. So far so good. The problem comes up when I re-load that url twice under say a minute. Now watc...
The latency for a datastore put is about 150ms - http://code.google.com/status/appengine/detail/datastore/2010/03/11#ae-trust-detail-datastore-put-latency. About how much CPUTime is used by a single datastore put with data size of 100 bytes, into an entity that has only 1 property, and no indexes? Also, does anyone know about how much...
I'm curious if it's common practice outside of the GAE platform to be able to defer tasks to background workers via webhooks. I find it particularly useful to speed up the front-end of webapps, by delegating any long process to background tasks. I'd like to hear about open source software allowing to implement a TaskQueue-like API, wit...
Are dashes(-) or any other special characters allowed in task names in task queues? ...
I can't figure out how to point unit tests at the queue config file. Unit Test snippet // TaskQueue setup LocalTaskQueueTestConfig tqConfig = new LocalTaskQueueTestConfig(); tqConfig.setQueueXmlPath("/war/WEB_INF/queue.xml"); Stack Trace java.lang.IllegalStateException: The specified queue is unknown : zip-fetch at com.go...
I was playing with Goole App Engine Task Queue API to learn how to use it. But I couldn't make it trigger locally. My application is working like a charm when I upload to Google servers. But it doesn't trigger locally. All I see from the admin is the list of the tasks. But when their ETA comes, they just pass it. It's like they runs but ...
I have created a new task queue and defined it in queue.yaml I am not sure how to start adding tasks to this queue? with the default queue it is simple taskqueue.add(...) how do we do it for a custom queue? ...
I downloaded and am running the latest SDK (in About GoogleAppleEngineLauncher, I see it is version 1.3.4.794), but when I open the SDK Console and go to the Task Queue section, I still see "Tasks will not run automatically. Select a queue to run tasks manually." I have not added the flag --disable_task_running, so I'm confused as to why...
In the video/PDF from "Data pipelines with Google App Engine" Brett puts "now / 30" into the task name noting that he will explain the reason later, but somehow he never does. :) http://www.youtube.com/watch?v=zSDC_TU7rtc#t=41m35 task_name = '%s-%d-%d' % (sum_name, int(now / 30), index) Do you have any idea about the reason? Does it h...
On my local dev environment, when an App Engine task that had been added to the the task queue hits an error, it is retried until successful. However, in the production environment, it's not. What I THINK is happening is that, because I have a custom 500 handler in urls.py, all errors are caught by this and the 500 error never bubbles to...
I am using task queue for certain data updation in GAE. My queue.xml file look like below <queue> <name>data-processing</name> <rate>20/s</rate> </queue> My queue processing servlet decrease credit by 1 for every task. While processing it need to check for credit availability and proceed further only if credit is av...
Is there a way to pass multiple parameters in a Queue in google-app-engine? I use the code below Queue queue = QueueFactory.getQueue("sms-queue"); queue.add(TaskOptions.Builder.url("/SQ").param("id",pId)); in my servlet this id is retrievd as a query string. long pID = Long.parseLong(req.getParameter("id")); I need to pass 6 ...
I'm trying to using taskqueues in Google App Engine. I have something like class TrackBuildHandler(webapp.RequestHandler): def post(self): track_name = self.request.get('track_name') taskqueue.add(url='/runbuild/track', params={'track_name': track_name}) class TrackBuildWorker(webapp.RequestHandler): def post(se...
I need to run 2 tasks around 2 minutes apart but need to insert them both at the same time into the queue. how do i do this? taskqueue.add('/task1', { 'args' : 'a' }) taskqueue.add('/task2', { 'args' : 'b' }) is there some way i can insert a time component into this execution? can you please illustrate with an example? ...
I am running thousands of tasks per hour in Google App Engine and around .1% of them are failing even after multiple retries. Ideally, I want them to stop trying and exit. However, due to app engine's design they seem to be just trying again and again. I know there is a backoff time which increases with every unsuccessful execution, but ...