Google App Engine: tasks vs threads?

Question

I have a local scientific application that uses threads to process independent pieces of a large calculation. My group would like this to become a web application, so now I'm figuring out how to port it (so please forgive any complete bonehead statements). I'm using Google App Engine to take care of the "web" part of it, but I'm still working out what other translations are appropriate.

My first inclination was to leave the calculation completely as is (e.g., digest some web form data into the appropriate format, pass that into calculator which spawns threads, etc).

However, I'm also reading about Queues + TaskOptions - that sort of looks like what I should be using instead of ExecutorService + Callable. The individual sub-calculations can take a bit to process (though they can also differ widely in amount of time required), so I guess ideally I'd like a user to be request the whole calculation and then be taken to a page which loads the results as they become available.

Are Queues + TaskOptions the right thing to be using? If not, what is? If so, are there convenient parallel examples to what I'm looking to do?

Lastly, my group has some short-term qualms about wide-release of the internals of the calculation so its details are all server-side - should that allay those concerns?

Eventually (after some publications etc) we plan to make those internals openly available and presumably then the web version could move the calculation client-side. Is there a preferable way to do implementation now that will make that future translation simpler? Or is that not even worth worrying about (since I already effectively have a "client-side" version in my local application)?

Answer 1

You cannot create threads on App Engine. Task queues+ Task options would be the way to go.

If you want to keep things internal - then keeping it on the server side would be the safest thing to do now. However you can think of exposing your work flow as a service, so that in the future clients can talk to the same service.

Answer 2

1. All requests (including tasks) in app engine have a time limit of 30 seconds. If your calculations will take longer than that, you will need to figure out a way to break them down into smaller chunks. App engine's sweet spot is web apps, not number crunching.

2. If you want to be able to share code between the client and server side, one option is to look into GWT. (google web toolkit) It will allow you to write a client using java source code that it then converts to javascript. This would be one method of reusing the number crunching code you already have.