should I drop Google App Engine?

Question

I am developing some kind of financial market simulation on GAE. Although I have attained much progress, I have begun to consider dismissing GAE and going for a Django + rdbms solution for the last few days. Let me state my reasons:

1. transactions: GAE supports transactions with single entity groups. if an application involves complex transactions, such as one in a financial market, you cannot use this mechanism (read: no transaction mechanism is available). A solution addressing this issue has been developed by some noble users, but not been released yet, and is said to be available in java only.

2. full-text search: ironic it is, but a very primitive API exists at present. Moreover, an improvement is not mentioned in the roadmap.

3. paging: developing a friendly paging mechanism is not for the faint-hearted (or for the deadline-pressed).

4. background process: Batches are important in financial applications, and for the time being I have to experiment with Cron. But I conceive that background processes are in the roadmap and will be available in a short time.

I chose GAE in order to shorten development time and cope with the scalability issues that may occur in the future easily. I really like GAE, it has some excellent features, but addressing these issues together will take too much effort, and likely cancel out any benefits of using GAE in the first place.

What would you do if you were me?

Answer 1

How much work would it be to move it? Interesting to hear about your experiences with GAE.

I always think its better to have more control, and if you are hitting against brick walls you should move. That's my opinion, hope it helps!

Answer 2

As you learned, often one does not discover all the issues and pitfalls of a development environment like GAE untill after one has used it for a while. This is most likely true of any environment.

Before committing to an alternate environment ( Django + rdbms), how familiar are you with that platform? If you are new to it, you may just find that you run into other issues.

If on the other hand you are throughly familiar with Django + rdbms then it is another story. Of course one wonders if you are familiar with them, why did you pick GAE in the first place? You must have felt that it offered some benefits that you will be giving up by dropping GAE.

Perhaps you need to make a list of the pluses and minuses for each platform and then try to balance the lists to make a decision. Ultimately there are no magic bullets, and you will undoubtably have some issues no matter wich way you go.

Answer 3

if you need background transactions+cloud-like scalability Amazon Elastic Cloud Computing is the only possiblity that i know of. its a bit harder to maintain, since you basically get your whole operating system, but this gives you also the flexibility to run anything you want..

Answer 4

As Andreas say take a look at Amazon Web Services, it is a serious production environment and will give you many things you may not have yet thought about including a Content Delivery network (saves your bandwidth), Simple db and the interesting (but I've yet to think of a use I can put it to) Machanical Turk service, Oh year and EC2 virtual servers - run whatever you like on these.

Answer 5

After chewing over the subject, I have decided to go for a django + rdbms solution. It seems that GAE needs time to reach maturity.

Answer 6

1. transactions: Your correct, GAE can not handle complex queries, but nothing with a Big-Table approach would be able to handle complex calculations like a RDB.

2. full-text search: Having worked with the 'primitive' API for full-text search you are right to stay away, it is very lack luster and by no means scalable (cannot combing more than four terms to search, if I remember correctly).

3. background process: The CRON process available with the latest release isn't half bad, then again I think your looking to do much more with background processes.

Answer 7

Hi Shanya. I am also interested in developing a market simulator. I'm pretty au-fait with django / python etc and would like to hear how you are doing. Any chance of getting in touch ? I'm not sure how to send a priv message though and don't really want to put my email addy up here !