Could a site like Twitter run in App Engine?

Question

Just wondering to know. Anyone has thoughts about it?

Answer 1

Yes, it could. There are Java, Python and persistent storage supported. That's enough to build simple twitter analogue.

Answer 2

There is manual for GAE at the app-engine site

Answer 3

Depends on your requirements and the design for such a site. Either come up with a design and then see if it would fit on GAE. Else, design it such that it fits into GAE. If one of your requirement is scalability then GAE promises that

Answer 4

Based on your update, I guess your actual question is whether GAE (Google AppEngine) could handle a volume comparable to Twitter's volume.

Twitter gets something on the order of 20-30 million tweets per day. In addition to supporting the posting of tweets, they also have to serve them up to everyone reading them. I'll pull numbers out of the air and guess that the typical tweet is read 20 times (the real number is skewed: a few tweets are read by vast numbers and others are probably never read by anyone; also a few systems attempt to read every tweet).

So the question is whether GAE can support 500 million requests per day.

According to GAE documentation, a paid (not free) application is allocated up to 500 requests per second. That's too low by a factor of 10, but it also says that you can request additional requests per day if needed. I am sure if you cared enough to pay for it that Google would be willing to dedicate the resources of 10 applications. Their infrastructure can certainly support it -- this is dwarfed by the number of searches the Google homepage supports daily.

So the answer is yes, GAE could support Twitter. (No other resource seems likely to pose as much of a problem as the number of requests.) A different question would be whether it would be BEST and be COST EFFECTIVE to run an app of this size on GAE; that would depend on other factors, but the answer is probably "No". At that scale, you can get savings from specialized hardware of your own (although it WOULD be nice to have Google paying all the administrative costs).

Answer 5

A Twitter-like microblogging application called Jaiku is running on App Engine today. It's owned by Google and open-sourced so you could build your own if you like.

Answer 6

I'm going to play devil's advocate and say that there are definitely a few limitations that will prevent App Engine from running every feature Twitter has:

• Custom real-time indexing for search. There are a few projects for full text search as well as a popular issue in the issue tracker for full-text search to be implemented in the future
• Comet support doesn't exist in App Engine. Some parts of Twitter's side use comet
• Twitter has a "Streaming API" which is basically an HTTP connection that never closes. This is also known as the firehose. Since requests have a 30 second limit, you won't be able to implement this

Twitter is a site with a very unique set of scalability problems. It's definitely possible to serve a similar site with Twitter's traffic characteristics on App Engine, but it is impossible to build an exact Twitter clone using only App Engine.