How do Singletons in Google App Engine (or more generally in a distributed server environment) work?

Question

Hi,

I am intrigued as to how singletons work in Google App Engine (or any distributed server environment). Given your application can be running in multiple processes (on multiple machines) at once, and requests can get routed all off the place, what actually happens under the hood when an app does something like: 'CacheManager.getInstance()'?

I'm just using the (GAE) CacheManager as an example, but my point is, there is a single global application instance of a singleton somewhere, so where does it live? Is an RPC invoked? In fact, how is global application state (like sessions) actually handled generally?

Regards, Shane

Answer 1

Caches are generally linked up with some sort of distributed replicated cache. For example, GAE uses a custom version of memcached to handle maintaining a shared cache of objects across a cluster, while maintaining the storage state in a consistent state. In general there are lots of solutions for this problem with lots of different tradeoffs to be made in terms of performance and cache coherence (eg, is it critical that all caches match 100% of the time, must the cache be written to disk to protect against loss, etc).

Here are some sample products with distributed caching features (most have documentation describing the tradeoffs of various approaches in great detail:

• memcached - C with lots of client APIs and language ports
• Ehcache - OSS Java cache, with widespread adoption
• JBoss Cache - Another popular Java OSS solution
• Oracle Coherence (formerly Tangosol Coherence) - Probably the best known Java commercial cache.
• Indexus Cache - A popular .Net OSS solution
• NCache - Likely the most popular .Net commercial caching solution

As you can see, there have been many projects that have approached this problem. One possible solution is to simply share a single cache on a single machine, however, most projects make some sort of replication and distributed failover possible.

Answer 2

I'm not sure on the specifics of GAE, but typically in a web app this size, you'll have multiple processes running over a number of machines (and then load balance between them). Within each process, if you're using a multi-threaded web server, you can be handling multiple requests. So this would allow you to share objects between requests within the same web server (and a singleton, for example, you would instantiate when the web app process starts).

If the web server is not multi-threaded, but rather multi-process, then you can't share objects between requests as far as I know, without talking to a separate caching process.

The GAE docs seem to support what they call "App Caching" which essentially allows you to do the same thing, but it wasn't clear to me from the docs whether they're doing this by using multi-threaded web servers, or some other caching process that is running alongside the web servers.

I'd be intrigued to know if CacheManager.getInstance() always resolves to the same object, or if it's only the same object for requests handled by the same web server. In reality, it doesn't matter as it's only being used to talk to the separate memcached process anyway.

Answer 3

The singletons in App Engine Java are per-runtime, not per-webapp. Their purpose is simply to provide a single point of access to the underlying service (which in the case of both Memcache and Users API, is accessed via an RPC), but that's purely a design pattern for the library - there's no per-app singleton anywhere that these methods access.