1 A-record for every subdomain (10000+); any potential issues? Any other solution?

Answer 1

Use the wildcard DNS entry, then use load balancing to distribute the load between servers, regardless of what client they are.

While you're at it, skip the URL rewriting step and have your application determine which account it is based on the URL as entered (you can just as easily determine what X is in X.domain.com as in domain.com?user=X).

EDIT: Based on your additional info, you may want to develop a "broker" that stores which clients are to access which servers. Make that public facing then draw from the resources associated with the client stored with the broker. Your front-end can be load balanced, then you can grab from the file/db servers based on who they are.

Answer 2

Why can't you use a load balancer?

Answer 3

If you use tinydns, you don't need to restart the nameserver if you modify its database and it should not be a bottleneck because it is generally very fast. I don't know whether it performs well with 10000+ entries though (it would surprise me if not).

http://cr.yp.to/djbdns.html

Answer 4

The front-end proxy with a wild-card DNS entry really is the way to go with this. It's how big sites like LiveJournal work.

Note that this is not just a TCP layer load-balancer - there are plenty of solutions that'll examine the host part of the URL to figure out which back-end server to forward the query too. You can easily do it with Apache running on a low-spec server with suitable configuration.

The proxy ensures that each user's session always goes to the right back-end server and most any session handling methods will just keep on working.

Also the proxy needn't be a single point of failure. It's perfectly possible and pretty easy to run two or more front-end proxies in a redundant configuration (to avoid failure) or even to have them share the load (to avoid stress).

I'd also second John Sheehan's suggestion that the application just look at the left-hand part of the URL to determine which user's content to display.

If using Apache for the back-end, see this post too for info about how to configure it.