Socket Programming, Java, Tomcat 6, Scaling

Question

I'm pretty new to web programming and I'm currently developing a web back end for a mobile application. Currently I have the users log in using servlet interactions and once they have full access to the application I need to open a Socket Connection so that I can provide server pushes. Now the problem I'm running into is how people handle thousands of concurrent socket connections. I've run into people talking about ThreadPools which seems pretty easy to implement and NIO. Is there some framework that I can work with to ensure my servers are handling at least 20-30k concurrent connections. I could also forget TCP connections and go for Long-polling but from my understanding TCP is best option resource wise.

@Steve - I'm looking at the former: One serversocket with thousands of connections.

Answer 1

I would look into clustering the web end immediately and use that as your primary scaling mechanism. 30k connections is quite a lot and you don't have much room for growth before you hit a server limit of some kind. If the I/O itself isn't onerous I would just use lots of threads and servers with lots of horsepower and memory. Get it working that way so you can ship, and have a fallback plan to switch to multiplexed NIO if performance or scaling becomes a problem, but be warned that it's a radical overhaul and about ten times as complex to program as java.net. After several years' consideration I am more and more wondering whether NIO to economize on threads is really worth it: it adds several new problems of its own such as a need for push parsing; synchronization issues with the selector if there are worker threads that need to change the registration state of channels; lots of ways to get the code wrong; and the fact that the scheduling overhead moves out of the OS into your application, where you only have linear set-iterator data structures to deal with it unless you engage in yet another level of complexity. It's worth remembering that select() was invented for Unix to allow economizing on processes, which are expensive. Threads are pretty cheap really, and provide a very simple programming model with built-in context for handling a single connection. NIO barely manages this at all except via disciplined use of selection key attachments, much less naturally.