Lazy sockets - scalability?

Question

Asking more in theory, how could I build a server (or app) that uses lazy sockets? I'm envisioning a web app that moves all of its data via JSON exchanges to a central API-like servlet. I could leave the HTTP connection open for a bit after transferring all data, to write more to the client, as a sort of lazy push technology. The browser could also reconnect after a timeout closed the socket.

Interrogating that model, how would I write the app in a way that doesn't consume truckloads of memory by exponential thread branching? Would each thread handle one or multiple connections? How would each discover new data to transmit? -- presumably I would need dialogue between threads, rather than each thread actively search for data on its own? If I wrote a parent thread that spawned x+1 child threads, would each thread be 1:1 or many:1 with clients? Could I run into deadlocking issues? What's the memory footprint of the each additional connection handler?

Does anyone have thoughts on this? I'm wondering more what this would look like in theory than practice.

Answer 1

You could open several sockets and use select() or poll() to let the operating system find out which of them need work done. This way you only need one thread to handle an arbitrary number of sockets. Reads/writes are done non-blocking only when the operating system signals that there is data/buffer space available.

Answer 2

I'd suggest you take a look at Twisted. It uses a lightweight approach with a single process handling the async IO, and then multiple worker processes. As workers can block on file and database calls, it's easier to keep them synchronous in threaded workers. The bulk of the open sockets can then be handled by a single parent thread using async IO.