How does a non-forking web server work?

Question

Non-forking (aka single-threaded or select()-based) webservers like lighttpd or nginx are gaining in popularity more and more.

While there is a multitude of documents explaining forking servers (at various levels of detail), documentation for non-forking servers is sparse.

I am looking for a bird eyes view of how a non-forking web server works. (Pseudo-)code or a state machine diagram, stripped down to the bare minimum, would be great.

I am aware of the following resources and found them helpful.

• The World of SELECT()
• thttpd source code
• Lighttpd internal states

However, I am interested in the principles, not implementation details.

Specifically:

• Why is this type of server sometimes called non-blocking, when select() essentially blocks?

• Processing of a request can take some time. What happens with new requests during this time when there is no specific listener thread or process? Is the request processing somehow interrupted or time sliced?

Edit: As I understand it, while a request is processed (e.g file read or CGI script run) the server cannot accept new connections. Wouldn't this mean that such a server could miss a lot of new connections if a CGI script runs for, let's say, 2 seconds or so?

Answer 1

Basic pseudocode:

setup
while true
    select/poll/kqueue
    with fd needing action do
        read/write fd
        if fd was read and well formed request in buffer
            service request
        other stuff

• Though select() & friends block, socket I/O is not blocking. You're only blocked until you have something fun to do.
• Processing individual requests normally involved reading a file descriptor from a file (static resource) or process (dynamic resource) and then writing to the socket. This can be done handily without keeping much state.
• So service request above typically means opening a file, adding it to the list for select, and noting that stuff read from there goes out to a certain socket. Substitute FastCGI for file when appropriate.

EDIT:

• Not sure about the others, but nginx has 2 processes: a master and a worker. The master does the listening and then feeds the accepted connection to the worker for processing.

Answer 2

select() PLUS nonblocking I/O essentially allows you to manage/respond to multiple connections as they come in a single thread (multiplexing), versus having multiple threads/processes handle one socket each. The goal is to minimize the ratio of server footprint to number of connections.

It is efficient because this single thread takes advantage of the high level of active socket connections required to reach saturation (since we can do nonblocking I/O to multiple file descriptors).

The rationale is that it takes very little time to acknowledge bytes are available, interpret them, then decide on the appropriate bytes to put on the output stream. The actual I/O work is handled without blocking this server thread.

This type of server is always waiting for a connection, by blocking on select(). Once it gets one, it handles the connection, then revisits the select() in an infinite loop. In the simplest case, this server thread does NOT block any other time besides when it is setting up the I/O.

If there is a second connection that comes in, it will be handled the next time the server gets to select(). At this point, the first connection could still be receiving, and we can start sending to the second connection, from the very same server thread. This is the goal.

Search for "multiplexing network sockets" for additional resources.

Or try Unix Network Programming by Stevens, Fenner, Rudoff