Converting to Multi-Threaded Socket Application

Question

As I am currently doing this project in only C, I've up untill this point only used my webserver as a single threaded application. However, I dont want that anymore! So I have the following code that handles my Work.

void BeginListen()
{
        CreateSocket();

        BindSocket();

        ListenOnSocket();

        while ( 1 )
        {
      ProcessConnections();
        }
}

Now I've added fork(); before the start of ProcessConnection(); which helpes me allowing multiple connections! However, when I add code for daemoning the application found in this answer. I've encounted a little problem, using fork() will create a copy of my whole running app, which is the purpose of fork(). So, I'd like to solve this problem.

My ProcessConnection() looks like this

void ProcessConnections()
{
     fork();

        addr_size = sizeof(connector);

     connecting_socket = accept(current_socket, (struct sockaddr *)&connector, &addr_size);

        if ( connecting_socket < 0 )
        {
                perror("Accepting sockets");
                exit(-1);
        }

     HandleCurrentConnection(connecting_socket);


        DisposeCurrentConnection();
}

How would I do to simply just add a couple of lines above or after connecting=socket = accept... in order to make it accept more than one connection at the time? Can i use fork(); but when it comes down to DisposeCurrentConnection(); I want to kill that process and just have the parent-thread running.

Answer 1

Check the return value of fork(). If it is zero, you are the child process, and you can exit() after doing your work. If it is a positive number then it's the process ID of the newly created process. This can let you kill() the child processes if they are hanging around too long for some reason.

Answer 2

I'm not a 100% sure what it is that you're trying to do, buy off the top of my head, I'd prefer to do the fork after the accept, and simply exit() when you're done. Keep in mind though, that you need to react to the SIGCHLD signal when the child process exits, otherwise you'll have a ton of zombie-processes hanging around, waiting to deliver their exit-status to the parent process. C-pseudo-code:

for (;;) {
  connecting_socket = accept(server_socket);
  if (connecting_socket < 0)
    {
      if (errno == EINTR)
        continue;
      else
        {
          // handle error
          break;
        }
    }

  if (! (child_pid = fork ()))
    {
       // child process, do work with connecting socket
       exit (0);
    }
  else if (child_pid > 0)
    {
      // parent process, keep track of child_pid if necessary.
    }
  else
    {
      // fork failed, unable to service request, send 503 or equivalent.
    }
}

The child_pid is needed to (as already mentioned) to kill the child-process, but also if you wish to use waitpid to collect the exit status.

Concerning the zombie-processes, if you're not interested in what happened to the process, you could install a signal hander for SIGCHLD and just loop on waitpid with -1 until it there are no more child-processes, like this

while (-1 != waitpid (-1, NULL, WNOHANG))
  /* no loop body */ ;

The waitpid function will return the pid of the child that exited, so if you wish you can correlate this to some other information about the connection (if you did keep track of the pid). Keep in mind that accept will probably exit with errno set to EINTR, without a valid connection if a SIGCHLD is caught, so remember to check for this on accepts return.

EDIT:
Don't forget to check for error conditions, i.e. fork returns -1.

Answer 3

Talking about fork() and threads on unix is not strictly correct. Fork creates a whole new process, which has no shared address space with the parent.

I think you are trying to achieve a process-per-request model, much like a traditional unix web server such as NCSA httpd or Apache 1.x, or possibly build a multi-threaded server with shared global memory:

Process-per-request servers:

When you call fork(), the system creates a clone of the parent process, including file descriptiors. This means that you can accept the socket request and then fork. The child process has the socket request, which it can reply to and then terminate.

This is relatively efficient on unix, as the memory of the process is not physically copied - the pages are shared between the process. The system uses a mechanism called copy-on-write to make copies on a page-by-page basis when the child process writes to memory. Thus, the overhead of a process-per-request server on unix is not that great, and many systems use this architecture.

Answer 4

As per my comment, this server is not really multi-threaded, it is multi-process.

If you want a simple way to make it accept multiple connections (and you don't care too much about performance) then you can make it work with inetd. This leaves the work of spawning the processes and being a daemon to inetd, and you just need to write a program that handles and processes a single connection. edit: or if this is a programming exercise for you, you could grab the source of inetd and see how it does it

You can also do what you want to do without either threads or new processes, using select.

Here's an article that explains how to use select (pretty low overhead compared to fork or threads - here's an example of a lightweight web server written this way)

Also if you're not wedded to doing this in C, and C++ is OK, you might consider porting your code to use ACE. That is also a good place to look for design patterns of how to do this as I believe it supports pretty much any connection handling model and is very portable.