views:

155

answers:

3

I have used the select() function to implement servers, but now I have to implement it by myself (as a part of a project).

int select(int nfds, fd_set *readfds, fd_set *writefds, fd_set *exceptfds, struct timeval *timeout);

Can anyone please help me to implement this function? Any sources, articles? Can anyone explain what is happening in that function?

A: 

You can always look at open source implementations for inspiration and/or theory of operation.

I believe the basic premise is that the process calling select() is put on "wait lists" connected to each of the sockets in the request, and then the kernel can check whenever it goes to update the status of a socket which processe(s) are on that socket's lists, and make them runnable.

This is of course a very high-level description though that just glosses over many of the myriad details you'll need to cover to get this right.

unwind
Thank you for the correcting me with my post.. :) And thanks for the input
Morpheus
+1  A: 

select is an operating system primitive. It cannot be implemented using other portable constructs, such as pthreads.

You can think of it as a differently-specialized version of pthread_cond_wait, which receives a signal associated with a condition variable. pthread_cond_wait stalls the current thread until the signal is received, then (optionally) verifies that the signal was associated with the appropriate mutex, then acquires the mutex, and continues along. select receives a signal, confirms (or at least guarantees) its receipt, and passes the information along. It also deals with signals that were received before it was called.

So, pthread_cond_wait is not really primitive in terms of implementation; it's designed to be very safe semantically. That is the spirit of pthreads, for better or worse. Atomic variables, available in various libraries, provide an unsafe primitive-implementation alternative to pthreads for synchronization jobs, but signaling involves scheduling and unscheduling threads, which is pretty platform specific. (Well, on second thought, I guess you could implement select with spinlocks. I shudder at the thought.)

It's a great exercise to try, though.

Potatoswatter
"pthread_cond_wait is not really primitive in terms of implementation". It's pretty darn close, though: http://birrell.org/andrew/papers/ImplementingCVs.pdf
Steve Jessop
@Steve: It's a lot bigger than "unschedule this thread." Even that pseudocode is written in high, or at least usable-level constructs.
Potatoswatter
@Potatoswatter: Oh, I see. I was taking "primitive" from the point of view of the C programmer - something is an OS primitive if it's an operation which can only be implemented by the OS. I guess you're taking it from the POV of the kernel programmer - something is a primitive if it doesn't call any other functions.
Steve Jessop
@Steve Jessop: It's a primitive in the sense that only the OS can truly implement it. It's inherently connected with the OS internals...
Matt Joiner
A: 

If you're implementing select, then presumably you (or your colleagues) are implementing a significant part of the OS interface. So, it depends how you're implementing the I/O subsystem, and the scheduler.

If you're not implementing the scheduler, then you need some kind of "control centre" which uses synchronisation primitives (probably condition variables) to track changes in the states of the file descriptors you're interested in.

After initially checking the current state of the file descriptors listed, fundamentally, how select works is that it doesn't return until the timeout expires, or one of the descriptors it cares about changes state (in a way that it cares about). This could be handled entirely in the scheduler, so that the thread is unavailable for scheduling until the required conditions occur, or the thread could block until some change occurs that it might be interested in, check whether it really is interested, and then either return if it is, or block again if it isn't. At the extreme of this approach, a completely dumb (and inefficient) implementation could have a single condition variable that every caller to select waits on, and which is broadcast by the I/O system every time a state change occurs (as Potatoswatter points out, you also have to record the state).

If you're not implementing the I/O subsystem, then the only way to implement select is on top of some other, similar primitive which your I/O subsystem does support. Alternatively, take on the task of implementing I/O: wrap everything in your own layer, and make select available only through that layer. The implementation of this layer depends on the underlying API, but if all else fails you can create one thread per file descriptor, do blocking I/O ops in that thread, and then notify your "control centre" when the fd is readable, writable, or has errors.

Get the design right first, which probably means studying how other OSes do it. Otherwise you will spend the next 2 years fixing race conditions.

Steve Jessop
Trying to implement select using condvars is a somewhat common pitfall. Signaling a condition variable does nothing if nobody is waiting on it, and there's no way to tell that anyone received the signal. You won't spend just 2 years fixing the races. The essential ingredient is the job queue implementing the condition of the variable. Adding a job requires exclusive access to the queue, and the whole point of `select` is that writes are non-blocking.
Potatoswatter
That's why you have to check the current state before waiting (and atomically with starting the wait, which is what condvars are for). Non-blocking I/O shouldn't be confused with taking locks - it's fine for a non-blocking I/O operation to briefly block on a lock protecting some structure, just as long as it doesn't block until actual data leaves/arrives (although you have to be more careful with such things in an RTOS). I'll take your word for it that there are problems with using condvars, but I'm not proposing *just* a condvar, of course one must also track the state of the fds.
Steve Jessop
(I didn't downvote you, just checking back on this page…) In that case, there is no difference between `select` and a plain work queue, which is not usually considered in the same class. `select` is important in NUMA programming and networks, where the data isn't coming from local threads. OP says he uses it for servers. A proxy thread for each network connection, all feeding a condvar-controlled queue per server thread, would certainly implement the same functionality. And on second thought, he hints that it might just be homework, so maybe that's exactly what he wants.
Potatoswatter
@Potatoswatter: sure, there's certainly a quality-of-implementation issue there: the simplest approach is rubbish because every incoming packet can trigger a mass of threads to wake. Perhaps my answer would have been more concise if I just said what you did: you *could* implement it like a work queue, but given details about your scheduler and your I/O system you'd *want* to do better. Really there are two questions here, "how do Unix OSes implement `select`", and "can anyone help me implement it?", and depending on the questioner's circumstances the answers might be completely different.
Steve Jessop
... so depending on those circumstances it's quite possible that my answer is indeed wrong.
Steve Jessop