Is there a good description of the "system call" mechanism used in OSes?

Question

I am looking for a good primer or technical description of the System Call mechanism that is used by operating systems to transition from user space to the kernel to invoke functions such as "open", "read", "write", etc...

Is there anything other than the Wikipedia entry? Websites, pdfs, books, source code, all are welcome :)

Answer 1

Well for source code, there are plenty of open source kernels to dive into.

As for books, Robert Love's book on the Linux kernel is very informative.

Answer 2

You may want to have a look at the minix kernel. It's open source, designed to be simple, and is used in a lot of Uni-level OS courses. Have a dig around in /usr/src/kernel/proc.c especially the sys_call function and surrounding functionality. Keep in mind that minix is a microkernel, so some things may be subtlety different to what you are used to.

Answer 3

If you want to purchase a book that is extremely useful for *nix programming. I would recommend "Advanced Programming in the UNIX Environment" by Stevens and Rago. It has in depth explanations, and code examples.

Answer 4

The exact method depends on the processor architecture and what operations it defines for transferring to kernel mode. One approach, and the traditional one on x86, was to use a software interrupt. It turns out this wasn't very fast for the general case so later, Intel added SYSCALL and AMD added SYSENTER. Windows XP and later choose an appropriate system call technique for the platform, at boot time.

You could choose to use specific software interrupt numbers for specific functions, but generally the processor doesn't have enough interrupts to cover all the system functions, so it's necessary to make one of the registers contain the function number required. If you're doing that anyway, it's not much of a hardship to only use the one system call function.

Windows CE, before version 6.0, uses a side-by-side process virtual address model that actually allows processes to call into each other directly. The page protections are set up so that when this is done, an access violation fault occurs: the kernel gets control, fixes up the process address space (moving the called process into slot 0), fixes up slot-0-based arguments to point to the calling process, and returns to user mode. Because the return address is in another process, when the function call returns, the reverse process occurs. Unfortunately this model only allows very small virtual address spaces for each process (32MB) and a low number of processes (32), so Windows CE 6.0 reverts to a more traditional system call model.

Answer 5

For a good explanation of system calls in Linux, look at the sample device drivers in Linux Device Drivers.

Answer 6

It's architecture dependent, and requires an understanding of computer architecture. Tanenbaum's "Structured Computer Organisation" has a good summary of the basics of a system call. For more, read any textbook on operating system design.