What's the best way to get to know linux or BSD kernel internals?

Question

I'd like to gain better knowledge of operating system internals. Process management, memory management, and stuff like that.
I was thinking of learning by getting to know either linux or BSD kernel.
Which one kernel is better for learning purposes?
What's the best place to start?
Can you recommend any good books?

Answer 1

• Linux Device Drivers
• Linux Core Kernel Commentary
• Operating Systems Design and Implementation

I had previously bought these books on recommendation for the same purpose but I never got to studying them myself so only take them as second-hand advice.

Answer 2

As a Linux user I'd say Linux has a great community for people to learn about the kernel. http://kernelnewbies.org is a great place to start asking questions and learning about how the kernel works. I can't make a book reccomendation, but once you've read the starting material on kernelnewbies the source is very well documented.

Answer 3

In college, I had an operating systems class where we used a book by Tanenbaum. In the class, we implemented a device driver in the Minix operating system. It was a lot of fun, and we learned a lot.

One thing to note though, if you pick Minix, it is designed for learning. It is a microkernel, while Linux and BSD are a monolithic kernel, so what you learn may not be 100% translatable to be able to work with Linux or BSD, but you can still gain a lot out of it, without having to process quite as much information.

As a side note, if you've read Just for Fun, Linus actually was playing with Minix before he wrote Linux, but it just wasn't enough for his purposes.

Answer 4

Aside from the good books already mentioned (Opeating System Design & Implementation is particularly good), get a hold of a 1.x release Linux Kernel, load it into VMWare or VirtualBox and start playing around from there.

You will need to spend a lot of time browsing source code. For this, check out http://lxr.linux.no/ which is a browsable linked version of the source and makes life a lot easier. For the very first version of Linux (0.01) check out http://lxr.linux.no/linux-old+v0.01/. The fun begins at http://lxr.linux.no/linux-old+v0.01/boot/boot.s. As you progress from version to version, check out the ChangeLog and dig into those parts that have changed to save you re-reading the whole thing again.

Once you've gotten a hold of the concepts, look at 2.0, then 2.2, etc. Be prepared to sink A LOT of time into the process.

Answer 5

When I was at uni I spent a semester studying operating systems, and as part of this had an assignment where we had to implement a RAM-based filesystem in Linux.

It was a fantastic way to get to understand the internals of the Linux keurnel and to get a grasp on how everything fits together - And a heck of a lot of fun playing around with how it interacts with standard tools too.

Answer 6

There's no substitute for diving into the code. Try to find a driver or subsystem that you're interested in and poke around with it. With tools like VMware Workstation it's super easy to make whatever changes you want, snapshot the VM, and run your modified kernel. If the kernel panics on boot, who cares? Just jump back to the snapshot and fix the problem.

For books, I strongly recommend Linux Kernel Development by Robert Love. It's a wonderfully written book -- lots of information, organized sanely, and humorous... not dry reading at all.

Answer 7

I haven't tried it myself, but you can go to Linux From Scratch and start building your own Linux distribution. Sounds like something that'll take a junkload of time, but will result in an intimate knowledge of the guts of the Linux kernel and how each part works. Of course, you can supplement this learning by following any of the other tips here.

Answer 8

Noting the lack of BSDs here, I figured I'd chip in:

• The Design and Implementation of the FreeBSD Operating System (dead-tree book)
• Unix and BSD Courses (courses and videos)
• FreeBSD Architecture Handbook (online book)

I haven't taken any of the courses myself, but I've heard Marshall Kirk McKusick speak on other occasions, and he is really good at what he does.

And of course the BSD man pages, which are an excellent resource as they are maintained to a far greater extent than your average Linux man-page. Take for instance the uvm(9) man-page, describing the virtual memory interface in OpenBSD.

Not quite related, but I'll also recommend the video History of the Berkeley Software Distributions as it gives a nice introduction to the BSD parts of the UNIX history and culture as well as plenty of hilarious anectodes from back when.

Answer 9

Take Mike Stone's advice and start with Minix. That's what Linus did! The textbook is really well written, and Tannenbaum does a great job of showing how the various features are implemented in a real system.

Answer 10

Nobody seems to have mentioned that code-wise BSD is much cleaner and more consistent. The documentation's way better too (as already mentioned). But since there's a whole lot of fiddling with whatever system you choose - I'd pick the one you use more often.

Answer 11

I recommend you the BSD kernels! BSD kernels have far fewer hackers so following their evolution is easier. Either BSD and Linux kernels have great hackers, but some people argue that BSD lower fame filters out novice ones. Also taking design decisions is easier when the sources are not being updated 100 times a day.

Among the BSD choices, my favorite one is NetBSD. It might not be the pain-free choice you want for your desktop, but because it has a strong focus on portability, the quality is quite good. I think this part say it all:

Some systems seem to have the philosophy of “If it works, it's right”. In that light NetBSD's philosophy could be described as “It doesn't work unless it's right”

If you have been working long enough, you will know that NetBSD is a quite joy for learning good coding. Although professionally you will find more chances with Linux

Whichever choice you take, start joining their mail lists, follow the discussions. Study some patches and finally try to do your own bug-fixing. Regarding books, search for Diomidis Spinellis articles and his book. It is not exactly a kernel book, but has NetBSD examples and helps a lot to tackle large software.