views:

204

answers:

4

I will use the Linux NTFS driver as an example.

The Linux kernel NTFS driver only has very limited write support in the kernel, and after 5 years it is still considered experimental.

The same development team creates the ntfsmount userspace driver, which has almost perfect write support.

Likewise, the NTFS-3G project which is written by a different team also has almost perfect write support.

Why has the kernel drive taken so much longer? Is it much harder to develop for?

Saying that there already exists a decent userspace application is not a reason why the kernel driver is not compelte.

NOTE: Do not migrate this to superuser.com. I want a programing heavy answer, from a programming perspective, not a practical use answer. If the question is not appropriate for SO, please advise me as to why so I can edit it so it is.

A: 

The kernel's code must meet a much higher security and safety(i.e. bug-free) standard than user-mode code.

DeadMG
This is true, dunno why it's voted down...
Longpoke
agreed, kinda hard to tell when someone considers a down-vote a self evident critique...
msw
Probably voted down because the kernel ntfs driver is worse than the userspace driver.
Brendan Long
+4  A: 

I don't know the inside stories of the NTFS linux driver development but I can imagine some things that would make the userspace development go faster than the kernelspace one:

  • simpler API - the higher level of abstraction provided by userland libraries (memory management, for example) certainly eases the development
  • easier debuging - it's easier to debug a userspace process than debuging the kernel
  • process isolation - this is the one thing I would say is responsible for a faster development in the userspace. Your userspace filesystem driver doing bad things will result, in the worst case, in a corrupted filesystem and your driver process dying, while in kernelspace it could result in a complete system crash. This leads to faster debug cycles, which results in bugs being dealt with faster and faster development overall.
goedson
A: 

You have a lot more goodies available to you in userspace, so it's easier to develop software. And if the filesystem is sitting off in userspace, it's much harder for a fault in the filesystem to bring down the entire kernel.

This principle was taken to extreme in projects like Mach and Windows NT, in which the system was designed with a microkernel, and as many services as possible were put in userspace.

Norman Ramsey
Worth noting, unless very well thought out, IPC calls between services become a major headache to debug and performance bottleneck. GNU HURD, for instance, discovered this only after getting married to Mach, a mistake that they are now working to correct.
Tim Post
While architecturally NT is a microkernel, it is not often considered once because a lot of services which are not strictly part of the kernel are still in kernel mode for performance reasons (most notably the window manager for example)
Stewart
+2  A: 

Its important to note, especially with Linux that experimental can also mean "Code that someone dumped here, which at the time looked acceptable but may not be actively maintained".

I am a big fan of keeping file systems in user space, but I should also indicate that I am a big microkernel enthusiast. I find it practical and preferable to keep file systems in userspace for the following reasons:

Userspace File Systems Are Easier To Maintain

Take a moment to look at the ext3cow file system, a PHD project that gained a considerable amount of traction in a very short time. Its author graduated and then moved on to a career, with very little time to work on the file system. Because its out of tree, Linux's ever shifting internals between versions require anyone wanting to use it on a modern kernel to have in depth knowledge that not many possess.

If it used the FUSE API, it would be much easier to maintain, and the actual work that went into transforming ext3 into a copy on write file system would get more exposure. This also relates to in-kernel code that is gathering mold because nobody is brave enough (or bored enough) to touch it.

Userspace File Systems Are Easier To Debug

In userspace, you have awesome tools like Valgrind (and its friends like massif) which are invaluable tools and easy to use. The learning curve associated with kernel debugging is often too great for many people to just jump in and code. Note, I am making a clear separation of FUSE and micro kernel architecture, as noted in this answer. Some micro kernel based systems are extremely hard to debug, mostly due to races in communication between running services (vfs, block device, file system, ipc). In both instances, the code is easier to debug because its out of the kernel, provided having it out of the kernel does not introduce strange complexities.

In any event, I'll take GDB and Valgrind over noisy printk() debugging any day, or trying to make sense out of the rather cryptic kernel debugging hooks that are present in Linux. I'll also enjoy the ability to use whatever debugging (or even garbage collecting) malloc() implementation that I choose. The same goes with my C library of choice, provided that it works with FUSE. I'm not downing Linux's kernel library, but I like my creature comforts.

Userspace File Systems Are Easier To Use

Its a great benefit for underprivileged users to be able to mount and maintain whatever file system they want to use, but that is actually the end-game. If your file system is out of the kernel, it can advance independently of the kernel, which means users can upgrade tuned to your release cycle. You could conceivably achieve 6 milestone releases in the time it takes for Linux to advance to the next release candidate. This also allows distributions and OEM vendors to get your FS out in the wild where it gets the testing that it needs faster than it would if it was a kernel module.

Norman Ramsey already described the reliability factor associated with file systems as a service in a micro kernel architecture. However, reliability means not needing a reincarnation service that tends to hide (or put off) bugs and other issues. I do agree on the point that its nice if a failed root mount does not throw a kernel abort, but this is possible with monolithic FUSE enabled kernels as well.

In summary, writing a file system is hard enough without having to deal with running in kernel space. I'd much rather use the FUSE API, or study the IPC / VFS service implementation in a micro kernel based OS than write it as a kernel module.

Tim Post