Cross Compiling Linux Kernels and Debugging via VMware

Question

I'm considering doing some Linux kernel and device driver development under a vmware VM for testing ( Ubuntu 9.04 as a guest under vmware server 2.0 ) while doing the compiles on the Ubuntu 8.04 host.

I don't want to take the performance hit of doing the compiles under the VM.

I know that the kernel obviously doesn't link to anything outside itself so there shouldn't be any problems in that regard, but

1. are there any special gotcha's I need to watch out for when doing this?
2. beyond still having a running computer when the kernel crashes are there any other benefits to this setup?
3. Are there any guides to using this kind of setup?

Edit

I've seen numerous references to remote debugging in VMware via Workstation 6.0 using GDB on the host. Does anyone know if this works with any of the free versions of VMWare such as Server 2.0.

Answer 1

I'm not sure about ubuntu thing. Given that you are not doing a real cross compilation (i.e. x86->arm), I would consider using make-kpkg package. This should produce an installable .deb archive with kernel for your system. this would work for me on debian, it might for for you on ubuntu.

more about make-kpkg: http://www.debianhelp.co.uk/kernel2.6.htm

1. I'm not aware of any gotchas. But basically it depends what kind of kernel part you are working with. The more special HW/driver you need, the more likely VM won't work for you.

2. probably faster boots and my favorite is the possibility to take screenshot (cut'n'paste) of panic message.

3. try to browse to vmware communities. this thread looks very promising, although it dicusses topic for MacOS: http://communities.vmware.com/thread/185781

Answer 2

Compiling, editing, compiling is quite quick anyway, you don't recompile you whole kernel each time you modify the driver.

Before crashing, you can have deadlock, bad usage of resource that leads to unremovable module, memory leak etc ... All kind of things that needs a reboot even if your machine did not crash, so yes, this can be a good idea.

The gotchas can come in the form of the install step and module dependency generation, since you don't want to install your driver in the host, but in the target machine.