Avoiding sleep while holding a spinlock

Question

Hello!

I've recently read section 5.5.2 (Spinlocks and Atomic Context) of LDDv3 book:

Avoiding sleep while holding a lock can be more difficult; many kernel functions can sleep, and this behavior is not always well documented. Copying data to or from user space is an obvious example: the required user-space page may need to be swapped in from the disk before the copy can proceed, and that operation clearly requires a sleep. Just about any operation that must allocate memory can sleep; kmalloc can decide to give up the processor, and wait for more memory to become available unless it is explicitly told not to. Sleeps can happen in surprising places; writing code that will execute under a spinlock requires paying attention to every function that you call.

It's clear to me that spinlocks must always be held for the minimum time possible and I think that it's relatively easy to write correct spinlock-using code from scratch.

Suppose, however, that we have a big project where spinlocks are widely used. How can we make sure that functions called from critical sections protected by spinlocks will never sleep?

Thanks in advance!

Answer 1

What about enabling "Sleep-inside-spinlock checking" for your kernel ? It is usually found under Kernel Debugging when you run make config. You might also try to duplicate its behavior in your code.

Answer 2

One thing I noticed on a lot of projects is people seem to misuse spinlocks, they get used instead of the other locking primitives that should have be used.

A linux spinlock only exists in multiprocessor builds (in single process builds the spinlock preprocessor defines are empty) spinlocks are for short duration locks on a multi processor platform.

If code fails to aquire a spinlock it just spins the processor until the lock is free. So either another process running on a different processor must free the lock or possibly it could be freed by an interrupt handler but the wait event mechanism is much better way of waiting on an interrupt.

The irqsave spinlock primitive is a tidy way of disabling/ enabling interrupts so a driver can lock out an interrupt handler but this should only be held for long enough for the process to update some variables shared with an interrupt handler, if you disable interupts you are not going to be scheduled.

If you need to lock out an interrupt handler use a spinlock with irqsave.

For general kernel locking you should be using mutex/semaphore api which will sleep on the lock if they need to.

To lock against code running in other processes use muxtex/semaphore To lock against code running in an interrupt context use irq save/restore or spinlock_irq save/restore

To lock against code running on other processors then use spinlocks and avoid holding the lock for long.

I hope this helps