Is "the optimized delay" a myth or is it real?

Question

Hi

From time to time you hear stories that are meant to illustrate how good someone is at something, and sometimes you hear about the guy how is so into code optimization that he optimizes his delay loop.

Since this really sounds like it's a strange thing to do as it's much better to start a "timer interrupt" instead of a optimized buzy wait, and nobody ever tend to tells you the name of the optimizing hacker.

That has left me to wonder if it is a urban myth or is it real?

What do you say, reality or fiction?

Thanks Johan

---

Update: It sounds like ShuggyCoUk was on to something, wonder if we can find a example.

Update: Just a little clarification, this question is about the "delay" function it self and how that is implemented, not how and where you call it. And what that purpose was, and how that system became better.

Update: It's no myth, those guys seems to exist

Thanks ShuggyCoUk

Answer 1

The version I've always heard is of a group of hardware programmers who developed a special instruction that optimised the idle (not busy) loop of their operating system. This is mentioned in Kernighan & Pike's book The Practice Of Programming, but even there they admit it may be an Urban Myth.

Answer 2

I've heard stories of programmers who intentionally put in long delay loops early in projects and removed them later as "optimizations" to impress management. Never figured out if the stories were apocryphal or not.

Answer 3

This has more than a kernel of truth about it...

Spin wait can be much better than a signal based interrupt or a yield.

• You trade some throughput for much reduced latency.
  • Often this is vitally important within an OS itself.
• You allow yourself the freedom to do operations not possible within an interrupt handler
  • memory allocation for example.
• You can get considerably finer grained control of the interval waited since you can essentially measure the cycle count.

However spin waits are tricky to get right.

• If you can you should use use proper idle instructions which:
  • can power down parts of the core, improving power usage/heat dissipation and even allowing other cores to go faster.
  • In Hyper Thread based CPUs you allow the other logical thread to use the full CPU pipeline while you spin.
  • an instruction you might think was a no-op could cause the CPU to execute them out of order via the super scalar execution units. The resulting code may get unforeseen out of order artefacts which force the CPU to apply a great deal of effort in terms of stalls and memory barriers which are unwanted.

This is why you let someone else write the spin wait loop for you in most cases..

• In Linux there is the cpu_relax macro

  • on arm this is barrier()
  • on x86 this is rep_nop()

• In Windows there is YieldProcessor

  • Accessible in .Net via Thread.SpinWait

• OS X eschews providing a standard implementation unless you are in the kernel

  • see this document and note that it encourages the use only of lck_spin_t

As to some citations of using PAUSE for spin waits:

• PostGresSQL
• Linux
  • See also the note that this is better on non P4 as well due to reducing power