Interrupt handling and transferring data from and to application code - low latency method theories

Question

I have a Fibre Optic link, with a proprietary Device Driver and message format.
The link goes into a PCIe card. Running on a RHEL 5.2 (2.6.18-128~)
I have mmap'ed the interface on the card for setup and FIFO access etc, but cannot use this for interrupts as this should be done via the kernel-space interrupt routine (correct me if required!)

If the provided driver was assumed to be "the best" then would be the expected access times between the interrupt occurring and the User Space API interrupt call wait method returning?

WaitForInterrupt(); 
// Interrupt occurs    <-- time to get to here

I am seeing times of around 500 milliseconds microseconds!?
Whereas I have the mmaped access down to < 10 microseconds!

Could/is this a case of bad DeviceDriver and or the user space library API implementation?

Or does switching between the kernel space and user space take this sort of time?

Answer 1

500ms is many orders of magnitudes larger than what a simple switch between userspace/kernel takes, but as someone mentioned in comments, linux is not a real time OS, so there's no guarantee 500ms "hickups" won't show up now and then.

It's quite impossible to tell what the culprit is, the device driver could simpliy be trying to bundle up data to be more efficient.

That said, we've had endless troubles with some custom cards and interactions with both APIC and ACPI, requireing a delicate balance of bios settings, what card goes into which PCI slot and whether a particular video card screws up everything - likely a cause of a dubious driver interacting with more or less buggy bios/video-cards..

Answer 2

If you have a recent kernel, you can use the perf sched tool to measure the latency, and see where the time is being used. (500us does sound a tad on the high side, depending on your processor, how many tasks are running, ...)

Answer 3

If you're able to reliably exceed 500us on a system that's not heavily loaded, I think you're looking at a bad driver implementation (or its userspace wrapper/counterpart).

In my experience the latency to wake a user thread on interrupt should be less than 10us, though (as others have said) Linux provides no latency guarantees.