Can't reach speeds of dd

Question

Hi,

I'm writing C code with some real-time constraints. I tested out the speed I can write to a disk with dd:

dd if=/dev/zero of=/dev/sdb bs=32K count=32768 oflag=direct

This writes 1GB of zeros to /dev/sdb in 32K block sizes

I reach about 103 MB/s with this

Now I programmatically do something similar:

open("/dev/sdb",O_WRONLY|O_CREAT|O_DIRECT|O_TRUNC, 0666);

I get a timestamp value write from a 32K buffer to /dev/sdb 10,000 times (in a for loop) get another timestamp value do a bit of number crunching to get the rate in MB/s and it is about 49 MB/s

Why can't I reach the same speed as dd? An strace reveals the same open command that I use.

Answer 1

Check what system calls dd makes, not just the open but also the subsequent reads and writes. Using the right buffer sizes can make a significant difference in this kind of large copy. Note that /dev/zero is not a good test for benchmarking if your final goal is a disk-to-disk copy.

If you can't match dd's speed by matching it system call for system call... well, read the source.

Answer 2

I'm leaving the part about matching the system calls to somebody else. This answer is about the buffering part.

Try benchmarking the buffer size you use. Experiment with a range of values.

When learning Java, I wrote a simple clone of 'copy' and then tried to match it's speed. Since the code did byte-by-byte read/writes the buffer size was what really made the difference. I wasn't buffering it myself but I was asking the read to fetch chunks of a given size. The bigger the chunk, the faster it went - up to a point.

As for using 32K block size, remember that the OS still uses separate IO buffers for user-mode processes. Even if you are doing something with specific hardware, i.e. you're writing a driver for a device that has some physical limitation, e.g. a CD-RW drive with sector sizes, the block size is only part of the story. The OS will still have it's buffer too.