Only disk writes but iotop shows reads as well

Question

Hi,

I'd like to understand the following issue:

A process is doing write sys call only, in an infinite loop. When I bring up iotop I would expect to see non-zero write speed and zeroed read speed related to that process. But iotop tells read and write can be equal (depending on single write size). Have a look at the C code:

#include <sys/types.h>
#include <sys/stat.h>
#include <unistd.h>
#include <fcntl.h>
#include <stdio.h>
#define BUFSIZE 1000000

char buf[BUFSIZE];
const int write_size = 4000;

int main(){
  int fd;
  if ((fd = open("filename", O_RDWR | O_CREAT, 0666)) < 0){
    return -1;
  }
  ssize_t ret;
  while (1){
    ret = write(fd, buf, write_size);
    if (ret != write_size){
      return -1;
    }
  }
  return 0;
}

If you assign different values to 'write_size' you'll see different read speeds in iotop. If the value is as in the code above, iotop shows read and write are equal.

Important:
The issue appears only under certain conditions:
- The file must be created and filled with data (lets say at least 8GB) before running the code

OS conf:
Debian lenny, 2TB disk, (both xfs and ext4 tested), uname -a Linux g-6 2.6.26-bpo.1-xen-amd64 #1 SMP Mon Jan 12 14:32:40 UTC 2009 x86_64 GNU/Linux

Thanks in advance for solving the mystery,
Michal.

Answer 1

You are opening an existing file, with data. You are overwriting it (no O_APPEND). Thus, when writing X data, a filesystem needs to get the content (= block), put your writing over it, then push back the block on the disk.

Answer 2

You open the file in overwrite mode but you write in chunks that aren't a multiple of the block size on disk (4000 instead of 4096). So every now and then, the process will flush buffers and it will need to read a block from disk to make up for the gap.

Add O_TRUNC to make the mystery go away. Note that the process will now hang during startup because it takes some time to delete 8GB.

Answer 3

Try to open the file with O_WRONLY | O_CREAT, if you want to open it just for writing. It may solve the issue - and looks reasonable when you say that the file must contain data before you call open().