Probing for filesystem block size

Question

I'm going to first admit that this is for a class project, since it will be pretty obvious. We are supposed to do reads to probe for the block size of the filesystem. My problem is that the time taken to do this appears to be linearly increasing, with no steps like I would expect.

I am timing the read like this:

double startTime = getticks();
read = fread(x, 1, toRead, fp);
double endTime = getticks();

where getticks uses rdtsc instructions. I am afraid there is caching/prefetching that is causing the reads to not take time during the fread. I tried creating a random file between each execution my program, but that is not alleviating my problem.

What is the best way to accurately measure the time taken for a read from disk? I am pretty sure my block size is 4096, but how can I get data to support that?

Answer 1

The usual way of determining filesystem block size is to ask the filesystem what its blocksize is.

#include <sys/statvfs.h>
#include <stdio.h>
int main() {
    struct statvfs fs_stat;
    statvfs(".", &fs_stat);
    printf("%lu\n", fs_stat.f_bsize);
}

But if you really want, open(…,…|O_DIRECT) or posix_fadvise(…,…,…,POSIX_FADV_DONTNEED) will try to let you bypass the kernel's buffer cache (not guaranteed).

Answer 2

You may want to use the system calls (open(), read(), write(), ...) directly to reduce the impact of the buffering done by the FILE* stuff. Also, you may want to use synchronous I/O somehow. One ways is opening the file with the O_SYNC flag set (or O_DIRECT as per ephemient's reply). Quoting the Linux open(2) manual page:

   O_SYNC The file is opened for synchronous I/O.  Any  write(2)s  on  the
          resulting  file  descriptor will block the calling process until
          the data has been physically written to the underlying hardware.
          But see NOTES below.

Another options would be mounting the filesystem with -o sync (see mount(8)) or setting the S attribute on the file using the chattr(1) command.