When running performance tests file system cache hit or miss can significantly influence test results. Therefore generally before running such tests used files are evicted from system cache. How to do that on Linux?
Clarification: If possible, the solution should not require root privileges.
As a superuser you can do the following:
To free pagecache:
To free dentries and inodes:
To free pagecache, dentries and inodes:
This operation will not "lose" any data (caches are written out to disk before their data is dropped), however, to really make sure all cache is cleaned, you should sync first. E.g. all caches should be cleared if you run
sync; echo 3 > /proc/sys/vm/drop_caches
As I said, only a superuser (root) may do so.
This is a problem (with superuser privileges). Is there any other solution not requiring this? Maybe reading file with O_DIRECT and writing same data back? Does it make sense?
Regarding use of O_DIRECT: that would perturb the results in another way. The kernel will attempt to DMA the filesystem data directly into your read() buffer, so it can be handed up to your application without any additional copy being done. Without O_DIRECT the kernel DMAs the file data into the page cache, and copies it from the page cache to your read() buffer.
This is fine if your app is really going to use O_DIRECT in production. If you run performance tests with O_DIRECT and then remove O_DIRECT for production, your performance test will be unrealistic.
Ha, I have the answer:
#include <unistd.h>
#include <fcntl.h>
int main(int argc, char *argv[]) {
int fd;
fd = open(argv[1], O_RDONLY);
fdatasync(fd);
posix_fadvise(fd, 0,0,POSIX_FADV_DONTNEED);
close(fd);
return 0;
}
This is from http://insights.oetiker.ch/linux/fadvise.html
If you can put the test data in a separate filesystem then mounting the filesystem afresh for the test will give you empty caches.
If you list the test fileystem in /etc/fstab with the "user" option then you can mount it for the test without being superuser
There is a command line utility by Eric Wong that makes it easy to invoke posix_fadvise:
http://git.bogomips.org/cgit/pcu.git/tree/README
It's then as simple as
$ pcu-fadvise -a dontneed filename-to-evict