Performance of fopen() on win32

Question

I am trying to write some code that works on both Linux and Win32. The most noticeable difference I find between them (in my code) is performance of fopen().
Following code takes 5 sec on my Ubuntu and the same code takes more than 100 sec on windows XP. I would like to make a note here that ubuntu is VM while XP is on a real machine.

    time_t start = time(NULL);
    for(int i=0; i < 100000; ++i){
        FILE *fp = fopen("a.txt", "a");
        if (fp != NULL)
        {
            fprintf(fp, "Hello World");
            fclose(fp);
        }
    }
    time_t end = time(NULL);

    printf("\n It took %d seconds \n", end-start);

Clearly fopen() is the cause of this difference. I want to know why is it such a big difference?

Answer 1

Do you use a Virus Scanner? If yes disable it first!

And some API Calls are slower on windows. E.G. your C:\ will be translated to /harddrive/something first (just an example).

Answer 2

Clearly fopen() is the cause of this difference

No it's more likely to be filesystem flushing.
On one system when you write, or more likely call fclose(), it blocks until the bytes are physically on disk (or at least until the disk says they are) - on the other the filesystems returns straight away, even if the flies are still being written

Answer 3

This isn't relevant. This is not a normal usage scenario for I/O functions, so they don't have to be optimized for this case. Maybe windows uses an syncronous flush() while linux uses asyncronous.

Answer 4

There is a lot more than just the API being used here.

You are opening a file on the file system.
So the type of file system being used will affect time, as well as the hard ware speeds of the devices implementing the file system. There are just too many factors that you are not taking into account that you can accurately say X is the culprit for the slow speeds.

Answer 5

If you are using Visual C++, be aware that by default stdio now uses mutexes to enable safe multithreading. These can be turned off with #define _CRT_DISABLE_PERFCRIT_LOCKS. I'm not certain, but I think Linux stdio implementations usually assume single-threaded behaviour, so don't have this locking overhead.

More info here: http://msdn.microsoft.com/en-us/library/ms235505%28v=VS.80%29.aspx

Answer 6

I would do the following:

1. Try it on a real hardware machine similar spec to the intended target
2. Use the same machine on the Windows and Linux tests - dual boot it or something.
3. Disable all third party add-ons on the windows box, especially AV software (NB: If your company IT department does not allow this, explain to them that this is a software development lab machine and need not be subject to their policy if suitably separated from the main office network)

In all likelihood they are not doing the same thing. The filesystem is probably responsible. Either the win32 box is doing a lot more flushing - possibly because it has less ram available due to other tasks - or the Linux box is running on hardware which fakes flushing and introduces another level of caching - i.e. cheats.

What level of durability do you require in your application? If the files absolutely must not disappear on power failure (e.g. mail server receiving mail) then you should fsync() them. fclose() does not guarantee to do this