views:

1521

answers:

9

I'm desperately looking for cheap ways to lower the build times on my home PC. I just read an article about disabling the Last Access Time attribute of a file on Windows XP, so that simple reads don't write anything back to disk.

It's really simple too. At a DOS-prompt write:

fsutil behavior set disablelastaccess 1

Has anyone ever tried it in the context of building C++ projects? Any drawbacks?

[Edit] More on the topic here.

+3  A: 

I haven't tried this on a Windows box (I will be tonight, thanks) but the similar thing on Linux (noatime option when mounting the drive) sped things up considerably.

I can't think of any uses where the last access time would be useful other than for auditing purposes and, even then, does Windows store the user that accessed it? I know Linux doesn't.

paxdiablo
+3  A: 

I'd suggest you try it and see if it makes a difference.

However I'm pessimistic about this actually making any difference, since in the larger/clean builds you'll be writing out large amounts of data anyway, so adjusting the file access times wouldn't take that much time (plus it'd probably be cached anyway).

I'd love to be proven wrong though.


Results:

Ran a few builds on the code base at work in both debug and release configurations with the last access time enabled, and disabled.

Our source code is about 39 MB (48 MB size on disk), and we build about half of that for the configuration that I built for these tests. The debug build generated 1.76 GB of temporary and output files, while the release generated about 600 MB of such data. We build on the command line using a combination of Ant and the Visual Studio command line built tools.

My machine is a Core 2 Duo 3GHz, with 4GB of ram, a 7200rpm hdd, running Windows XP 32 bit.

Building with the last access time disabled:

Debug times = 6:17, 5:41

Release times = 6:07, 6:06

Building with the last access time enabled:

Debug times = 6:00, 5:47

Release times = 6:19, 5:48

Overall I did not notice any difference between the two modes, as in both cases the files are most likely in the system cache already so it should just be reading from memory.

I believe that you'll get the biggest bang for your buck by just implementing proper precompiled headers (not the automatically generated ones that Visual Studio creates in a project). We implemented this a few years ago at work (when the code base was far smaller) and it cut down our build time to a third of what it was.

Daemin
Interesting! However, I would expect that the real gain is when touching a single file that has lots of includes, and then rebuild: the dependency checking should start looking for a whole bunch of file, which could trigger a write of the last access time. Maybe...
Carl Seleborg
A: 

it will improve the performance a little. Other than that it won't do much more (you won't be able to see when the file was last accessed of course). I have it turned of by default when I install windows XP using nLite to cut of the bloat I don't need.

Sven
+1  A: 

It's a good alternative, but it will affect some tools. Like the Remote Storage Service, and other utilies that depend on file access statistics to optimize your file system (i.e. Norton Defrag)

+7  A: 

From SetFileTime's documentation:

"NTFS delays updates to the last access time for a file by up to one hour after the last access."

There's no real point turning this off - the original article is wrong, the data is not written out on every access.

EDIT:

As to why the author of that article claimed a 10x speed-up, I think he attributed his speed-up to the wrong thing: he also disabled 8.3 filename generation. To generate an 8.3 filename for a file, NTFS has to basically generate each possibility in turn then see if it's already in use (no reference; I'm sure Raymond has talked about it but can't find a link). If your files all share the same first six characters, you will be bitten by this problem, and the corrolary is you should put characters which differentiate files in the first six characters so they don't clash. Turning off short name generation will prevent this.

Mike Dimmick
Good info. However, the author of the original article mentions a 10x speedup - there's got to be something that's slow with the last access time mechanism.
Carl Seleborg
A: 

I don't want to draw attention away from the "last access time" question, but there might be other ways to speed up your builds. Not knowing the context and your project setup, it's hard to say what might be slow, but there might be some things that might help:

Create "uber" builds. That is, create a single compilation uber.cpp file that contains a bunch of lines like

#include "file1.cpp"
#include "file2.cpp"

You might have trouble with conflicting static variable names, but those are generally easy to sort out. Initial setup is kind of a pain, but build times can increase dramatically. For us, the biggest drawback is that in developer studio, you can't right click a file and say 'compile' if that file is part of an uber build. It's not a big deal though. We have seperate build configurations for 'uber' builds which compile the uber files but exclude the individual cpp files from the build process. If you need more info, leave a comment and I can get you that. Also, the optimizer tends to do a slightly better job with uber builds.

Also, do you have a large number of include files, or a lot of depencendies between include files? If so, that will drastically slow down build times.

Are you using precompiled headers? If not, you might look into that as a solution as that will help as well.

Slow build times are usually tracked down to lots of file I/O. That is by far the biggest time sink in a build -- just opening, reading and parsing all of the files. If you cut down file I/O, you will improve build times.

Anyway, sorry to slightly derail the topic slightly, but the suggestion at hand to change how the last access time of a file is set seemed to be somewhat of a 'sledgehammer' solution.

Mark
Hi Mark, thanks for the suggestions. I know about the Überbuild trick, which doesn't quit cut it in my case. Actually, there are many, many things I can do to speed things up, but I want to start with those that will get me the most bang-for-the-buck.
Carl Seleborg
A: 

For busy servers, disabling last access time is generally a good idea. The only potential downside is if there are scripts that use last access time to, for instance, tell that a file is no longer being written.

That said, if you're looking to improve build times on a C++ project, I highly recommend reading Recursive Make Considered Harmful. The article is about a decade old, but the points it makes about how recursive definitions in our build scripts cause long build times is still well worth understanding.

A: 

Disabling access time is useful when using ssd's (solid state drives - cards,usb drives etc) as it reduces the number of writes to the drive. All solid state storage devices have a life which is measured by the number of writes that can be made to each individual address. Some media specify a minimum of 100's of thousands and some even 1 million. Operating systems and other executables can access many files in a single operation as well as user document accesses. This would apply to eee pc's, embedded systems and others.

A: 

To Mike Dimmick:

Try to connect USB drive with many files and copy them to your internal drive. That's also the case in addition to program compilation (which is described in original post).

mrd0