How much is File I/O a performance factor in web development?

Question

I know the mantra is that the database is always the long pole in the tent anytime a page is being generated server-side.

But there's also a good bit of file i/o going on on a web server. Scripted code is replete with include/require statements. Moreover, its typically a practice to store templated html outside the application in files which are loaded and filled in accordingly.

How much of a role does file i/o play when concened with web development? Does it ever become an issue? When is it too much? Do web servers/languages cache anything?

Has it ever really mattered in your experience?

Answer 1

I would say that file IO speed only becomes an issue if you are serving tons of static content. When you are processing data, and executing code to render the pages, the time to read the page itself from disk is negligible. File I/O is important in cases where the static files you are serving up are unable to fit into memory, such as when you are serving video or image files. It could also happen with html files, but since the size is so small with html files, this is less likely.

Answer 2

10 years ago, disks were so much faster than processors that you didn't have to worry about it so much. You'd run out of CPU (or saturate your NIC) before disk became an issue. Nowadays, CPUs and gigabit NICs could make disk the bottleneck, BUT....

Most non-database disk usage is so easily parallelizable. If you haven't designed your hosting architecture to scale horizontally by adding more systems, that's more important than fine-tuning disk access.

If you have designed to scale horizontally, usually just buying more servers is cheaper than trying to figure out how to optimize disk. Not to mention, things like SSD or even RAM disks for your templates will make it a non-issue.

It's very rare to have a serving architecture that scales horizontally, popular enough to cause scalability problems, but not profitable enough to afford another 1u in your rack.

Answer 3

File I/O will only become a factor (for static content and static page includes) if your bandwidth to the outside world is similar to your disk bandwidth. This would imply either you have a really fast connection, are serving content on a fast LAN, or have really slow disks (or are having a lot of disk contention). So most likely the answer is no.

Of course, this assumes that you are not loading a large file only for a small portion of the file.

Answer 4

File I/O is one of many factors, including bandwidth, network connectivity, memory etc, which may affect the performance of a web application. The most effective way to determine if file I/O is causing you any issues is to run some profiling on the server and see if this represents is the bounding factor on your performance.

A lot of this will depend upon what types of files you are loading from disk, lots of small files will have very different properties to a few large files. Web servers can cache files, both internally in memory and can indicate to a client that a file (e.g. an image) can be cached and so does not need to be requested every time.

Answer 5

Do not prematurely optimize. Its evil, or something.

However, I/O is about the slowest thing you can do on a computer. Try to keep it at a minimum, but don't let Knuth see what you're doing.