Hello all,
I found an error log file that is 20GB in terms of size!
I wanted to know if it will be slow for Apache to write to this?
I ask because the way I understand it is that Apache opens the error_log file upon start and writes to it, which shouldn't be that slow - Is this true?
Or will this sort of file size reduce speed of logging errors?
Thanks all
Appending to a 20GB file costs the same amount as appending to a 0B file, so it won't slow Apache down.
It will be a pain to process that file, have you considered rotating your logs hourly?
as far as i know he only appends the data so the size of the error log is of no immediate consequence, BUT what is a 20 GB Error log good for? You can't really find anything in it, i recommend you take a look at logrotate http://www.linuxcommand.org/man_pages/logrotate8.html
Apache opens the log files when it starts up (and closes/reopens them when you send a HUP), and then keeps them open for the life of the server. As such, there's a one-time file opening and seeking cost, after which data is simply appended to the log.
Even if it was doing an open/seek/write/close operation, seeking to the end of a file is a relatively lightweight operation. Doing this hundreds or thousands of times a second (for a busy site) would start getting expensive.
20gig of error data, however, does seem excessive. That's a LOT of 404's and whatnot. Depending on how long you've had this log kicking around, you might want to go in with a machete and start weeding out some of the more obvious errors (like many repeated failed requests for robots.txt or favicon.ico returning 404s).
You can use Apache's own "rotatelogs" to automatically rotate things for you, or use the system's own log cleanup facilities.