is there a size limit to a text file?

Question

HI All

I am creating a log file for our website which will log every log-in by all the users to our orders area. I wish to know if you think its good to enter this log info in just one single file or should this be split up once the log file hits a certain size? My concern is that this file will get rather large over time, but im not certain of the size limit of a text file?

thank you

Answer 1

There is no direct limit in filesize and when just appending to the text file you won't notice if the file gets longer and longer. However when you try to read it, large files are slower than smaller files. You might want to split the file regularily then, or look at a database solution instead.

Answer 2

Depending on the traffic you can create a log file for every day, week or month.

Answer 3

Which operating system? In general, I'd expect that Windows and Unix both support at least 64-bit file lengths, i.e. there's no important theoretical limitation: the actual/real limitation is the amount of available space you have on disk.

Answer 4

There are no filesize limits but you should still split your files to make viewing them later much easier

Answer 5

There are limits, but that depends on:

1. The underlying file system (even if the kernel supports large files, the FS may not)
2. The capabilities of the underlying kernel's VFS layer (not all operating systems support large files, however most that can run web servers do).

Sometimes, you have facilities like logrotate which can automatically handle chunking log files. Other times you do not.

Would you want to parse and sort a 10GB log file? If the underlying OS has no facilities to chunk / archive the logs, you need to handle that in your own code.

Its as simple as naming your log file after the month-year, unless you are sure that something else will chunk your logs for you. To be portable, have your logging function check it out.

Note, even on systems that have facilities to rotate logs, often the log rotator has to be told that your logs exist. One huge file is, under any circumstances .. questionable.

Answer 6

Isn't there a limit to a text file in Windows? I presume there is as I remember doing a copy paste of some text into a Notepad(in Windows) over and over and notepad complained that it exceeds memory. Now is this a limitation of notepad or the OS itself.