I have an application that runs slow. This is because of a huge amount of loggings at DEBUG and INFO levels inside the code. I have made some modifications in the code and changed the log level to WARN and it works well now.
But there is only one log file (currently at 1.6GB). I want to use a RollingFileAppender to have more, smaller, files. What is the best (maximum) size that I should use for the appender’s MaxFileSize property so that performance won’t degrade?
That really depends on many factors so to answer the question, you'd have to run a profiler with various file sizes. But since log4j only writes to the log file, you can simply create files of various sizes on your system and time how long it takes.
To be able to find errors in the file, I suggest to use a DailyRollingFileAppender, though. This will make it much more simple to look for something "yesterday" or "two weeks ago".
Having smaller chunks will make your log files more manageable, but it will not improve performance.
It seems that your limitation is the hdd. One solution to the performance issue would be to have WARN level logged in one file, and DEBUG and INFO in another. Ideally, you would have this larger file kept on a dedicated fast hdd.
Another solution to the performance issue would be to tweak logging to different package. You rarely need INFO on all packages, because parsing 2 GB of data would be hard to do, especially in real time.
Answer to the smallest size question:
It should be as large as your tools can handle without trouble. Let's say that you would use a log viewer to watch the log file. Some log viewers will perform badly on files, let's say, bigger than 10 MB. But, again, on 1 GB of log data generated in ... 1 hour maybe, you won't be able to watch it in real time.