what is more efficient: read in file from java or shell script?

Question

I'm trying to figure out what is more efficient in terms of server load which is pretty big at the moment, additional overload wouldn't be a great idea. Here is what I need to do :

I have a log file which changes, sometimes every second other times every few minutes or so which is not really relevant to this question. I'm trying to find out whether it is more efficient to start up java file with a cron job or to write shell script which will be executed by cron also, this is all under linux. Which is the better idea?

Answer 1

Checking log files are mostly I/O anyway so the actual CPU time in both cases are negligeable here. So what matters is the startup time, and spawning a shell script in Linux is magnitudes faster than starting up the JVM.

Answer 2

In general Perl is considered to be best for text parsing, but since all you need to do is print the changes to console you should simply do a tail -f rather than cron jobs etc.

Answer 3

As Peter said, it's dependent on what the program is supposed to do.

Generally, starting a Java program has quite a bit of overhead in comparison to a shell script. If some complex operations have to be done, however, java may well be your best bet.

I personally would choose Python for scripts for which shell is not really suited and for which Java may be overkill :)

Als be aware that system administrators can often read and understand shell scripts quite well, but Java is a different matter. That may be a problem, or maybe not.

Answer 4

there are many tools in *nix that parses files efficiently. eg grep,tail. These tools are coded in C with very efficient algorithms for parsing files. Definitely go for these shell tools. No Java please. Firstly, starting it up is slow. You can't compare it with running a C program like grep. Secondly, you will find it troublesome (in terms of compilation) to troubleshoot your script if anything goes wrong.

Answer 5

If you have to spawn n^2 processes for n lines of log files, you have to optimize even beyond bash. If you want to spawn only a sinlge process, language will not make a difference, imho.

I would rather check out in which language I/O and text processing is better for your needs. If a simple text processing in Java is too much on a "server", doing the same in assembly will be too much as well.