how to redirect complete output of a script

Question

I have a simple cronjob running every day at 18:35:

05 18 * * * ~/job.sh 2>&1 >> ~/job.log

So the output of ~/job.sh should be written into ~/job.log. In job.sh, there are some echo commands and a few python scripts are executed, e.g.:

echo 'doing xyz'
python doXYZ.py

Now, whatever output the python scripts produce, they are not written into ~/job.log. I only see the echo text in ~/job.log. How can I redirect the complete output of the shell script to ~/job.log?

Answer 1

Have you tried ~/job.sh &>> ~/job.log ?

Answer 2

You need to redirect the output of the python script. Learning Python 2nd Edition recommends the following:

import sys
sys.stdout = open('log.txt', 'a')   # Redirects output to file
...
print x, y, x                       # shows up in log.txt

and goes on to say:

"Here, we reset sys.stdout to a maually-opened output file object opened in append mode. After the reset, every print statement anywhere in the program will write its text to the end of file log.txt. instead of the original output stream."

Answer 3

Arkaitz has the simplest solution. However, to see what's wrong with your snippet we need to go into the bash manual:

Note that the order of redirections is significant. For example, the command

  ls > dirlist 2>&1

directs both standard output and standard error to the file dirlist, while the command

 ls 2>&1 > dirlist

directs only the standard output to file dirlist, because the standard error was duplicated from the standard output before the standard out‐ put was redirected to dirlist.

So apparently the output redirection only redirects to the target of the other stream, not to the other stream itself. When bash parses your command line it will come upon the 2>&1 clause and redirect stderr to the target of stdout, which at this point is still the console (or whatever is attached to cron's stdout). Only after this will stdout be redirected.

So what you really want is:

05 18 * * * ~/job.sh >>~/job.log 2>&1

Answer 4

I'm quite sure that this should work in a general case. The stdout file descriptor is changed by the OS when you use shell redirection.

However, if your Python script itself directly writes to the screen (perhaps by opening /dev/tty), it won't get captured in your log.txt. Is that the case? Does it fail even for a simple python program that just does a

print "Hello"

?