Hadoop job fails when invoked by cron

Question

I have created the following shell script for invoking a hadoop job:

#!/bin/bash
/opt/hadoop/bin/hadoop jar /path/to/job.jar com.do.something <param-1> ... <param-n> &
wait %1
STATUS=$?
if [ $STATUS -eq 0 ]
then    
    echo "SUCCESS" | mailx -s "Status: "$STATUS -r "[email protected]" "[email protected]"
    exit $STATUS
else
    echo "FAILED" | mailx -s "Status: "$STATUS -r "[email protected]" "[email protected]"
    exit $STATUS
fi

When I run the above script manually like this:

$ ./path/to/job.sh

Hadoop job executes sucessfully and returns an exit status "0".

Now, to automate the job execution everyday, I have configured a cron job to run the above script like this:

0 22 * * * /path/to/job.sh

But, now job is not submitted to Hadoop and I get a exit status "1".

Few things to note here:

• The user account under which cron job is configured is UserA
• UserA is also Hadoop System User
• The cluster is dedicated for running this job
• The script is executable

I would like to know why the job is not running when cron invokes it ?

Answer 1

the env of running from cron could be different from your regular shell. You may want to check that, e.g. JAVA_HOME, PATH etc.