script unable to find directories/files when running from qsub cluster script

Question

I'm calling several unix commands and python on a python script from a qsub shell script, meant to run on a cluster. The trouble is that when the script executes, something seems to go awry in the shell, so that directories and files that exist are not found. For example, in the .out output files of qsub I see the following errors:

cd: /valid/dir/name: No such file or directory
python valid/script/name.py 
python: can't open file 'valid/script/name.py': [Errno 2] No such file or directory

so the script cannot cd into a dir that definitely exist. Similarly, calling python on a python script that definitely exists yields an error.

any idea what might be going wrong here, or how I could try to debug this?

thanks very much.

Answer 1

the errors are pretty self explanatory. make sure /valid/dir/name is actually a directory. You can put this in your shell script to create this directory if not found.

if [ ! -d "/valid/dir/name" ];then
   mkdir -p /valid/dir/name
fi

As for Python error, because there's no "valid" directory, it gives you error.