Dear all, I have the following way to submit a job with cluster using qsub: Submitting jobs from standard input To submit a PBS job by typing job specifications at the command line, the user types qsub [options] <return> then types any directives, then any tasks, followed by (in UNIX) C...
Dear all, I have no problem using the following command of AWK as a stand alone command, without any error: $ awk '$9 != "NTM" && $9 != ""' myfile.txt | less -Sn But when I apply them inside Perl's script for qsub (i.e. running job in linux cluster) command, like this: use strict; use Data::Dumper; use Carp; use File::Basename; my...
I have a 64-node cluster, running PBS Pro. If I submit many hundreds of jobs, I can get 64 running at once. This is great, except when all 64 jobs happen to be nearly I/O bound, and are reading/writing to the same disk. In such cases, I'd like to be able to still submit all the jobs, but have a max of (say) 10 jobs running at a given ...
I'm calling several unix commands and python on a python script from a qsub shell script, meant to run on a cluster. The trouble is that when the script executes, something seems to go awry in the shell, so that directories and files that exist are not found. For example, in the .out output files of qsub I see the following errors: cd...
I'm currently working on a cluster using the ClusterVisionOS 3.1. This will be my first time working with a cluster, so i probably haven't tried the "obvious". I can submit a single job to the cluster with the "qsub" command(this i got working properly) But the problem starts when submitting multiple jobs at once. I could write a script...
I would like to submit jobs via qsub on Sun Grid Engine (now: Oracle Grid Engine?). I do not wish to use the -sync yes option or qrsh, because I want my controlling program to be single-threaded and able to launch many jobs at a time. These options would block my controlling program's thread. However, I would like to receive the exit ...
I have a function (neural network model) which produces figures. I wish to test several parameters, methods and different inputs (meaning hundreds of runs of the function) from python using PBS on a standard cluster with Torque. Note: I tried parallelpython, ipython and such and was never completely satisfied, since I want something sim...
Hello how can I parametrize a shell script that is executed on a grid (started with qsub) ? I have a shell script, where I use getopts to read the parameters. When I start (qsub script.sh -r firstparam -s secondparam ..) this working script with qsub I receive error messages, qsub: invalid option -- s qsub: illegal -r value ...
I have access to a 128-core cluster on which I would like to run a parallelised job. The cluster uses Sun GridEngine and my program is written to run using Parallel Python, numpy, scipy on Python 2.5.8. Running the job on a single node (4-cores) yields an ~3.5x improvement over a single core. I would now like to take this to the next lev...
I would like to run a script when all of the jobs that I have sent to a server are done. for example, I send ssh server "for i in config*; do qsub ./run 1 $i; done" And I get back a list of the jobs that were started. I would like to automatically start another script on the server to process the output from these jobs once all are c...