Jobs Array

PBSpro provides job arrays for running collections of almost-identical jobs. Job arrays are useful where you want to run the same program over and over on different input files. PBS can process a job array more efficiently than it can the same number of individual jobs.

Each job in a job array is called a “subjob”.  All subjobs in a job array share a single job script, including PBS directives and the shell script portion. All subjobs have the same attributes, including resource requirements and limits.

The job script may invoke different commands based on the subjob index. The commands may be scripts themselves.
Job arrays are required to be rerunnable.
Subjobs must satisfy individually the limits set on queues.

Submitting a Job array :

qsub -J <index start>-<index end>[:stepping factor] ...

Environmental variables for Job Arrays:

PBS_ARRAY_INDEX Used for subjobs Subjob index in job array, e.g. 7
Used for subjobs
Identifier for a job arrays. Sequence number of job array, e.g. 5222371[].frontal1
PBS_JOBID Used for jobs, subjobs Identifier for a job or a subjob. For subjob, sequence number and subjob index in brackets, e.g. 5222371[2].frontal1



#PBS -q main
#PBS -j oe
#PBS -l select=1:ncpus=1:mem=1600mb:mpiprocs=1:ompthreads=1
#PBS -l walltime=00:30:00 #PBS -r y
if ( ! ($?wk) ) then
    echo  "ERROR: wk variable must be defined <master|slave>"
if ( $wk == "master" ) then
    echo I am the master process, I prepare the data and launch 4 workers
    qsub -J 1-4 -v wk=slave $0
else if ( $wk == "slave" ) then
    echo I am the slave process number ${PBS_ARRAY_INDEX}
    slave_application_${PBS_ARRAY_INDEX} < input.file_${PBS_ARRAY_INDEX}
    echo "ERROR : Unknown worker $wk"
exit 0


This can be launched with the command:

qsub -v wk=master <script_name>