Jobs Array

PBSpro provides job arrays for running collections of almost-identical jobs. Job arrays are useful where you want to run the same program over and over on different input files. PBS can process a job array more efficiently than it can the same number of individual jobs.

Each job in a job array is called a “subjob”. All subjobs in a job array share a single job script, including PBS directives and the shell script portion. All subjobs have the same attributes, including resource requirements and limits.

The job script may invoke different commands based on the subjob index. The commands may be scripts themselves.
Job arrays are required to be rerunnable.
Subjobs must satisfy individually the limits set on queues.

Submitting a Job array :

qsub -J <index start>-<index end>[:stepping factor] ...

Environmental variables for Job Arrays:

PBS_ARRAY_INDEX	Used for subjobs	Subjob index in job array, e.g. 7
PBS_ARRAY_ID	Used for subjobs	Identifier for a job arrays. Sequence number of job array, e.g. 5222371[].frontal1
PBS_JOBID	Used for jobs, subjobs	Identifier for a job or a subjob. For subjob, sequence number and subjob index in brackets, e.g. 5222371[2].frontal1

Example:

#!/bin/csh
#PBS -q main
#PBS -j oe
#PBS -l select=1:ncpus=1:mem=1600mb:mpiprocs=1:ompthreads=1
#PBS -l walltime=00:30:00 #PBS -r y
echo $PBS_O_WORKDIR
cd ${PBS_O_WORKDIR}
if ( ! ($?wk) ) then
    echo "ERROR: wk variable must be defined <master|slave>"
    exit
endif
if ( $wk == "master" ) then
    echo I am the master process, I prepare the data and launch 4 workers
master_application
    qsub -J 1-4 -v wk=slave $0
else if ( $wk == "slave" ) then
    echo I am the slave process number ${PBS_ARRAY_INDEX}
    slave_application_${PBS_ARRAY_INDEX} < input.file_${PBS_ARRAY_INDEX}
else
    echo "ERROR : Unknown worker $wk"
   exit
endif
exit 0

This can be launched with the command:

qsub -v wk=master <script_name>