pbsdsh

PBS scripts provide a way to combine calculations into single jobs. The pbsdsh command is used to have specific cores execute shell scripts or programs within a multi-core job.

When run without option -c or -n, pbsdsh will spawn the program on all vnodes allocated to the job. With the “-c ” option, the script is spawned copies times. With the “-n ”, the script is spawned on the node_index-th vnodes allocated. With the -o option, no obit request is made for spawned tasks and the pbsdsh will not wait for the tasks to finish.

Double dash must come after the pbsdsh options and before the program and its arguments.

Each shell script begins execution from the user's home directory, and each shell script needs to include any appropriate change directory commands. Each shell script begins execution with only the PBS and default settings, so each shell script needs to contain the appropriate environment settings or/and the module load commands for any further software modules required for its calculations.

PBS script example using pbsdsh command:

#!/bin/bash
#PBS -q main
#PBS -j oe
#PBS -l select=5:ncpus=1:mem=1600mb:mpiprocs=1:ompthreads=1
#PBS -l walltime=00:03:00
#PBS -r y
echo $PBS_O_WORKDIR
cd ${PBS_O_WORKDIR}
#
echo $PBS_NODEFILE
cat $PBS_NODEFILE
echo
#
pbsdsh -v -- $PBS_O_WORKDIR/myscript.sh
wait
#
for proc in `seq 0 4`
do
pbsdsh -o -v -n $proc -- $PBS_O_WORKDIR/myscript$proc.sh
done
wait
#
exit 0

With “-o” option or with background launching (&), do not forget to wait until the end of tasks.