Hybrid MPI/OpenMP jobs

PBS script:

#PBS -q large
#PBS -l walltime=1:00:00
#PBS -l select=5:ncpus=24:mem=63000mb:mpiprocs=2:ompthreads=12
#PBS -W group_list=PRACE_T1FWB
…

 

This script requests:

  • the queue large
  • a 1 hour wall-time
  • 5 chunks with 24 cpus, 63000mb of memory, 2 MPI tasks per chunk and 12 threads per MPI task.
  • OMP_NUM_THREADS is set to 12 by PBSpro on MPI rank 0 process and has to be exported according to MPI implementation on other MPI processes.
  • It is a PRACE_T1FWB project.
  • The MPI machine file can have the following form:
    node0340
    node0340
    node0346
    node0346
    node0431
    node0431
    node0425
    node0425