HP XC System Software User's Guide Version 3.2

The srun command, used by the mpirun command to launch the MPI tasks in parallel in the
lsf partition, determines the number of tasks to launch from the SLURM_NPROCS environment
variable that was set by LSF-HPC; this environment variable is equivalent to the number provided
by the -n option of the bsub command.
Any additional SLURM srun options are job specific, not allocation-specific.
The mpi-jobname is the executable file to be run. The mpi-jobname must be compiled with
the appropriate HP-MPI compilation utility. For more information, see the section titled Compiling
applications in the HP-MPI User's Guide.
Example 5-7 shows an MPI job that runs a hello world program on 4 cores on 2 compute
nodes.
Example 5-7 Submitting an MPI Job
$ bsub -n4 -I mpirun -srun ./hello_world
Job <24> is submitted to default queue <normal>.
<<Waiting for dispatch ...>>
<<Starting on lsfhost.localdomain>>
Hello world!
Hello world! I'm 1 of 4 on host1
Hello world! I'm 3 of 4 on host2
Hello world! I'm 0 of 4 on host1
Hello world! I'm 2 of 4 on host2
You can use the LSF-SLURM External Scheduler option to add capabilities at the job level and
queue level by including several SLURM options in the command line. For example, you can
use it to submit a job to run one task per node, or to submit a job to run on specific nodes.
“LSF-SLURM External Scheduler” discusses this option.
Example 5-8 shows an MPI job that uses the LSF-SLURM External Scheduler option to run the
same hello world program on each of 4 compute nodes.
Example 5-8 Submitting an MPI Job with the LSF-SLURM External Scheduler Option
$ bsub -n4 -ext "SLURM[nodes=4]" -I mpirun -srun ./hello_world
Job <27> is submitted to default queue <normal>.
<<Waiting for dispatch ...>>
<<Starting on lsfhost.localdomain>>
Hello world!
Hello world! I'm 1 of 4 on host1
Hello world! I'm 2 of 4 on host2
Hello world! I'm 3 of 4 on host3
Hello world! I'm 4 of 4 on host4
Some preprocessing may need to be done:
• If the MPI job requires the use of an appfile
• If there is another reason that prohibits the use of the srun command as the task launcher
This preprocessing should determine the node host names to which mpirun's standard task
launcher should launch the tasks. In such scenarios, you need to write a batch script; there are
several methods available for determining the nodes in an allocation. One method is to use the
SLURM_JOBID environment variable with the squeue command to query the nodes. Another
method is to use the LSF-HPC environment variables such as LSB_HOSTS and LSB_MCPU_HOSTS,
which are prepared by the HP XC job starter script.
5.3.3 Submitting a Parallel Job Using the SLURM External Scheduler
The LSF-SLURM external scheduler option provides additional capabilities at the job level and
queue level by allowing the inclusion of several SLURM options in the LSF-HPC command line.
5.3 Submitting a Parallel Job 57