HP-MPI Version 1.
© Copyright 1979-2008 © Hewlett-Packard Development Company, L.P Legal Notices The information in this document is subject to change without notice. Hewlett-Packard makes no warranty of any kind with regard to this manual, including, but not limited to, the implied warranties of merchantability and fitness for a particular purpose.
Table of Contents 1 HP-MPI V1.2 for Windows Release Note.....................................................................7 1.1 Announcement..................................................................................................................................7 1.2 What’s in This Version.......................................................................................................................7 1.2.1 Platforms Supported.................................................................
List of Figures 1-1 4 Job Allocation................................................................................................................................
List of Tables 1-1 1-2 1-3 1-4 Directory Structure..........................................................................................................................7 mpicc Utility..................................................................................................................................31 mpif90 Utility................................................................................................................................32 Manpage Categories............................
1 HP-MPI V1.2 for Windows Release Note 1.1 Announcement This release note describes the release of HP-MPI Version 1.2 for Windows®. HP-MPI V1.2 is the November 2008 release of HP-MPI, the HP fully functional implementation of the Message Passing Interface standard for Windows. HP-MPI V1.2 for Windows is supported on servers and workstations running Windows HPC Server 2008 (HPCS). 1.2 What’s in This Version HP-MPI is a high-performance implementation of the Message Passing Interface standard.
Table 1-1 Directory Structure (continued) Subdirectory Contents lib HP-MPI libraries licenses Repository for HP-MPI license file man HP-MPI manpages in HTML format sbin HP-MPI Remote Launch service 1.2.3 Benefits and Features MPI has become the industry standard for distributed technical applications and is widely supported on most technical computing platforms. HP-MPI offers several benefits.
— — -mpif90 -mpisyntax The following HP-MPI HP-UX/Linux features are NOT available in this HP-MPI Windows release: • Support for MPI-2 dynamic processes • mpiexec, mpiclean, mpirun.all • MPICH compatibility mode • Deferred deregistration of memory on RDMA networks • -l < user> option (change user ID for job execution) 1.2.4 Description of Features The following section provides brief descriptions of the new features included in this release.
Uses block scheduling to place ranks on allocated nodes. Nodes are processed in the order they were allocated by the scheduler, with each node being fully populated up to the total number of CPUs before moving on to the next node. Only valid when the -ccp option is used. Cannot be used with the -f, -hostfile, or -hostlist options. Specifies the headnode of the CCP cluster that should be -ccpcluster used to run the job. Assumed to be the local host if omitted.
-nopass -iscached -clearcache is required to allow the remote process to access network resources (such as file shares). The password provided is encrypted using SSPI for authentication. The password is cached so that future mpirun commands uses the cached password. Passwords are cached in encrypted form, using Windows Encryption APIs. Executes the mpirun command with no password. If a password is cached, it is not accessed and no password is used to create the remote processes.
-kill Authenticates with remote service and terminates the remote process indicated by the pid. The process is terminated as the authenticated user. So, if the user does not have permission to terminate the indicated process, the process will not be terminated. mpidiag authentication options are the same as mpirun authentication options. These include: -pass, -cache, -clearcache, -iscached, -token/-tg, -package/-pk.
headnode> clusrun /all mkdir "C:\Program Files (x86)\Hewlett-Packard\HP-MPI\licenses" To copy the license file to all nodes, enter the following on one command line: headnode> clusrun /all copy "\\netshare\path\mpi.lic" "C:\Program Files (x86)\Hewlett-Packard\HP-MPI\licenses" 1.3.2.1 Importing the HP-MPI Job Template to the Head Node A new feature for HPCS is the ability to schedule jobs using a generic job template. To schedule jobs using the HP-MPI Job Template, import the template to your head node.
/tasks="RegisterService,!RegisterService\StartService" During the installation, this option registers the HP-MPI Remote Launch service, but does not start the service. To start the service select Administration Tools→Services. Highlight HP-MPI Remote Launch, and select Start. 1.3.3.1 Remote Launch Service Information The HP-MPI Remote Launch service is located in "%MPI_ROOT%\sbin\ HPMPIWin32Service.exe". MPI_ROOT must be located on a local disk or the service does not run properly.
-restart restart an installed service NOTE: All remote services must use the same port. If you are not using the default port, make sure you select a port that is available on all remote nodes. 1.3.4 Upgrade Instructions If HP-MPI is already installed, uninstall the existing version. This can be done with the clusrun command by running the uninstall executable in MPI_ROOT with the /verysilent flag. The uninstall executable name is generally the same on all nodes.
1.4.1 Setting environment variables Environment variables can be used to control and customize the behavior of an HP-MPI application. The environment variables that affect the behavior of HP-MPI at run time are described in the mpienv(1) manpage. In all run modes, HP-MPI enables environment variables to be set on the command line with the -e option.
1.5 Running HP-MPI from HPCS There are two ways to run HP-MPI under HPCS: command line and scheduler GUI. Both ways can be used to access the functionality of the scheduler. The command line scheduler options are similar to the GUI options. The following instructions are in the context of the GUI, but equivalent command line options are also listed.
1.6 Running HP-MPI on Windows 2008 To run HP-MPI on Windows 2008 systems, use the appfile mode or the -hostlist/-hostfile flags. For remote processes to have access to network resources (such as file shares), a password, which is used to create processes on the remote nodes, must be provided. The password is SSPI encrypted before being sent across the network. Passwords are provided using the -pass or -cache flags. Authentication does not require a password, but remote access to network resources does.
Below are some useful Windows 2008 environment variables for naming the job name or stdout/err fileshare: • CCP_CLUSTER_NAME - Cluster name • CCP_JOBID - Job ID • CCP_JOBNAME - Job name • CCP_TASKCONTEXT - Task ‘content’ (jobid.taskid) • CCP_TASKID - Task ID An example job description file for a saved job, XMLDescriptionFile.xml, is included in the help directory. This contains a template for a single saved HP-MPI job. To use this description file, submit a job by selecting the HPC Job Manager.
IMPORTANT: In the Job Resources section of the New Job window, you must select Core and you must provide Minimum and Maximum resource types. 4. 20 Select Task List (from the left menu list) then left-click Add (on the right menu list) to create a new task within the job. HP-MPI V1.
5. In the above example, the following line has been added into the Command line: by selecting the text box and entering: "%MPI_ROOT%\bin\mpirun.exe" -ccp -netaddr 172.16.150.0 -TCP \\node\share\pallas.exe 6. Specify stdout, stderr, and stdin (if necessary). In the above example, the stderr and stdout files are specified using HPCS environment variables defined by the job. This is an easy way to create output files unique for each task. \\node\share\%CCP_JOBNAME%-%CCP_TASKCONTEXT%.out 7.
8. To change task properties such as Resources and Environment Variables, highlight the task in the New Job window, and change the task properties in the lower portion of the window. IMPORTANT: Set the Exclusive entry to True so the job manager will schedule the MPI job tasks evenly across the processes allocated to the job. 9. 22 To restrict the run to a set of machines, select the nodes in the Resource Selection window. HP-MPI V1.
NOTE: This step is not necessary. The job will select from any available processors if this step is not done. NOTE: For convenience, Job Description Files can be created and saved by clicking Save Job as .... 10. To run the job, click Submit. NOTE: To have some of the HP-MPI job parameters correctly set for you automatically, select the HP-MPI job template on the Job Details screen. This properly selects various job properties and simplifies entering an HP-MPI job via the GUI.
For example: > job new /jobname:[example job]/numprocessors:12 /projectname:HPMPI /exclusive:true Job Queued, ID: 242 This will create a job resource and return a jobid, but not submit it. > job add 242 /stdout:"\\node\share\%CCP_JOBNAME%-%CCPTASKCONTEXT%.out" /stderr:"\\node\share\%CCP_JOBNAME%-%CCPTASKCONTEXT%.err" /exclusive:true "%MPI_ROOT%\bin\mpirun" -ccp -prot -netaddr 172.16.150.20/24 -TCP \\node\share\rank.exe -arg1 -arg2 > job submit /id:242 1.7.
C:\> Documents and Settings\smith>s:S:\> cd smithS:\smith> "%MPI_ROOT%\bin\mpirun.exe" -ccp -np 6 -hostlist mpiccp1,mpiccp2 HelloWorld.exe mpirun: Submitting job to scheduler and exiting Submitting job to ccp scheduler on this node mpirun: HPMPI Job 1117 submitted to cluster mpiccp1 Here the S: drive is interpreted as the mapped network drive. The rank HelloWorld.exe is located in the current directory, and the stdout and stderr files are placed in the current working directory.
NOTE: Specify the bitness using -mpi64 or -mpi32 for mpicc to link in the correct libraries. Verify you are in the correct 'bitness' compiler window. Using -mpi64 in a Visual Studio 32-bit command window does not work. 3. Run the hello_world executable file: > "%MPI_ROOT%\bin\mpirun" -np 4 hello_world.exe where -np 4 specifies 4 as the number of processors to run. 4. Analyze hello_world output. HP-MPI prints the output from running the hello_world executable in non-deterministic order.
6. Submit the job. The machine resources are allocated and the job is run. >job submit /id:4288 To run Multiple-Program Multiple-Data (MPMD) applications or other more complex configurations that require further control over the application layout or environment, dynamically create an appfile within the job using the utility "%MPI_ROOT%\bin\mpi_nodes.exe" as in the following example.
(HPMPI.vsprops for 32-bit applications, HPMPI64.vsprops for 64-bit applications) in Configuration→Platform. Select this page by either double-clicking the page or by right-clicking on the page and selecting Properties. Go to the User Macros section. Set MPI_ROOT to the desired location (for example, the installation location of HP-MPI). This should be set to the default installation location: %ProgramFiles (x86)%\Hewlett-Packard\HP-MPI NOTE: This is the default location on 64-bit machines.
1.8.6 Running with an Appfile Using HPCS Using an appfile with HPCS has been greatly simplified in this release of HP-MPI. The previous method of writing a submission script that uses mpi_nodes.exe to dynamically generate an appfile based on the HPCS allocation is still supported. However, the preferred method is to allow mpirun.
Hello world! I'm 0 of 4 on n01 Hello world! I'm 2 of 4 on n02 1.8.8 Running with a Hostfile Using HPCS Perform Steps 1 and 2 from “Building and Running on a Single Host” (page 25). 1. 2. Change to a writable directory on a mapped drive. The mapped drive must be to a shared folder for the cluster. Create a file "hostfile" containing the list of nodes on which to run: n01 n02 n03 n04 3. Submit the job to HPCS. X:\demo> "%MPI_ROOT%\bin\mpirun" -ccp -hostlist hostfile -np 8 hello_world.
1.9 C Command-Line Basics The utility "%MPI_ROOT%\bin\mpicc" is included to aid in command line compilation. To compile with this utility, set MPI_CC to the path of the command line compiler you want to use. Specify -mpi32 or -mpi64 to indicate if you are compiling a 32- or 64-bit application. Specify the command line options that you normally pass to the compiler on the mpicc command line. The mpicc utility adds additional command line options for HP-MPI include directories and libraries.
The mpif90 utility adds additional command line options for HP-MPI include directories and libraries. The -show option can be specified to mpif90 to display the command generated without executing the compilation command. See the manpage mpif90(1) for more information. To construct the desired compilation command, the mpif90 utility needs to know what command line compiler is to be used, the bitness of the executable that compiler will produce, and the syntax accepted by the compiler.
http://www.docs.hp.com/en/highperfcomp.html 1.12 Known Problems and Workarounds • The Fortran interface provided in HP-MPI assumes a style calling convention of C-byreference. That is, function names are not decorated as _function@bytes, arguments are passed by reference, string length arguments are passed by value after the other arguments, and the caller cleans the stack.
If the license needs to be placed in another location which cannot be found by the previous search, the user can set the environment variable LM_LICENSE_FILE to explicitly specify the location of the license file. For more information, see http://licensing.hp.com. 1.14.1.1 Installing License Files A valid license file contains the system hostid and the associated license key. License files can be named either as license.dat or any name with extension of *.lic (like mpi.lic, for example).
Table 1-4 Manpage Categories Category Manpages Description General MPI.1 Describes the general features of HP-MPI Compilation mpicc.1 mpif90.1 Describes the available compilation utilities Runtime mpidebug.1 mpienv.1 mpimtsafe.1 mpirun.1 mpistdio.1 autodbl.1 Describes runtime utilities, environment variables, debugging, and thread-safe and diagnostic libraries. 1.16 Software Availability in Native Languages There is no information on non-English languages for HP-MPI for Windows systems. 1.
