True Scale Fabric OFED+ Host Software Release Notes February 2014 Order Number: H31512002US
INFORMATION IN THIS DOCUMENT IS PROVIDED IN CONNECTION WITH INTEL PRODUCTS. NO LICENSE, EXPRESS OR IMPLIED, BY ESTOPPEL OR OTHERWISE, TO ANY INTELLECTUAL PROPERTY RIGHTS IS GRANTED BY THIS DOCUMENT.
OFED+ Host SW Contents 1.0 Overview of the Release ............................................................................................5 1.1 Introduction .......................................................................................................5 1.2 Audience ............................................................................................................5 1.3 If You Need Help .................................................................................................
OFED+ Host SW True Scale Fabric OFED+ Host Software RN 7.2.2.0.
OFED+ Host SW 1.0 Overview of the Release 1.1 Introduction These Release Notes provide a brief overview of the changes introduced into the Intel® True Scale Fabric OFED+ by this release. References to more detailed information are provided where necessary. The information contained in this document is intended for supplemental use only; it should be used in conjunction with the documentation provided for each component.
OFED+ Host SW and PSM_MULTIRAIL_MAP enable this feature. More details can be found in the Intel® True Scale Fabric OFED+ Host Software User Guide. • This release includes fixes in performance-related issues for IPoIB and verbs which provide significant improvement in IPoIB bandwidth and latency and improvement in RDMA/verbs bandwidth. • Added support for RedHat EL 6.3 and RedHat EL 5.9 • qlgc_srp and qlgc_vnic packages are dropped from version 7.2 onwards. On new installation or upgrades of version 7.
OFED+ Host SW • Shared Memory (SHMEM) is included in the ./INSTALL TUI menus as a selection. SHMEM will be installed by default and will also be installed if mpi or psm_mpi or mpidev is selected on the command line. To function, SHMEM requires that at least one MPI be installed on the system. This requirement of one MPI being installed is not enforced by ./INSTALL TUI. SHMEM is a user-level communications library for one-sided operations.
OFED+ Host SW 1.5 Operating Environments Supported The Release 7.2.2.0.8 version of OFED+ Host Software allows for the Operating Systems listed in Table 1. Table 1.
OFED+ Host SW Table 2. CPU Model of Linux Kernel Model uname /proc/cpuinfo EM64T x86_64 Intel CPUs Opteron* x86_64 AMD CPUs Note: Other combinations (such as i586 uname) are not currently supported. February 2014 Order Number: H31512002US True Scale Fabric OFED+ Host Software RN 7.2.2.0.
OFED+ Host SW 1.6 Qualified Parallel File Systems Lustre and IBM General Parallel File System (GPFS) listed below have been tested for use with this release of the Intel® OFED+ host software using the operating systems listed below: • Lustre 2.3 — RHEL 6.3 • Lustre 2.4.1 — RHEL 6.4 • IBM GPFS 3.5.0.14 — RHEL 6.4 Refer to the Intel® True Scale Fabric OFED+ Host Software User Guide for the latest configuration recommendations for optimizing Lustre and GPFS performance with Intel® True Scale Fabric. 1.
OFED+ Host SW 1.8 Hardware Supported Table 4 list the hardware supported in this release. Table 4. Hardware Supported HCAs QLE7340 QLE7342 QME7342 QME7362 QMH7342 MHQH29-* MHQH19-* MHQH19B-XTR MHQH29B-XTR MHQH29B-XSR MCX354A-QCAT MCX353A-QCAT NC543i (HP SL390 G7 in-built InfiniBand Host Channel Adapter) CX-3 LOM down QDR 46M2199 46M2203 1.9 Installation Requirements 1.9.1 Software and Firmware Requirements All Intel IB software on a given node must be at a compatible release level.
OFED+ Host SW Note: The installation process attempts to uninstall any existing distribution versions of OFED, however some rpms included in the distribution packaging of OFED may not be completely uninstalled. It is recommended to uninstall any OFED rpms which come with the distribution prior to installing IntelIB. Note: FastFabric may be used to install the IntelIB-Basic package on all nodes in the cluster.
OFED+ Host SW 1.10.2 Changes to Operating System Support Table 6 shows the new operating systems supported for the releases listed. Table 6. Changes to Operating System Support Release Supported Operating System Added 7.2.0.0.42 RHEL 5 X86_64 (AMD Opteron and Intel EM64T): • (Update 9) 2.6.18-348.el5.x86_64 RHEL 6 X86_64 (AMD Opteron and Intel EM64T): • (Update 3) 2.6.32-279.el6.x86_64 CentOS X86_64 (AMD Opteron and Intel EM64T): • (Update 5.8) 2.6.18-308.el5.x86_64 • (Update 6.3) 2.6.32-279.el6.
OFED+ Host SW 1.10.3 Changes to Software Components Table 7 shows the new software components supported for the releases listed. Table 7. Changes to Software Component Support Release 1.10.4 Supported Software Added or Changed 7.2.0.0.42 Intel® 7.2.1.1.22 Intel® True Scale Fabric OFED+ Host Software 7.2.2.0.
OFED+ Host SW • OFED SDP has not been qualified for this release. IPoIB is recommended for data transfers. 1.12 Product Limitations The following is a list of product limitations for this release: • Intel products will auto-negotiate with devices that utilize IBTA-compliant auto-negotiation. When attaching Intel products to a third-party device, the bit error rate is optimized if the third-party device utilizes attenuation-based tuning. 1.
OFED+ Host SW • The OpenSHMEM effort (see http://www.openshmem.org) is defining a standardized API specification for SHMEM. Although it is premature to claim compliance, Intel® SHMEM aims to be compliant with the OpenSHMEM 1.0 specification. Intel provides a SHMEM API that is compatible with the OpenSHMEM 1.0 specification, other than any omissions or bugs documented in these release notes.
OFED+ Host SW Table 9.
OFED+ Host SW True Scale Fabric OFED+ Host Software RN 7.2.2.0.
OFED+ Host SW 2.0 System Issues for Release 7.2 2.1 Introduction This section provides a list of the resolved Issues in the OFED+ Host Software that were verified by this release. It also lists the open Issues with a description and workaround for each. 2.2 Resolved Issues in this Release Table 10 is a list of issues that are resolved in this and the previous two releases. Table 10. Resolved Issues Product Release Description TrueScale/ Tools 7.2.0.0.
OFED+ Host SW Table 10. Resolved Issues (Continued) Product Release Description True Scale Driver 7.2.1.1.22 The True Scale driver no longer causes a deadlock related to mmap_sem locks and a copy from userspace. IFS/ FastFabric 7.2.2.0.8 Result of iba_verifynodes for C-states are no longer misleading on SLES 11. IFS/ HCA 7.2.2.0.8 OFED+ now works properly with SLES11SP3 kernel 3.0.93-0.8. True Scale Fabric OFED+ Host Software RN 7.2.2.0.
OFED+ Host SW 2.3 Known Issues The subsections below catalog the known open issues for the release as well as a description and a workaround by component. 2.3.1 Severity This document provides a level of severity for each issue listed The levels are: • Critical – Could result in a service outage • Major – Could degrade system performance • Minor – Could cause minimal impact to ongoing operations • None – No operational impact 2.3.2 Open Issues Table Table 11 is the list of open issues for Release 7.2.
OFED+ Host SW Table 11. Open Issues (Continued) Product/ Component Severity Description Workaround Any MVAPICH2 job attempted on fabrics with PCIe HCAs and Third Party or Intel HCAs, must zero the MV2_USE_SRQ environment variable as show in the example of the NAS CG benchmark: IFS/ MPI Major MVAPICH2 jobs run between PCIe HCAs and Third Party or Intel HCAs, may not complete successfully. The test may abort with an ibv_post_recv error. cd directory_containing_benchmark /usr/mpi/gcc/mvapich2-1.
OFED+ Host SW Table 11. Open Issues (Continued) Product/ Component IFS/ Open SM IFS/ IPoIB NEW IFS/ MPI2 NEW IFS/ Rolls/Kits IFS/ Rolls/Kits February 2014 Order Number: H31512002US Severity Description Workaround Minor When using opensm, after bouncing ports on a node, the port may not return to an active state for a period of time.
OFED+ Host SW Table 11. Open Issues (Continued) Product/ Component Severity Description Workaround When installing Moab, the following error is seen: IB Third Party/ Other Minor [nsgib103 .ssh (Thu May 12 05:43:36)]# ldconfig ldconfig: /usr/local/lib/libsqlite3.so.0 is not a symbolic link Move/Delete libsqlite3.so.0 files and execute ldconfig command. ldconfig can create symbolic link properly and the error message will not appear.
OFED+ Host SW Appendix A Performance Gain Conditions Test The following example shows how to determine if conditions 1 and 2, described in the first bullet of “Release 7.1.1 Enhancements” on page 6 hold: $ numactl --hardware available: 2 nodes (0-1) node 0 cpus: 0 1 2 3 4 5 6 7 ... node 1 cpus: 8 9 10 11 12 13 14 15 ... If numactl --hardware shows more than 1 NUMA node, then your OS supports NUMA.
