Intel® Xeon® Processor C5500/ C3500 Series Datasheet - Volume 1 February 2010 Order Number: 323103-001
INFORMATION IN THIS DOCUMENT IS PROVIDED IN CONNECTION WITH INTEL® PRODUCTS. NO LICENSE, EXPRESS OR IMPLIED, BY ESTOPPEL OR OTHERWISE, TO ANY INTELLECTUAL PROPERTY RIGHTS IS GRANTED BY THIS DOCUMENT.
Contents 1.0 Features Summary .................................................................................................. 24 1.1 Introduction ..................................................................................................... 24 1.2 Processor Feature Details ................................................................................... 27 1.2.1 Supported Technologies .......................................................................... 27 1.3 SKUs ........................
2.2 2.3 2.1.10.2 Second Level Address Translation ..............................................54 2.1.11 Address Translations ...............................................................................55 2.1.11.1 Translating System Address to Channel Address ..........................55 2.1.11.2 Translating Channel Address to Rank Address .............................56 2.1.11.3 Low Order Address Bit Mapping .................................................56 2.1.11.4 Supported Configurations .......
2.3.5 2.3.6 2.3.7 2.3.8 2.3.9 2.4 2.5 Addressing ............................................................................................ 95 SMBus Initiated Southbound Configuration Cycles....................................... 97 SMBus Error Handling ............................................................................. 97 SMBus Interface Reset ............................................................................ 97 Configuration and Memory Read Protocol................................
2.6 2.7 2.5.11.2 Coherent Write Flow .............................................................. 117 2.5.11.3 Eviction Policy ....................................................................... 117 2.5.12 Outgoing Request Buffer (ORB) .............................................................. 118 2.5.13 Time-Out Counter ................................................................................. 118 PCI Express Interface .....................................................................
2.7.1 2.7.2 2.7.3 3.0 DMI Error Flow..................................................................................... 134 Processor/PCH Compatibility Assumptions................................................ 134 DMI Link Down .................................................................................... 134 PCI Express Non-Transparent Bridge..................................................................... 135 3.1 Introduction ................................................................
3.19 PCI Express Configuration Registers (NTB Primary Side) ....................................... 170 3.19.1 Configuration Register Map (NTB Primary Side)......................................... 170 3.19.2 Standard PCI Configuration Space (0x0 to 0x3F) - Type 0 Common Configuration Space .............................................................................. 175 3.19.2.1 VID: Vendor Identification Register .......................................... 175 3.19.2.
3.20 3.19.4.7 CORERRMSK: Correctable Error Mask ...................................... 210 3.19.4.8 ERRCAP: Advanced Error Capabilities and Control Register ......... 211 3.19.4.9 HDRLOG: Header Log ............................................................ 211 3.19.4.10 RPERRCMD: Root Port Error Command Register ........................ 212 3.19.4.11 RPERRSTS: Root Port Error Status Register .............................. 212 3.19.4.12 ERRSID: Error Source Identification Register .......................
3.21 3.20.2.13 SB45BASE: Secondary BAR 4/5 Base Address ........................... 249 3.20.2.14 SUBVID: Subsystem Vendor ID (Dev#3, PCIE NTB Sec Mode) ..... 250 3.20.2.15 SID: Subsystem Identity (Dev#3, PCIE NTB Sec Mode) .............. 250 3.20.2.16 CAPPTR: Capability Pointer ..................................................... 250 3.20.2.17 INTL: Interrupt Line Register .................................................. 251 3.20.2.18 INTPIN: Interrupt Pin Register......................................
3.21.1.20 SPAD[0 - 15]: Scratchpad Registers 0 - 15............................... 293 3.21.1.21 SPADSEMA4: Scratchpad Semaphore....................................... 294 3.21.1.22 RSDBMSIXV70: Route Secondary Doorbell MSI-X Vector 7 to 0 ... 295 3.21.1.23 RSDBMSIXV158: Route Secondary Doorbell MSI-X Vector 15 to 8 296 3.21.1.24 WCCNTRL: Write Cache Control Register .................................. 297 3.21.1.25 B2BSPAD[0 - 15]: Back-to-back Scratchpad Registers 0 - 15 ...... 297 3.21.1.
5.6 5.7 Configuration Register Ordering Rules ................................................................ 319 Intel® VT-d Ordering Exceptions ........................................................................ 319 6.0 System Address Map .............................................................................................. 320 6.1 Memory Address Space .................................................................................... 321 6.1.1 System DRAM Memory Regions ................
7.4 7.5 7.6 Virtual Legacy Wires (VLW) .............................................................................. 346 Platform Interrupts .......................................................................................... 347 Interrupt Flow................................................................................................. 347 7.6.1 Legacy Interrupt Handled By IIO Module IOxAPIC ..................................... 348 7.6.2 MSI Interrupt ...........................................
10.1 10.2 10.3 10.4 Introduction .................................................................................................... 370 10.1.1 Types of Reset ..................................................................................... 370 10.1.2 Trigger, Type, and Domain Association .................................................... 370 Node ID Configuration ...................................................................................... 371 CPU-Only Reset ...............................
12.2 12.1.2.2 DDR Channel B Signals .......................................................... 420 12.1.2.3 DDR Channel C Signals .......................................................... 421 12.1.2.4 System Memory Compensation Signals .................................... 421 12.1.3 PCI Express* Signals ............................................................................ 422 12.1.4 Processor SMBus Signals ....................................................................... 422 12.1.
14.4 14.5 14.3.3 Processor Integrated I/O TAP Controller................................................... 517 14.3.4 TAP Interface ....................................................................................... 518 TAP Port Timings ............................................................................................. 520 Boundary-Scan Register Definition .....................................................................
Figures 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 Intel® Xeon® Processor C5500/C3500 Series on the Picket Post Platform -- UP Configuration .......................................................................................................... 25 Intel® Xeon® Processor C5500/C3500 Series on the Picket Post Platform -- DP Configuration ...........................................................................
50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 NTB Port on Intel® Xeon® Processor C5500/C3500 Series Connected to Root Port - NonSymmetric............................................................................................................. 141 NTB Port Connected to Non-Intel® Xeon® Processor C5500/C3500 Series System - NonSymmetric...........................................................................................
Tables 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 Available SKUs ........................................................................................................ 27 Terminology ............................................................................................................ 32 Processor Documents .......................................................................................
54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 Processor’s Intel® QuickPath Interconnect Physical Layer Attributes .............................. 107 Intel® QuickPath Interconnect Link Layer Attributes .................................................... 108 Intel® QuickPath Interconnect Routing Layer Attributes ...............................................
106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 Inbound Memory Address Decoding .......................................................................... 337 Interrupt Source in IOxAPIC Table Mapping ............................................................... 340 I/OxAPIC Table Mapping to PCI Express Interrupts .....................
161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 Processor Absolute Minimum and Maximum Ratings .................................................... 501 Voltage and Current Specifications............................................................................ 502 VCC Static and Transient Tolerance........................................................................... 505 VCC Overshoot Specifications......................................................................
Revision History Date Revision Description February 2010 001 First release February 2010 Order Number: 323103-001 Intel® Xeon® Processor C5500/C3500 Series Datasheet, Volume 1 23
Features Summary 1.0 Features Summary 1.1 Introduction This Datasheet describes DC and AC electrical specifications, signal integrity, differential signaling specifications, pinout and signal definitions, interface functional descriptions, and additional feature information pertinent to the implementation and operation of the Intel® Xeon® processor C5500/C3500 series on its respective platform.
Features Summary Figure 1. Intel® Xeon® Processor C5500/C3500 Series on the Picket Post Platform -- UP Configuration DDR 3 DDR 3 DDR 3 x4 PCIe Pr oc es so r x4 PCIe CSI PECI { x4 PCIe DMI x8 PCIe { x4 PCIe x1 PCIe 6 SATA x1 PCIe x1 PCIe 12 USB 2.
Features Summary Intel® Xeon® Processor C5500/C3500 Series on the Picket Post Platform -- DP Configuration 12 USB 2.
Features Summary 1.2 Processor Feature Details • SKUs supporting one, two, and four cores • Separate 32 kB instruction and 32 kB data L1 cache — L1 data and instruction cache are implemented as two redundent caches each of which is parity protected • A 256-KB shared instruction/data L2 cache with ECC for each core • Up to 8 MB shared instruction/data L3 cache with ECC, shared among all cores • SKUs at different power and performance levels supporting UP (uni-processor) and DP (dual-processor) 1.2.
Features Summary Table 1. Available SKUs TDP (W) Intel® HyperThreading Tech LLC Cache (MB) Cores/ Threads Up to 2.13 GHz Yes 4 2/4 1.73 No No 2 1.33 No Yes 2 Base Clock Speed (GHz) Processor Number1 DP Capable LC3528 No 35 1.73 LC3518 No 23 P1053 No 30 Note: 1. Turbo Freq Thermal Profile (High TCase) Intel® QuickPath Link Speed DDR3 Memory Memory Channels 79.6° C (nominal) 94.6° C (short) NA 1066/ 800 2 1 79.5° C (nominal) 94.
Features Summary — Single Channel Mode — Independent Channel Mode — Spare Channel Mode — Mirrored Mode — Lockstep Mode — Dual-channel: Modes; Symmetric (Interleaved); Asymmetric — Intel® Flex Memory Technology • Command launch modes of 1n/2n • Various RAS modes • On-Die Termination (ODT) • Intel® Fast Memory Access (Intel® FMA): — Just-in-Time Command Scheduling — Command Overlap — Out-of-Order Scheduling • Asynchronous DRAM Refresh (ADR) 1.4.
Features Summary — Does not support dynamic lane reversal. • Supports Half Swing “low-power/low-voltage” mode. • Message Signaled Interrupt (MSI and MSI-X) messages. • Polarity inversion. 1.4.4 Direct Media Interface (DMI) • Compliant to Direct Media Interface Second Generation (DMI2). • Four lanes in each direction. • 2.5 GT/s point-to-point DMI2 interface to PCH is supported. • Uses the 100-MHz PCI Express reference clock (supplied through PCH). • 64-bit downstream host address format.
Features Summary 1.5 Power Management Support 1.5.1 Processor Core • Full support of ACPI C-states as implemented by the following processor core & package C-states: — Core: C0, C1E, C3, C6 — Package: C0, C3, C6 • Enhanced Intel SpeedStep® Technology 1.5.2 System • S0, S1, S3, S4, S5 1.5.3 Memory Controller • Conditional self-refresh (Intel® Rapid Memory Power Management (Intel® RMPM) • Dynamic power-down • Asynchronous DRAM Refresh 1.5.4 PCI Express • L0, L0s, L1, L3 1.5.
Features Summary 1.8 Terminology Table 2.
Features Summary Table 2. Terminology (Sheet 2 of 2) Term PCH 1.9 Description Platform Controller Hub. The new, 2009 chipset with centralized platform capabilities including the main I/O interfaces along with display connectivity, audio features, power management, manageability, security and storage features. The PCH may also be referred to by the name Intel® 3420 chipset.
Features Summary Table 4. PCH Documents Document Document Number Ibex Peak Platform Controller Hub (PCH) - External Design Specification (EDS) 401376 Ibex Peak Platform Controller Hub (PCH) - Thermal Mechanical Specifications & Guidelines 407051 Notes: 1. Contact your Intel representative for the latest revision of this item. Table 5. Public Specifications Document Number/ Location Document Advanced Configuration and Power Interface Specification 3.0 http://www.acpi.
Interfaces 2.0 Interfaces This chapter describes the interfaces supported by the processor. 2.1 System Memory Interface The complete list of supported memory configurations is preliminary, and is subject to change before product launch. 2.1.1 System Memory Technology Supported The Intel® Xeon® processor C5500/C3500 series contains an integrated memory controller (IMC). The memory interface supports up to three DDR3 channels. Each channel consists of 64 bit data and 8 ECC bits.
Interfaces Table 6.
Interfaces Intel® Xeon® Processor C5500/C3500 Series with RDIMM Only Support Table 7.
Interfaces Table 9. DDR3 System Memory Timing Support Transfer Rate (MT/s) tCL (tCK) tRCD (tCK) tRP (tCK) CWL (tCK) CMD Mode Notes 800 5 5 5 5 1n and 2n 1 800 6 6 6 5 1n and 2n 1 7 7 7 8 8 8 6 1n and 2n 1 1333 8 8 8 7 2n 1 1333 9 9 9 7 2n 1 1066 Notes: 1. System Memory timing support is based on availability and is subject to change. 2.1.3.1 System Memory Operating Modes The IMC contains three DDR3 channel controllers.
Interfaces Table 10. Mapping from Logical to Physical Channels Channel Mode Independent Channels Lockstep 2.1.3.2 Mirroring Logical to Physical Disabled 1:1 relationship, but may not be the same number. Enabled A pair of physical channels are combined to form a single logical redundant channel. Requests to logical channel A are handled by physical channels A and B. Disabled A pair of physical channels are accessed in parallel to form a single logical channel.
Interfaces Figure 3. Independent Code Layout x 4 x 4 x 4 x 4 x 4 x 4 x 4 x 4 CB [7:0] x 4 x 4 x 4 x 4 x 4 x 4 x 4 x 4 x 4 x 4 DQ[71:0] Symbol on DRAM pins DIMM Channel 1 Transfer 0 Transfer 1 2.1.3.
Interfaces 2.1.3.5 Mirrored Channel Mode The following modes of operation are required to implement mirroring. 2.1.3.5.1 Mirroring Redundant Mode Software puts the Integrated Memory Controller into this mode whenever the memory image in both channels is identical, or where they differ, the contents are not valid. In addition, both WDBs must be empty before enabling this mode.
Interfaces to receive inputs from a mirrored partner. If both channels fail simultaneously, an uncorrectable error must be signaled. Mirror mode only recovers from a single error (Resilvering is not supported). 2.1.3.6 Lockstep Mode Lockstep Mode refers to splitting cache lines across channels. In this mode, the same address is used on both channels, and an error on either channel is detected. The ECC code used by the memory controller can correct 4 bits out of 72 bits.
Interfaces 2.1.3.6.1 Limitations Lockstepped channels must be populated identically. That is, each DIMM in one channel must have an identical corresponding DIMM in the alternate channel; identical in number ranks, banks, rows, and columns. DIMMs may be of different speed grades, but the memory controller will be configured to operate all DIMMs according to the slowest parameters present. Only channels A and B support lockstep, the third channel is unused in lockstep mode.
Interfaces Figure 5. Dual-Channel Symmetric (Interleaved) and Dual-Channel Asymmetric Modes Dual Channel Interleaved (memory sizes must match) Dual Channel Asymmetric (memory sizes can differ) CL CL CH. B Top of Memory CH. B Top of Memory CH. A CH.A-top DRB CH. A CH. B CH. A CH. B CH. A 0 0 Channel selector controlled by DCC[10:9] 2.1.3.7.
Interfaces 2.1.4 DIMM Population Requirements In all modes, the frequency of system memory is the lowest frequency of all memory modules placed in the system, as determined through the SPD registers on the memory modules. 2.1.4.1 General Population Requirements All DIMMs must be DDR3 DIMMs. Registered DIMMs must be ECC only; Unbuffered DIMMs can be ECC or non-ECC. Mixing Registered and Unbuffered DIMMs is not allowed. It is allowed to mix ECC and non-ECC Unbuffered DIMMs.
Interfaces Figure 7. DIMM Population Within a Channel Fill Third Processor Table 11. Fill First D I M M D I M M D I M M 2 1 0 CLK: P2/N2 Chip Select: 2/3 ODT: 4/5 CKE: 0/2 Note: Fill Second P1/N1 P0/N0 4/5/6/7 0/1/2/3 0/1 2/3 1/3 0/2 ODT[5:4] is muxed with CS[7:6]#.
Interfaces Table 12. UDIMM Population Configurations Within a Channel for Three Slots per Channel Configuration Number POR Speed 1N or 2N DIMM2 DIMM1 DIMM0 1 DDR3-1333, 1066, & 800 1N Empty Empty Single-rank 2 DDR3-1333, 1066, & 800 1N Empty Empty Dual-rank 3 DDR3-1066 & 800 2N Empty Single-rank Single-rank 4 DDR3-1066 & 800 2N Empty Single-rank Dual-rank 5 DDR3-1066 & 800 2N Empty Dual-rank Single-rank 6 DDR3-1066 & 800 2N Empty Dual-rank Dual-rank 2.1.4.2.
Interfaces Table 13. DIMM Population Configurations Within a Channel for Two Slots per Channel (Sheet 2 of 2) Configuration # POR Speed 1N or 2N DIMM1 DIMM0 8 DDR3-800 1N Single-rank Quad-rank 9 DDR3-800 1N Dual-rank Quad-rank 10 DDR3-800 1N Quad-rank Quad-rank Table 14.
Interfaces to system memory Just-in-Time to make optimal use of Command Overlapping. Thus, instead of having all memory access requests go individually through an arbitration mechanism forcing requests to be executed one at a time, they can be started without interfering with the current request allowing for concurrent issuing of requests. This allows for optimized bandwidth and reduced latency while maintaining appropriate command spacing to meet system memory protocol. 2.1.5.
Interfaces Figure 9. Error Signaling Logic 2.1.7.1 Enabling SMI/NMI for Memory Corrected Errors The MC_SMI_SPARE_CNTRL register has enables for SMI and NMI interrupts. Only one should be set. Whichever type of interrupt is enabled will be triggered if: • a DIMM error counter exceeds the threshold, • redundancy is lost on a mirrored configuration, or • a sparing operation completes. This register is set by hardware once operation is complete. Bit is cleared by hardware when a new operation is enabled.
Interfaces MC_SMI_SPARE_CNTRL1 register holds an SMI_ERROR_THRESHOLD1 to which the counters are compared. If any counter exceeds the threshold, the enabled interrupt will be generated, and status bits are set to indicate which counter met threshold. 2.1.7.3 Identifying the Cause of An Interrupt Table 15 defines how to determine what caused the interrupt. Table 15. Causes of SMI or NMI Cause Recommended platform software response. MC_SMI_SPARE_DIMM_ERROR_STATUS.
Interfaces 2.1.10 Memory Address Decode Memory address decode is the process of taking a system address and converting it to rank, bank, row and column address on a memory channel. Memory address decode is performed in two levels. The first level selects the socket (in DP systems) and memory channel, and generates a channel address. The second level decodes the channel address into the rank, bank, row and column addresses. 2.1.10.1 First Level Decode Figure 10 below shows the address decode flow.
Interfaces 2.1.10.1.2 Channel Interleaving Cache lines (linearly increasing addresses) in an address range can interleave across 1, 2, 3, 4, or 6 memory channels. 2.1.10.1.3 Logical to Physical Channel Mapping The MC_CHANNEL_MAPPER register and the lockstep bit define the mapping of logical channels decoded by the first level decode and physical channels in the Integrated Memory Controller.
Interfaces 2.1.10.2 Second Level Address Translation Second level address translation converts the channel address resulting from the first level decode into rank, bank row and column addresses. The Integrated Memory Controller uses DIMM interleaving to balance loads. The channel address can be divided into 8 ranges and each range supports an interleave across 1,2 or 4 ranks. The MC_RIR_LIMIT_CH[2:0]_[7:0] and MC_RIR_WAYS_CH[2:0]_[31:0] registers define the ranges and rank interleaves. 2.1.10.2.
Interfaces 2.1.11 Address Translations 2.1.11.1 Translating System Address to Channel Address This operation could be considered the final step of Level 1 decode. It removes the “gaps” introduced by Level 1 decode to produce a contiguous channel address. This maintains the independence of Level 1 and Level 2 decode. Independence simplifies the memory mapping problem that must be solved by BIOS.
Interfaces 2.1.11.2 Translating Channel Address to Rank Address This section describes how gaps are removed from the channel address to form a contiguous address space for each rank. Gaps from one to three cache lines in size result from interleaving across ranks on a channel. Gaps larger than 512 MB are a result from the interleaves below. The Integrated Memory Controller uses DIMM interleaving to balance loads. Interleaving assigns low order address bits to variable DRAM bits.
Interfaces Table 18. Critical Word First Sequence of Read Returns Transfer Most Significant 8B to GQ Least Significant 8B to GQ SysAdrs[3]=1 SysAdrs[3]=0 First pair SysAdrs[5], SysAdrs[4] SysAdrs[5], SysAdrs[4] Second pair SysAdrs[5], !SysAdrs[4] SysAdrs[5], !SysAdrs[4] Third pair !SysAdrs[5], SysAdrs[4] !SysAdrs[5], SysAdrs[4] Fourth pair !SysAdrs[5], !SysAdrs[4] !SysAdrs[5], !SysAdrs[4] The mapping of System Address bits to read return transfers are the same for lockstep.
Interfaces 2.1.11.4 Supported Configurations The following table defines the DDR3 organizations that the Integrated Memory Controller supports. Table 20. DDR Organizations Supported 2.1.12 DDR Protocol Support The Integrated Memory Controller will use burst length of 4 for lockstep and 8 for independent channels. The Integrated Memory Controller will not vary burst length during operation. 2.1.13 Refresh The Integrated Memory Controller will issue refreshes when no commands are pending to a rank.
Interfaces The delay between ZQ commands and subsequent operations and the rate of ZQCS commands are defined in the ZQ Timings register. No other commands will be issued between bank closure and ZQCS. Initial calibration will be performed by initiating a ZQCL command using the DDR3CMD register. The ZQCL command initiates a calibration sequence in the DRAM that updates driver and termination values. Specific DRAM vendors will specify a rate to initiate ZQCS calibration commands.
Interfaces 2.1.14.2.3 Precharge Power Down If power-down occurs when all banks are idle, this mode is referred to as precharge power-down. If power-down occurs when there is a row active in any bank, this mode is referred to as active power-down. A DRAM in power-down deactivates its input and output buffers, excluding CK, CK#, ODT, and CKE. The Integrated Memory Controller will not actively seek precharge power down. If requests stop for the power down delay, the channel will de-assert CKE.
Interfaces Table 22. Dynamic IO Power Savings Features (Sheet 2 of 2) Power Savings Feature On Condition Time to Turn Off Off Condition Time to Turn On Disable mixer and amp in phase interpolators for data drivers No data to drive. Derived from write CAS.
Interfaces Figure 11. Mapping Throttlers to Ranks Throttle Mode EXTTS (Some DIMM Hot) MinThrottle DutyCycle 256 Off Metastability M On Throttle Now Duty Cycle Virtual Temp Sensor 0 Throttle Now Duty Cycle Virtual Temp Sensor 1 Ch0ThrottleRank[7:0] Throttle Now Rank Mapping Ch1ThrottleRank[7:0] Duty Cycle Virtual Temp Sensor 2 Ch2ThrottleRank[7:0] Throttle Now Duty Cycle Virtual Temp Sensor 3 Channel 0 Channel 1 Channel 2 2.1.14.5.
Interfaces 2.1.14.5.3 Virtual Temperature Counter A counter will track the temperature above ambient of the hottest RAM in each rank. On each DRAM cycle, it will be incremented to reflect heat produced by DRAM activity and decremented to reflect cooling. This counter is saturating, it does not add when all ones, nor does it subtract when all zeros. 2.1.14.5.4 Per Command Energy On each DCLK, the virtual temperature counter is increased to model the heat produced by the command issued in a previous DCLK.
Interfaces 2.1.14.5.7 Response to Throttling Trigger When throttling is triggered, CKE will be de-asserted. DRAM issue will be suppressed to the hot rank except for refreshes. After 256 DCLKs, command issue will be allowed according to the MinThrottleDutyCycle field. After that, command issue will be permitted if temperature is above threshold. This should be the case as the worst case cooling after 256 DCLKs of precharge power down should be sufficient to allow many commands.
Interfaces unrestrained BW such that the average over many seconds is that which is thermally supported. The choppiness will be lessened by configuring MinThottleDutyCycle no lower than that required by the specific DIMMs plugged into each system. 2.1.14.7 Advanced Throttling Options There are two closed loop throttling considerations which can be addressed by a thermal control agent (Management Hardware via PECI or SW periodically running on a core).
Interfaces Table 24. Refresh for Different DRAM Types Type Open Loop Platform or some DRAM on the socket does not support ETR. DRAM temperatures are limited to 85 degrees. All DRAM on the socket and the platform support ERT. Throttling does not limit DRAM temperature below 88 degrees. Closed Loop Refresh interval is always tREFI. There is no 2x refresh response. Self refresh entry is not delayed. Refresh rate is always 1x.
Interfaces 2.1.14.10 Rank Sharing Throttling logic is shared when more than four ranks are present as described in the following tables. With 1 or 2 Single or Dual rank DIMMs on a channel, there are no more than four ranks present. Each throttler is associated with a single rank. The logical ranks are not consecutive due to the motherboard routing required to support three DIMMs on a channel. Table 25.
Interfaces Table 27. Thermal Throttling Control Fields (Sheet 1 of 2) Register Dynamically Validated Parameter Bits One per Description MC_THERMAL_CONTROL THROTTLE_MODE 2 Channel Defines the source of throttling information to be DDR_THERM# signal, virtual temperature sensor, or Throttle_Now configuration bit. Throttling can also be disabled with this field. MC_THERMAL_CONTROL THROTTLE_EN 1 Channel DRAM commands will be throttled.
Interfaces Table 27. Thermal Throttling Control Fields (Sheet 2 of 2) Register Dynamically Validated MC_THROTTLE_OFFSET Parameter RANK Bits 8 One per Description Throttler Compared against bits [36:29] of virtual temperature to determine the throttle point. Recommended value is 255. MC_COOLING_COEF YES RANK 8 Throttler Heat removed from DRAM in 8 DCLKs. This should be scaled relative to the per command weights and the initial value of the throttling threshold.
Interfaces • CRC check byte used to efficiently and atomically confirm accurate data delivery. • Synchronization at the beginning of every message minimizes device timing accuracy requirements. Generic PECI specification details are out of the scope of this document and instead can be found in RS - Platform Environment Control Interface (PECI) Specification, Revision 2.0.
Interfaces PECI-based access to DRAM thermal readings and throttling control coefficients provide a means for Board Management Controllers (BMCs) or other platform management devices to feed hints into on-die memory controller throttling algorithms. These control coefficients are accessible using PCI configuration space writes via PECI reference are documented in Section 2.2.2.5. 2.2.1.
Interfaces 2.2.2.2 GetDIB() The processor PECI client implementation of GetDIB() includes an 8-byte response and provides information regarding client revision number and the number of supported domains. All processor PECI clients support the GetDIB() command. 2.2.2.2.1 Command Format The GetDIB() format is as follows: Write Length: 1 Read Length: 8 Command: 0xf7 Figure 14. GetDIB() Byte # Byte Definition 2.2.2.2.
Interfaces 2.2.2.2.3 Revision Number All clients that support the GetDIB command also support Revision Number reporting. The revision number may be used by a host or originator to manage different command suites or response codes from the client. Revision Number is always reported in the second byte of the GetDIB() response. The Revision Number always maps to the revision number of the supported PECI Specification. Figure 16.
Interfaces Figure 17. GetTemp() Byte # Byte Definition 0 1 2 3 Client Address Write Length 0x01 Read Length 0x02 Cmd Code 0x01 4 5 6 7 FCS Temp[7:0] Temp[15:8] FCS Example bus transaction for a thermal sensor device located at address 0x30 returning a value of negative 10° C: Figure 18. GetTemp() Example Byte # Byte Definition 2.2.2.3.
Interfaces 255 for legacy processor and 254 for non-legacy processor. The client will return all 1’s in the data response and ‘pass’ for the completion code for all of the following conditions: • Unimplemented Device • Unimplemented Function • Unimplemented Register Figure 19. 31 PCI Configuration Address 28 27 20 Reserved 19 Bus 15 Device 14 12 11 0 Function Register PCI configuration reads may be issued in byte, word, or dword granularities. 2.2.2.4.
Interfaces 2.2.2.4.2 Supported Responses The typical client response is a passing FCS, a passing Completion Code (CC) and valid Data. Under some conditions, the client’s response will indicate a failure. Table 31. PCIConfigRd() Response Definition Response 2.2.2.5 Meaning Abort FCS Illegal command formatting (mismatched RL/WL/Command Code) CC: 0x40 Command passed, data is valid CC: 0x80 Error causing a response timeout.
Interfaces Multi-Domain Support: Yes (see Table 41) Description: Writes the data sent to the requested register address. Write Length dictates the desired write granularity. The command always returns a completion code indicating the pass/fail status information. Write commands issued to illegal Bus Numbers, or unimplemented Device / Function / Register addresses are ignored but return a passing completion code. See Section 2.2.4.2 for details regarding completion codes. Figure 21.
Interfaces 2.2.2.6.1 Capabilities Table 34. Mailbox Command Summary Command Name Request Type Code (byte) MbxSend Data (dword) MbxGet Data (dword) Description Ping 0x00 0x00 0x00 Verify the operability / existence of the mailbox. Thermal Status Read/Clear 0x01 Log bit clear mask Thermal Status Register Read the thermal status register and optionally clear any log bits.
Interfaces These status bits are a subset of the bits defined in the IA32_THERM_STATUS MSR on the processor, and more details on the meaning of these bits may be found in the Intel® 64 and IA-32 Architectures Software Developer’s Manual, Vol. 3B. Both status and sticky log bits are managed in this status word. All sticky log bits are set upon a rising edge of the associated status bit, and the log bits are cleared only by Thermal Status reads or a processor reset.
Interfaces 2.2.2.6.5 Icc-TDC Read Icc-TDC is the Intel® Xeon® processor C5500/C3500 series TDC current draw specification. This data may be used to confirm matching Icc profiles of processors in DP configurations. It may also be used during the processor boot sequence to verify processor compatibility with motherboard Icc delivery capabilities. This command returns Icc-TDC in units of 1 Amp. 2.2.2.6.6 TCONTROL Read TCONTROL is used for fan speed control management.
Interfaces Table 36. 2.2.2.6.
Interfaces The Intel® Xeon® processor C5500/C3500 series supports software initiated T-state throttling and automatic T-state throttling as part of the internal Thermal Monitor response mechanism (upon TCC activation). The PECI T-state throttling control register read/write capability is managed only in the PECI domain. In-band software may not manipulate or read the PECI T-state control setting.
Interfaces Figure 26. MbxSend() Command Data Format 1 0 Byte # Byte Definition 2 Request Type 3 4 Data[31:0] Because a particular MbxSend() command may specify an update to potentially critical registers inside the processor, it includes an Assured Write FCS (AW FCS) byte as part of the write data payload. See the RS - Platform Environment Control Interface (PECI) Specification, Revision 2.0 for a definition of the AW FCS protocol.
Interfaces The 4-byte data defined above is sent in standard PECI ordering with LSB first and MSB last. Table 38. MbxSend() Response Definition Response Bad FCS Meaning Electrical error CC: 0x4X Semaphore is granted with a Transaction ID of ‘X’ CC: 0x80 Error causing a response timeout. Either due to a rare, internal timing condition or a processor RESET condition or processor S1 state. Retry is appropriate outside of the RESET or S1 states.
Interfaces Figure 28. MbxGet() Byte # Byte Definition 0 1 2 3 Client Address Write Length 0x02 Read Length 0x05 Cmd Code 0xd5 4 10 5 11 6 Transaction ID FCS Completion Code 7 10 5 8 11 6 9 LSB Response Data[31:0] 10 11 MSB FCS The 4-byte data response defined above is sent in standard PECI ordering with LSB first and MSB last. Table 39. MbxGet() Response Definition Response Aborted Write FCS Meaning Response data is not ready. Command retry is appropriate.
Interfaces 2.2.2.9.2 Transaction ID For all MbxSend() commands that complete successfully, the passing completion code (0x4X) includes a 4-bit Transaction ID (‘X’). That ID is the key to the mailbox and must be sent when retrieving response data and releasing the lock by using the MbxGet() command. The Transaction ID is generated internally by the processor and has no relationship to the originator of the request.
Interfaces Table 40. Table 41. Domain ID Definition Domain ID Domain Number 0b01 0 0b10 1 Multi-Domain Command Code Reference Command Name Domain 0 Code Domain 1 Code GetTemp() 0x01 0x02 PCIConfigRd() 0xC1 0xC2 PCIConfigWr() 0xC5 0xC6 MbxSend() 0xD1 0xD2 MbxGet() 0xD5 0xD6 2.2.4 Client Responses 2.2.4.
Interfaces Table 43. Device Specific Completion Code (CC) Definition Completion Code 0x00..0x3F 0x40 0x4X 0x50..0x7F Description Device specific pass code Command Passed Command passed with a transaction ID of ‘X’ (0x40 | Transaction_ID[3:0]) Device specific pass code CC: 0x80 Error causing a response timeout. Either due to a rare, internal timing condition or a processor RESET condition or processor S1 state. Retry is appropriate outside of the RESET or S1 states.
Interfaces 2.2.6 Temperature Data 2.2.6.1 Format The temperature is formatted in a 16-bit, 2’s complement value representing a number of 1/64 degrees centigrade. This format allows temperatures in a range of ±512°C to be reported to approximately a 0.016°C resolution. Figure 29. Temperature Sensor Data Format MSB Upper nibble MSB Lower nibble S x x x Sign 2.2.6.2 x x LSB Upper nibble x x Integer Value (0-511) x x LSB Lower nibble x x x x x x Fractional Value (~0.
Interfaces Table 45. Error Codes and Descriptions Error Code Description 0x8000 General Sensor Error (GSE) 2.2.7 Client Management 2.2.7.1 Power-up Sequencing The PECI client is fully reset during processor RSTIN# assertion. This means that any transactions on the bus will be completely ignored, and the host will read the response from the client as all zeroes.
Interfaces 2.2.7.2 Device Discovery The PECI client is available on all processors, and positive identification of the PECI revision number can be achieved by issuing the GetDIB() command. The revision number acts as a reference to the RS - Platform Environment Control Interface (PECI) Specification, Revision 2.0 document applicable to the processor client definition. See Section 2.2.2.2 for details on GetDIB response formatting. 2.2.7.
Interfaces Table 48. PECI Client Response During S1 Command 2.2.7.6 Response Ping() Fully functional GetDIB() Fully functional GetTemp() Fully functional PCIConfigRd() Fully functional PCIConfigWr() Fully functional MbxSend() Fully functional MbxGet() Fully functional Processor Reset The Intel® Xeon® processor C5500/C3500 series PECI client is fully reset on all RSTIN# assertions.
Interfaces function) space or in memory mapped space. The PECI registers are not within the IIO portion of the processor, and therefore cannot be accessed from the SMBus. — In a dual processor configuration, the SMBus Master (BMC for example) must use the SMBus slave local to each processor to access the IIO registers in that processor as remote peer to peer IO and configuration cycles are not supported.
Interfaces Every configuration read or write first consists of an SMBus write sequence which initializes the Bus Number, Device, and so on. The term sequence is used since these variables may be written with a single block write or multiple word or byte writes. Once these parameters are initialized, the SMBus master can initiate a read sequence (which performs a configuration register read) or a write sequence (which performs a configuration register write).
Interfaces The SMBus interface uses an internal register stack that is filled by the SMBus master before a request to the config master block is made. Table 50 provides a list of the bytes in the stack and their descriptions. Table 50. Internal SMBus Protocol Stack SMBus Stack usage for bus/dev/func commands (cmd[5] = 0) Description Command Command Command byte Byte Count Byte Count The number of bytes for this transaction when Block command is used.
Interfaces Table 51. SMBus Slave Address Format Slave Address Field Bit Position Slave Address Source [4] 0 [3] 1 [2] Inversion of DMI_PE_CFG# strap pin [1] 0 [0] Read/Write# bit. This bit is in the slave address field to indicate a read or write operation. It is not part of the SMBus slave address.
Interfaces 2.3.6 SMBus Initiated Southbound Configuration Cycles The platform SMBus master agent that is connected to an IIO slave SMBus agent can request a configuration transaction to a downstream PCI-Express device. If the address decoder determines that the request is not intended for this IIO (i.e. not the IIO’s bus number), it sends the request to port with the bus address. All requests outside ofthis range are sent to the legacy ESI port for a master abort condition. 2.3.
Interfaces • The master holds SCL continuously high for 50 us. • Force a platform reset. Note: Since the configuration registers are affected by the reset pin, SMBus masters will not be able to access the internal registers while the system is in reset. 2.3.9 Configuration and Memory Read Protocol Configuration and memory reads are accomplished through an SMBus write(s) and later followed by an SMBus read.
Interfaces 2.3.9.1 SMBus Configuration and Memory Block-Size Reads Figure 31. SMBus Block-Size Configuration Register Read S Write address for a Read sequence Read data sequence 1110_1X0 W A Rsv[3:0] & Addr[11:8] A S 1110_1X0 W A Sr 1110_1X0 R Data [15:8] A A Cmd = 11010010 A Byte cnt = 4 A Regoff [7:0] A PEC A Cmd = 11010010 A Byte cnt = 5 A Status A Bus Num A Dev / Func A Data [31:24] A Data [23:16] A P Poll until Status[7] = 0 Figure 32.
Interfaces 2.3.9.2 SMBus Configuration and Memory Word-Size Reads Figure 33. SMBus Word-Size Configuration Register Read Write address for a Read sequence Read Sequence Poll until Status[7] = 0 Figure 34.
Interfaces 2.3.9.3 SMBus Configuration and Memory Byte Reads Figure 35.
Interfaces Figure 36.
Interfaces 2.3.9.4 Configuration and Memory Write Protocol Configuration and memory writes are accomplished through a series of SMBus writes. As with configuration reads, a write sequence is first used to initialize the Bus Number, Device, Function, and Register Number for the configuration access. The writing of this information can be accomplished through any combination of the supported SMBus write commands (Block, Word or Byte). Note: On the SMBus, there is no concept of byte enables.
Interfaces 2.3.9.6 SMBus Configuration and Memory Word Writes Figure 39. SMBus Word-Size Configuration Register Write S 1110_1X0 W A Cmd = 10011001 A Bus Num A Dev / Func A PEC A P S 1110_1X0 W A Cmd = 00011001 A Rsv[3:0] & Addr[11:8] A Regoff [7:0] A PEC A P S 1110_1X0 W A Cmd = 00011001 A Data [31:24] A Data [23:16] A PEC A P S 1110_1X0 W A Cmd = 01011001 A Data [15:8] A Data [7:0] A PEC A P Figure 40.
Interfaces Figure 42. 2.
Interfaces Figure 43.
Interfaces 2.4.2 Physical Layer Implementation The physical layer of the Intel® QPI bus is the physical entity between two components, it uses a differential signalling scheme, and is responsible for the electrical transfer of data. 2.4.2.1 Processor’s Intel® QuickPath Interconnect Physical Layer Attributes The processor’s Intel® QuickPath Interconnect Physical layer attributes are summarized in Table 54 below. Table 54. Processor’s Intel® QuickPath Interconnect Physical Layer Attributes Feature 2.4.
Interfaces 2.4.4 Intel® QuickPath Interconnect Probing Considerations When a Logic Analyzer probe is present on the Intel® QuickPath Interconnect links (for hardware debug purposes), the characteristics of the Intel® QuickPath Interconnect link are changed. This requires slightly different transmitter equalization parameters and retraining period. It is expected that these alternate parameters will be stored in BIOS. There is no mechanism for automatically detecting the presence of probes.
Interfaces 2.4.7 Intel® QuickPath Interconnect Address Decoding On past FSB platforms, the processors and I/O subsystem could direct all memory and I/O accesses to the North Bridge. The processor’s Intel® QuickPath Interconnect is more distributed in nature. The memory controller is integrated inside the processor. Therefore, a processor may be able to resolve memory accesses locally or may have to send it to another processor.
Interfaces 2.4.9.2 Intel® QuickPath Interconnect Coherent Protocol Attributes Table 57.
Interfaces 2.4.9.5 Fault Handling Table 60. Intel® QuickPath Interconnect Fault Handling Attributes Interrupt Attribute Support Machine check indication through Int No Time-out hierarchy for fault diagnosis Only via 3-strike counter Packet elimination for error isolation between partitions No Abort time-out response Only via 3-strike counter 2.4.9.6 Reset/Initialization Table 61.
Interfaces 2.5.2 Link Layer There are 128 Flit (Flow control unit of transfer) link layer credits to be split between VN0 and VNA virtual channels from the IIO. One VN0 credit is used per Intel® QPI message class in the normal configuration, which consumes a total of 26 Flits in the Flit buffer. For UP systems, with the six Intel® QPI message classes supported, this will leave the remaining 102 Flits to be used for VNA credits.
Interfaces VNA and VN0 follow similar ordering to the message class being transported on it. With Home message class requiring ordering across VNA/VN0 for the same cache line, all other message classes have no ordering requirement. 2.5.3 Protocol Layer The protocol layer is responsible for translating requests from the core into the Intel® QPI domain, and for maintaining protocol semantics. The IIO is a Intel® QPI caching agent. It is also a fully-compliant ‘IO’ (home) agent for non-coherent I/O traffic.
Interfaces 2.5.5.1 NodeID Generation This section contains an overview of how source address decoder generates the NodeID. There are assumed fields for each decoder entry. In the case of some special decoder ranges, the fields in the decoder may be fixed or shifted to match different address ranges, but the basic flow is similar across all ranges. Table 64 defines the fields used per memory source address decoder. The process for using these fields to generate a NodeID is: 1. Match Range 2.
Interfaces Table 65. Field I/O Decoder Entries Type Address Base/Range Size (Bytes) attr Interleave CSR Register Comments VGA/CSeg Memory 000A_0000 128K MMIO None QPIPVSAD Space can be disabled. LocalxAPIC Memory FEE0_0000 1M IPI 8 deep table QPIPAPICSAD Which bits of address select the table entry is variable. Special Requests Targeting DMI. Memory N/A N/A Varies None QPIPSUBSAD Peer-to-peer between PCie and DMI is not supported.
Interfaces 2.5.8 Inbound Coherent The IIO only sends a subset of the coherent transactions supported in Intel® QPI. This section describes only the transactions that are considered coherent. The determination of Coherent versus Non-Coherent is made by the address decode. If a transaction is determined coherent by address decode, it may still be changed to noncoherent as a result of its PCI Express attributes. The IIO supports only source broadcast snooping, with the Invalidating Write Back flow.
Interfaces Table 66. Feature Profile Control Register Attribute UP Profile DP Profile Notes disable enable In UP profile all inbound requests are sent to a single target NodeID. Source Address decoder enable QPIPCTRL RW Address bits QPIPMADDATA RW <=40 bits [39:0] NodeID width QPIPCTRL RO 3-bit Remote P2P 1 RO disable Poison QPIPCTRL RW enable When disabled any uncorrectable data error will be treated identically to a header parity.
Interfaces 2.5.12 Outgoing Request Buffer (ORB) When an inbound request is issued onto Intel® QPI an ORB entry is allocated. This list keeps all pertinent information about the transaction header needed to complete the request. It also stores the cache line address for coherent transactions to allow conflict checking with snoops (used for conflict checking for other requests). When a request is issued, a RTID (Requestor Transaction ID) is assigned based on NodeID. The ORB depth is 64 entries. 2.5.
Interfaces 2.6 PCI Express Interface This section describes the PCI Express* interface capabilities of the processor. See the latest PCI Express Base Specification, Revision 2.0 for PCI Express details. The processor has four PCI Express controllers, allowing the sixteen lanes to be controlled as a single x16 port, or two x8 ports, or one x8 port and two x4 ports, or four x4 ports. 2.6.
Interfaces Figure 45. Packet Flow through the Layers 2.6.1.1 Transaction Layer The upper layer of the PCI Express architecture is the Transaction Layer. The Transaction Layer's primary responsibility is the assembly and disassembly of Transaction Layer Packets (TLPs). TLPs are used to communicate transactions, such as read and write, as well as certain types of events. The Transaction Layer also manages flow control of TLPs. 2.6.1.
Interfaces During link training, the processor will attempt link negotiation starting from the highest defined link width and ramp down to the nearest supported link width that passes negotiation. For example, when x16 support is defined the port will first attempt negotiation as a single x16. If that fails, an attempt is made to negotiate as a single x8 link. If that fails an attempt is made to negotiate as a single x4 link.
Interfaces The bifurcation control registers are sticky and BIOS can choose to program the register and cause an IIO reset and the appropriate bifurcation will take effect on exit from that reset. Table 68. Link Width Strapping Options PECFGSEL[2:0] 000 2.6.2.
Interfaces Table 69. Supported Degraded Modes in IIO Original Link Width1 x4 x2 Degraded Mode Link Width and Lanes Numbers x2 on either lanes 1-0, 0-1 x1 on either lanes 0, 1, 2, 3 x1 on either lanes 0, 1 1. This is the native width the link is running at when degraded mode operation kicks-in Entry into or exit from degraded mode are reported to software in the MISCCTRLSTS register, and also records which lane failed.
Interfaces 2.6.5.2 ASSERT_GPE / DEASSERT_GPE General Purpose Event (GPE) consists of two messages: Assert_GPE and Deassert_GPE. Upon receipt of a Assert_GPE message from a PCI Express port, the IIO forwards the message to the PCH. When the GPE event has been serviced, the IIO will receive a Deassert_GPE message on the PCI Express port. At this point the IIO can send the deassert_GPE message on DMI. 2.6.
Interfaces 2.6.7 Inbound Transactions Inbound refers to the direction towards main memory from I/O. 2.6.7.1 Inbound PCI Express Messages Supported Table 70 lists all inbound messages that may be received on a PCI Express downstream port (does not include DMI messages). In a given system configuration, certain messages are not applicable being received inbound on a PCI Express port. They will be called out as appropriate. Table 70.
Interfaces 2.6.8 Outbound Transactions This section describes the IIO behavior towards outbound transactions. Throughout the rest of the section, outbound refers to the direction from processor towards I/O. 2.6.8.1 Memory, I/O and Configuration Transactions Supported Table 71 lists the possible outbound memory, I/O and configuration transactions. Table 71.
Interfaces 2.6.10 Outbound Messages Supported Table 72 provides a list of all the messages supported as an initiator on a PCI Express port (DMI messages are not included in this table). Table 72. Outgoing PCI Express Message Cycles PCI Express Transaction Outbound Messages Intel Chipset-specifc Vendor-defined 2.6.10.1 Address Space or Message Reason for Issue Unlock Releases a locked read or write transaction previously issued on PCI Express.
Interfaces 2.6.12 Transaction Descriptor The PCI Express Base Specification, Revision 2.0 defines a field in the header called the Transaction Descriptor. This descriptor comprises three sub-fields: • Transaction ID • Attributes • Traffic class 2.6.12.1 Transaction ID The Transaction ID uniquely identifies every transaction in the system. The Transaction ID comprises four sub-fields described in Table 73. This table provides details on how this field in the Express header is populated by IIO. Table 73.
Interfaces 2.6.12.2 Attributes PCI Express supports two attribute hints described in Table 74. This table describes how these attribute fields are populated for requests and completions. Table 74. PCI Express Attribute Handling Attribute 2.6.12.3 Definition Relaxed Ordering Allows the system to relax some of the standard PCI ordering rules. Snoop Not Required This attribute is set when an I/O device controls coherency through software mechanisms.
Interfaces 2.6.14.3 Completions for Locked Read Requests LkRdCmp and RdCmp are aliased -i.e. either of these completion types can terminate a locked/non-locked read request. 2.6.15 PCI Express RAS The PCI Express Advanced Error Reporting (AER) capability is supported. See the PCI Express Base Specification, Revision 2.0 for details. 2.6.16 ECRC Support ECRC is not supported. ECRC is ignored and dropped on all incoming packets and is not generated on any outgoing packet. 2.6.
Interfaces When a transaction times out or receives a UR/CA response on a request outstanding on PCI Express, recovery in hardware is not attempted. UR/CA received does not cause any error escalation via AER mechanism and cause error escalation. A completion timeout condition is treated as a normal non-fatal error condition (and not as an advisory condition). An unexpected completion received from PCI Express port is treated as an advisory non-fatal error if the severity of it is set to non-fatal.
Interfaces Table 76. PCI Express Credit Mapping for Inbound Requests Flow Control Type Initial IIO Advertisement Definition Posted Request Header Credits (PRH) Tracks the number of posted requests the agent is capable of supporting. Each credit accounts for one posted request. 16(x4) 32(x8) 64(x16) Posted Request Data Credits (PRD) Tracks the number of posted data the agent is capable of supporting. Each credit accounts for up to 16 bytes of data.
Interfaces Table 77. PCI Express Credit Mapping for Outbound Requests (Sheet 2 of 2) Flow Control Type 2.6.22.1 Initial IIO Advertisement Definition Non-Posted Request Data Credits (NPRD) Tracks the number of non-posted data the agent is capable of supporting. Each credit accounts for up to 16 bytes of data. 12 (x4) 24 (x8) 48 (x16) Completion Header Credits (CPH) Tracks the number of completion headers the agent is capable of supporting.
Interfaces 2.7 Direct Media Interface (DMI2) The Direct Media Interface in the IIO is responsible for sending and receiving packets/ commands to the PCH. The DMI is an extension of the standard PCI Express specification with special commands/features added to mimic the legacy Hub Interface. DMI2 is the second generation extension of DMI. See the DMI Specification, Revision 2.0, for more DMI2 details. Note: Other references to DMI are referring to the same DMI2-compliant interface described above.
PCI Express Non-Transparent Bridge 3.0 PCI Express Non-Transparent Bridge 3.1 Introduction PCI Express* non-transparent bridge (NTB) acts as a gateway that enables high performance, low overhead communication between two intelligent subsystems, the local and the remote subsystems.
PCI Express Non-Transparent Bridge • Supports peer-to-peer transactions upstream and downstream across NTB. Capabilities for NTB are the same as defined for PCIE ports. See Section 3.7, “NTB Inbound Transactions” and Section 3.8, “Outbound Transactions” for details. • Supports sixteen, 32-bit scratch pad registers, (total 64B) that are accessible through the BAR0 configuration space.
PCI Express Non-Transparent Bridge Figure 46. Enumeration in System with Transparent Bridges and Endpoint Devices CPU T ra n s p a re n t B r id g e Type 1 T ra n s p a re n t B r id g e Type 1 End P o in t End P o in t End P o in t Type 0 T ra n s p a re n t B r id g e Type 1 End P o in t In contrast, a NTB provides logical isolation of resources in a system in addition to providing electrical isolation and system expansion capability.
PCI Express Non-Transparent Bridge Figure 47 shows a system with a NTB. The NTB provides address translation for transactions that cross from one memory space to the other. Figure 47.
PCI Express Non-Transparent Bridge 3.4 NTB Support in Intel® Xeon® Processor C5500/C3500 Series When using the NTB capability the Intel® Xeon® processor C5500/C3500 series will support the NTB functionality on port 0 only in either the 1x4 or 1x8 configuration. The NTB functionality is not supported in the single x16 port configuration. The BIOS must enable the NTB function. In addition, a software configuration enable bit will provide the ability to enable or disable the NTB port . 3.
PCI Express Non-Transparent Bridge 3.5.2 Connecting NTB Port on Intel® Xeon® Processor C5500/C3500 Series to Root Port on Another Intel® Xeon® Processor C5500/ C3500 Series System - Symmetric Configuration In the configuration shown in Figure 49, the NTB port on one Intel® Xeon® processor C5500/C3500 series (the system on the left), is connected to the root port of the Intel® Xeon® processor C5500/C3500 series system on the right.
PCI Express Non-Transparent Bridge 3.5.3 Connecting NTB Port on Intel® Xeon® Processor C5500/C3500 Series to Root Port on Another System - Non-Symmetric Configuration In the configuration shown in Figure 50, the NTB port on one Intel® Xeon® processor C5500/C3500 series (the system on the left), is connected to the root port of the system on the right.
PCI Express Non-Transparent Bridge Figure 51.
PCI Express Non-Transparent Bridge 3.6 Architecture Overview The NTB provides two interfaces and sets of configuration registers, one for each of the interfaces shown in Figure 52. The interface to the on-chip CPU complex is referred to as the local host interface. The external interface is referred to as the remote host interface.
PCI Express Non-Transparent Bridge Figure 52. Intel® Xeon® Processor C5500/C3500 Series NTB Port - Nomenclature Intel® Xeon® Processor C5500/C3500 Series Core Complex Local Host PCIE Root Port (RP) PCIE Non_Transparent Bridge (NTB) PCIE RCiEP PCIE EP The NTB port supports the Type 0 configuration header. The first 10 DW of the Type 0 configuration header is shown in Table 78. The NTB sets the following parameters in the configuration header.
PCI Express Non-Transparent Bridge Table 79. Class Code 23:16 15:8 7:0 Class Code Sub-Class Code Programming Interface Byte 0x06 (bridge) 0x80 (other bridge type) 0x00 • Header type is set to type 0 The base address registers (BAR) specify the address decode functions that will be supported by the NTB. • The Intel® Xeon® processor C5500/C3500 series NTB will support only 64b BARs. Intel® Xeon® processor C5500/C3500 series will not support 32b BARs.
PCI Express Non-Transparent Bridge Enumeration software can determine how much address space the device requires by writing a value of all 1's to the BAR and then reading the value back. Unimplemented Base Address registers are hardwired to zero. The size of each BAR is determined based on the weight of the least significant bit that is writable in the BAR address bits b[63:7] for a 64b BAR. (The minimum memory address range defined in PCIE is 4KB).
PCI Express Non-Transparent Bridge Figure 53 describes the three usage models and their behavior regarding crosslink training. Figure 53.
PCI Express Non-Transparent Bridge No Cross-link configuration is required: Hardware will automatically strap the port as an DSD/USP when the PPD register, Port Definition field, is set to “10”b (NTB/RP). The Intel® Xeon® processor C5500/C3500 series NTB will train as DSD/USP and the external RP will train as USD/DSP. No conflict occurs and link training proceeds without need for crosslink training. Note: When configured as a NTB/RP.
PCI Express Non-Transparent Bridge strap input PE_NTBXL is disabled and has no meaning. User should leave the PE_NTBXL pin strap unconnected in this configuration to save board space. Note: PPD, Crosslink Configuration Status field has been provided as a means to visually see the polarity of the final result between the pin strap and the BIOS option. 3.6.
PCI Express Non-Transparent Bridge 3. Enumeration SW running independently on each host will discover and set the base address pointer for both primary BAR2/3 and primary BAR4/5 registers (PB23BASE, PB45BASE) of the NTB associated with that same host. At this point all that is known is the size and location of the memory window. E.g. 4KB to 512GB prefetchable memory window placed on a size multiple base address.
PCI Express Non-Transparent Bridge — It is the responsibility of the remote host system to introduce the Intel® Xeon® processor C5500/C3500 series NTB into its hierarchy and is outside the scope of this document to describe that procedure. — If the attached system is another Intel® Xeon® processor C5500/C3500 series RP the EP is brought into the system as described in Case 1 above.
PCI Express Non-Transparent Bridge 3.6.6 Address Translation The NTB uses the BARs in the Type 0 configuration header specified above to define apertures into the memory space on the other side of the NTB. The NTB supports two sets of BARs, one on the local host interface and the other on the remote host interface. Each BAR has control and setup registers that are writable from the other side of the bridge. The address translation register defines the address translation scheme.
PCI Express Non-Transparent Bridge The address forwarded from one interface to the other is translated by adding a base address to the offset within the BAR that the address belongs to as shown in Figure 56. Figure 56. Direct Address Translation PCI Express utilizes both 32-bit and 64-bit address schemes via the 3DW and 4DW headers. To prevent address aliasing, all devices must decode the entire address range. All discussions in this section refer to 64-bit addressing.
PCI Express Non-Transparent Bridge The following registers are used to translate the local physical address to the remote guest address in the remote host system map (Transactions going across NTB from primary side to secondary side) Section 3.19.2.12, “PB23BASE: Primary BAR 2/3 Base Address” Section 3.19.2.13, “PB45BASE: Primary BAR 4/5 Base Address” Section 3.21.1.1, “PBAR2LMT: Primary BAR 2/3 Limit” Section 3.21.1.2, “PBAR4LMT: Primary BAR 4/5 Limit” Section 3.21.1.
PCI Express Non-Transparent Bridge The offset to the base of the 4 GB window on the incoming address is preserved in the translated address. 3.6.7 Requester ID Translation Completions for non-posted transactions are routed using Requester ID instead of the address. The NTB provides a mechanism to translate the Requester ID and the Completer ID from one domain to the other. The Requester ID consists of the Requester’s PCI bus number, device number and function number.
PCI Express Non-Transparent Bridge If the configuration is NTB/RP then the secondary side of the NTB will be per the PCI Express Base Specification, Revision 2.0. For this configuration the secondary side of the NTB must capture the Bus and Device numbers supplied with all Type 0 Configuration Write requests sent across the link to the NTB. For inbound reads received from the remote host, the NTB performs the address translation and launches the memory read on the local processor.
PCI Express Non-Transparent Bridge Figure 59.
PCI Express Non-Transparent Bridge 3.7 NTB Inbound Transactions This section talks about the NTB behavior for transactions that originate from an external agent on the PCIE link towards the PCI Express NTB port. Throughout this chapter, inbound refers to the direction towards the CPU from I/O. 3.7.
PCI Express Non-Transparent Bridge 3.7.2 Inbound PCI Express Messages Supported Table 82 lists all inbound messages that Intel® Xeon® processor C5500/C3500 series supports receiving on a PCI Express NTB secondary side. In a given system configuration, certain messages are not applicable being received inbound on a PCI Express port. They will be called out as appropriate. Table 82. PCI Express Transaction Incoming PCI Express Message Cycles Address Space or Message Unlock Silently dropped by NTB.
PCI Express Non-Transparent Bridge 3.8 Outbound Transactions This section describes the NTB behavior towards outbound transactions to an external agent on the PCIE link. Throughout the rest of the chapter, outbound refers to the direction from CPU towards I/O. 3.8.1 Memory, I/O and Configuration Transactions The IIO will generate outbound memory transactions to NTB MMIO space and to memory on an external agent connected to the secondary side of the NTB across the PCI Express link.
PCI Express Non-Transparent Bridge 3.8.2 Lock Support The NTB does not support lock cycles from either side of the NTB. The local host views the NTB as a RCiEP (primary side). The remote host views the NTB as a PCIE EP (secondary side). • Primary side: PCI Express-compliant software drivers and applications must be written to prevent the use of lock semantics when accessing a Root Complex Integrated Endpoint.
PCI Express Non-Transparent Bridge Table 84. PCI Express Transaction Outgoing PCI Express Message Cycles with Respect to NTB Address Space or Message Reason for Issue Unlock The NTB does not support lock cycles from either side of the NTB. Primary side: PCI Express-compliant software drivers and applications must be written to prevent the use of lock semantics when accessing a Root Complex Integrated Endpoint.
PCI Express Non-Transparent Bridge 3.8.3.1 EOI NTB is a Root Complex integrated End Point (RCiEP) with respect to the local host and as such should not receive EOI messages from the host when configured as a NTB. Note: Due to hardware simplification in the PCIE logic, the BIOS must set bit 26 Disable EOI in the Section 3.19.4.20, “MISCCTRLSTS: Misc. Control and Status Register” to prevent EOI message from being sent when configured as a NTB. 3.
PCI Express Non-Transparent Bridge Table 85. PCI Express Transaction ID Handling Field IIO as Completer IIO as Requester Bus Number Specifies the bus number that the requester resides on. The IIO fills this field in with its internal Bus Number that the PCI Express cluster resides on the IIOBUSNO: IIO Internal Bus Number. See Section 3.6.3.17, “IIOBUSNO: IIO Internal Bus Number” in Volume 2 of the Datasheet. Device Number Specifies the device number of the requester.
PCI Express Non-Transparent Bridge 3.10.3 Traffic Class The IIO does not optimize based on traffic class. The IIO can receive a packet with TC!= 0 and treat the packet as if it were TC = 0 from an ordering perspective. IIO forwards the TC filed as is on peer-to-peer requests and also returns the TC field from the original request on the completion packet sent back to the device. 3.
PCI Express Non-Transparent Bridge • BIOS enumerates the NTB in the local host address space. The mapping of the remote host interface to the other system is done subsequently by higher level platform software. • This mechanism avoids the race condition and timing relationship between when the two systems initialize. Each system initializes only its internal components and does not have any dependency on the availability and timing of the second system. 3.12.
PCI Express Non-Transparent Bridge 3.13 Reset Requirements The NTB isolates two independent systems. As such, a system reset on one system must not cause any reset activity on the second system. When one of the systems connected through the NTB port goes down, the corresponding PCIE link goes down. The second system will eventually detect that PCIE link down status and flush all pending transactions to/from the system that went down. 3.
PCI Express Non-Transparent Bridge Figure 60.
PCI Express Non-Transparent Bridge 3.16 MSI-X Vector Mapping Intel® Xeon® processor C5500/C3500 series provides four MSI-X vectors which are mapped to groups of PDOORBELL bits per Table 96, “MSI-X Vector Handling and Processing by IIO on Primary Side”. If the OS cannot support 4 MSI-X vectors but is capable of programming all of the MSI-X table and data registers, Section 3.21.2.1, “PMSIXTBL[0-3]: Primary MSI-X Table Address Register 0 - 3” , Section 3.21.2.
PCI Express Non-Transparent Bridge PBAR01BASE, SB01BASE, Offset 5ECH (CBDF), Bits All. This register does not capture the correct values for the BDF so should not be used for debug. The returned value for the completer ID in the Complettion packet will be incorrect. This will not impact functional operation with Intel chipsets since we do not check this field in the completion packet at the receiver. Other RPs outside of Intel is unknown. PBAR01BASE, SB01BASE, Offset 70CH (USMEMMISS), Bits All.
PCI Express Non-Transparent Bridge This section discusses the primary side registers. Figure 61. PCI Express NTB (Device 3) Type0 Configuration Space 0x150 ERRCAPHDR 0x100 PMCAP 0xE0 PXPCAPID 0x90 MSIXCAPID 0x80 MSICAPID 0x60 CAPPTR 0x34 0x40 0x00 PCI Device Dependent 0x160 ACSCAPHDR PCI Header XP3RUET_HDR_EXT Extended Configuration Space 0xFFF Figure 61 illustrates how each PCI Express port’s configuration space appears to software.
PCI Express Non-Transparent Bridge Table 88.
PCI Express Non-Transparent Bridge Table 89.
PCI Express Non-Transparent Bridge Table 90.
PCI Express Non-Transparent Bridge 3.19.2 Standard PCI Configuration Space (0x0 to 0x3F) - Type 0 Common Configuration Space This section covers primary side registers in the 0x0 to 0x3F region that are common to Bus 0, Device 3. The secondary side of the NTB is discussed in the next section and is located on NTB Bus M, Device 0. Comments at the top of the table indicate what devices/functions the description applies to.
PCI Express Non-Transparent Bridge 3.19.2.3 PCICMD: PCI Command Register (Dev#3, PCIE NTB Pri Mode) This register defines the PCI 3.0 compatible command register values applicable to PCI Express space. Register:PCICMD Bus:0 Device:3 Function:0 Offset:04h Bit Attr Default 15:11 RV 00h 10 RW 0 9 RO 0 Description Reserved. (by PCI SIG) INTxDisable: Interrupt Disable Controls the ability of the PCI-Express port to generate INTx messages.
PCI Express Non-Transparent Bridge Register:PCICMD Bus:0 Device:3 Function:0 Offset:04h Bit Attr Default 3 RO 0 Special Cycle Enable Not applicable to PCI Express must be hardwired to 0. 0 Bus Master Enable When this bit is Set = 1b, the PCIE NTB will forward Memory Requests upstream from the secondary interface to the primary interface.
PCI Express Non-Transparent Bridge 3.19.2.4 PCISTS: PCI Status Register The PCI Status register is a 16-bit status register that reports the occurrence of various events associated with the primary side of the “virtual” PCI-PCI bridge embedded in PCI Express ports and also primary side of the other devices on the internal IIO bus.
PCI Express Non-Transparent Bridge Register:PCISTS Bus:0 Device:3 Function:0 Offset:06h Bit Attr Default 11 RW1C 0 10:9 RO 0h Description Signaled Target Abort This bit is set when the NTB port forwards a completer abort (CA) completion status from the secondary interface to the primary interface. DEVSEL# Timing Not applicable to PCI Express. Hardwired to 0.
PCI Express Non-Transparent Bridge 3.19.2.5 RID: Revision Identification Register This register contains the revision number of the IIO. The revision number steps the same across all devices and functions i.e. individual devices do not step their RID independently. The IIO supports the CRID feature where in this register’s value can be changed by the BIOS. See Section 3.2.2, “Compatibility Revision ID” in Volume 2 of the Datasheet for details.
PCI Express Non-Transparent Bridge 3.19.2.7 CLSR: Cacheline Size Register Register:CLSR Bus:0 Device:3 Function:0 Offset:0Ch 3.19.2.8 Bit Attr Default Description 7:0 RW 0h Cacheline Size This register is set as RW for compatibility reasons only. Cacheline size for IIO is always 64B. IIO hardware ignore this setting. PLAT: Primary Latency Timer This register denotes the maximum time slice for a burst transaction in legacy PCI 2.3 on the primary interface.
PCI Express Non-Transparent Bridge 3.19.2.10 BIST: Built-In Self Test This register is used for reporting control and status information of BIST checks within a PCI Express port. It is not supported by Intel® Xeon® processor C5500/C3500 series. Register:BIST Bus:0 Device:3 Function:0 Offset:0Fh 3.19.2.11 Bit Attr Default 7:0 RO 0h Description BIST_TST: BIST Tests Not supported.
PCI Express Non-Transparent Bridge 3.19.2.12 PB23BASE: Primary BAR 2/3 Base Address The register is used by the processor on the primary side of the NTB to setup a 64b prefetchable memory window. Note: SW must program upper DW first and then lower DW. If lower DW is programmed first HW will clear the lower DW.
PCI Express Non-Transparent Bridge 3.19.2.13 PB45BASE: Primary BAR 4/5 Base Address The register is used by the processor on the primary side of the NTB to setup a second 64b prefetchable memory window. Note: SW must program upper DW first and then lower DW. If lower DW is programmed first HW will clear the lower DW. Register:PB45BASE Bus:0 Device:3 Function:0 Offset:20h Bit 63:nn 3.19.2.
PCI Express Non-Transparent Bridge 3.19.2.15 SID: Subsystem Identity (Dev#3, PCIE NTB Pri Mode) This register identifies a particular subsystem. Register:SID Bus:0 Device:3 Function:0 Offset:2Eh 3.19.2.16 Bit Attr Default Description 15:0 RWO 0000h Subsystem ID: This field must be programmed during BIOS initialization. When any byte or combination of bytes of this register is written, the register value locks and cannot be further updated.
PCI Express Non-Transparent Bridge 3.19.2.18 INTPIN: Interrupt Pin Register The INTPIN register identifies legacy interrupt INTx support. Register:INTPIN Bus:0 Device:3 Function:0 Offset:3Dh Bit 7:0 3.19.2.19 Attr RWO Default 01h Description INTP: Interrupt Pin This field defines the type of interrupt to generate for the PCI-Express port.
PCI Express Non-Transparent Bridge 3.19.3 Device-Specific PCI Configuration Space - 0x40 to 0xFF 3.19.3.1 MSICAPID: MSI Capability ID Register:MSICAPID Bus:0 Device:3 Function:0 Offset:60h 3.19.3.2 Bit Attr Default 7:0 RO 05h Description Capability ID Assigned by PCI-SIG for MSI. MSINXTPTR: MSI Next Pointer Register:MSINXTPTR Bus:0 Device:3 Function:0 Offset:61h Bit Attr Default 7:0 RWO 80h Description Next Ptr 3.19.3.
PCI Express Non-Transparent Bridge Register:MSICTRL Bus:0 Device:3 Function:0 Offset:62h Bit Attr Default Description Multiple Message Enable 6:4 RW 000b Applicable only to PCI Express ports. Software writes to this field to indicate the number of allocated messages which is aligned to a power of two. When MSI is enabled, the software will allocate at least one message to the device. A value of 000 indicates 1 message.
PCI Express Non-Transparent Bridge 3.19.3.4 MSIAR: MSI Address Register The MSI Address Register (MSIAR) contains the system specific address information to route MSI interrupts from the root ports and is broken into its constituent fields. Register:MSIAR Bus:0 Device:3 Function:0 Offset:64h Bit Attr Default Description 31:20 RW 0h 19:12 RW 00h 11:4 RW 00h 3 RW 0h 2 RW 0h 0: physical 1: logical 1:0 RO 0h Reserved.
PCI Express Non-Transparent Bridge 3.19.3.5 MSIDR: MSI Data Register The MSI Data Register contains all the data (interrupt vector) related to MSI interrupts from the root ports. Register:MSIDR Bus:0 Device:3 Function:0 Offset:68h Bit Attr Default 31:16 RO 0000h 15 RW 0h Description Reserved.
PCI Express Non-Transparent Bridge 3.19.3.6 MSIMSK: MSI Mask Bit Register The Mask Bit register enables software to disable message sending on a per-vector basis. Register:MSIMSK Bus:0 Device:3 Function:0 Offset:6Ch Bit Attr Default 31:02 RsvdP 0h Description Reserved Mask Bits 01:00 3.19.3.7 RW 00b For each Mask bit that is set, the PCI Express port is prohibited from sending the associated message.
PCI Express Non-Transparent Bridge 3.19.3.9 MSIXNXTPTR: MSI-X Next Pointer Register:MSIXNXTPTR Bus:0 Device:3 Function:0 Offset:81h Bit Attr Default 7:0 RWO 90h Description Next Ptr 3.19.3.10 This field is set to 90h for the next capability list (PCI Express capability structure) in the chain.
PCI Express Non-Transparent Bridge 3.19.3.11 TABLEOFF_BIR: MSI-X Table Offset and BAR Indicator Register (BIR) Register default: 00002000h Register:TABLEOFF_BIR Bus:0 Device:3 Function:0 Offset:84h Bit Attr Default 31:03 RO 00000400h 02:00 3.19.3.12 RO 0h Description Table Offset MSI-X Table Structure is at offset 8K from the PB01BASE address. See Section 3.19.3.13, “PXPCAPID: PCI Express Capability Identity Register” for the start of details relating to MSI-X registers.
PCI Express Non-Transparent Bridge 3.19.3.13 PXPCAPID: PCI Express Capability Identity Register The PCI Express Capability List register enumerates the PCI Express Capability structure in the PCI 3.0 configuration space. Register:PXPCAPID Bus:0 Device:3 Function:0 Offset:90h 3.19.3.14 Bit Attr Default 7:0 RO 10h Description Capability ID Provides the PCI Express capability ID assigned by PCI-SIG. Required by PCI Express Base Specification, Revision 2.0 to be this value.
PCI Express Non-Transparent Bridge 3.19.3.15 PXPCAP: PCI Express Capabilities Register The PCI Express Capabilities register identifies the PCI Express device type and associated capabilities. Register:PXPCAP Bus:0 Device:3 Function:0 Offset:92h Bit Attr Default 15:14 Rsvd P 00b 13:9 RO Reserved 00000b Interrupt Message Number Applies only to the RPs. This field indicates the interrupt message number that is generated for PM/HP events.
PCI Express Non-Transparent Bridge 3.19.3.16 DEVCAP: PCI Express Device Capabilities Register The PCI Express Device Capabilities register identifies device specific information for the device. Register:DEVCAP Bus:0 Device:3 Function:0 Offset:94h Bit Attr Default 31:29 Rsvd P 0h Reserved 28 RO 0b Function Level Reset Capability A value of 1b indicates the Function supports the optional Function Level Reset mechanism. NTB does not support this functionality.
PCI Express Non-Transparent Bridge Register:DEVCAP Bus:0 Device:3 Function:0 Offset:94h Bit Attr Default 8:6 RO 000 5 RO 1 4:3 RO 00b 2:0 RO 001b Description Endpoint L0s Acceptable Latency Does not apply to IIO RCiEP (Link does not exist between host and RCiEP) Extended Tag Field Supported IIO devices support 8-bit tag 1 = Maximum Tag field is 8 bits 0 = Maximum Tag field is 5 bits Phantom Functions Supported IIO does not support phantom functions.
PCI Express Non-Transparent Bridge 3.19.3.17 DEVCTRL: PCI Express Device Control Register (Dev#3, PCIE NTB Pri Mode) The PCI Express Device Control register controls PCI Express specific capabilities parameters associated with the device. Register:DEVCTRL Bus:0 Device:3 Function:0 Offset:98h PCIE_ONLY Bit Attr Default 15 RsvdP 0h Description Reserved.
PCI Express Non-Transparent Bridge Register:DEVCTRL Bus:0 Device:3 Function:0 Offset:98h PCIE_ONLY Bit 4 3 2 1 0 February 2010 Order Number: 323103-001 Attr RO RW RW RW RW Default Description 0 Enable Relaxed Ordering Not applicable since the NTB is never the originator of a TLP. This bit has no impact on forwarding of relaxed ordering attribute on peer requests. 0 Unsupported Request Reporting Enable Applies only to the PCI Express RP/PCI Express NTB secondary interface/DMI ports.
PCI Express Non-Transparent Bridge 3.19.3.18 DEVSTS: PCI Express Device Status Register The PCI Express Device Status register provides information about PCI Express device specific parameters associated with the device. Register:DEVSTS Bus:0 Device:3 Function:0 Offset: 9Ah Bit Attr Default 15:6 RsvdZ 000h Description Reserved. Transactions Pending: Does not apply. Bit is hardwired to 0 NTB is a special case bridging device following the rule below. The PCI Express Base Specification, Revision 2.
PCI Express Non-Transparent Bridge 3.19.3.19 PBAR23SZ: Primary BAR 2/3 Size This register contains a value used to set the size of the memory window requested by the 64-bit BAR 2/3 pair for the Primary side of the NTB. Register:PBAR23SZ Bus:0 Device:3 Function:0 Offset:0D0h Bit 7:0 3.19.3.20 Attr RWO Default Description 00h Primary BAR 2/3 Size Value indicating the size of 64-bit BAR 2/3 pair on the Primary side of the NTB. This value is loaded by BIOS prior to enumeration.
PCI Express Non-Transparent Bridge 3.19.3.21 SBAR23SZ: Secondary BAR 2/3 Size This register contains a value used to set the size of the memory window requested by the 64-bit BAR 2/3 pair for the Secondary side of the NTB. Register:SBAR23SZ Bus:0 Device:3 Function:0 Offset:0D2h Bit 7:0 3.19.3.22 Attr RWO Default Description 00h Secondary BAR 2/3 Size Value indicating the size of 64-bit BAR 2/3 pair on the Secondary side of the NTB. This value is loaded by BIOS prior to enumeration.
PCI Express Non-Transparent Bridge 3.19.3.23 PPD: PCIE Port Definition This register defines the behavior of the PCIE port which can be either a RP, NTB connected to another NTB or an NTB connected to a Root Complex. This register is used to set the value in the DID register on the Primary side of the NTB (located at offset 02h). This value is loaded by BIOS prior to running PCI enumeration.
PCI Express Non-Transparent Bridge 3.19.3.24 PMCAP: Power Management Capabilities Register The PM Capabilities Register defines the capability ID, next pointer and other power management related support. The following PM registers /capabilities are added for software compliance. Register:PMCAP Bus:0 Device:3 Function:0 Offset:E0h Bit Attr Default Description PME Support Indicates the PM states within which the function is capable of sending a PME message. NTB primary side does not forward PME messages.
PCI Express Non-Transparent Bridge 3.19.3.25 PMCSR: Power Management Control and Status Register This register provides status and control information for PM events in the PCI Express port of the IIO. Register:PMCSR Bus:0 Device:3 Function:0 Offset:E4h Bit Attr Default Description 31:24 RO 00h 23 RO 0h Bus Power/Clock Control Enable This field is hardwired to 0h as it does not apply to PCI Express. 22 RO 0h B2/B3 Support This field is hardwired to 0h as it does not apply to PCI Express.
PCI Express Non-Transparent Bridge 3.19.4 PCI Express Enhanced Configuration Space 3.19.4.1 VSECPHDR: Vendor Specific Enhanced Capability Header This register identifies the capability structure and points to the next structure.
PCI Express Non-Transparent Bridge 3.19.4.2 VSHDR: Vender Specific Header This register identifies the capability structure and points to the next structure. Register:VSHDR Bus:0 Device:3 Function:0 Offset:104h 3.19.4.3 Bit Attr Default Description 31:20 RO 03Ch VSEC Length This field indicates the number of bytes in the entire VSEC structure, including the PCI Express Enhanced Capability header, the Vendor-Specific header, and the Vendor-Specific Registers.
PCI Express Non-Transparent Bridge 3.19.4.4 UNCERRMSK: Uncorrectable Error Mask This register masks uncorrectable errors from being signaled.
PCI Express Non-Transparent Bridge 3.19.4.5 UNCERRSEV: Uncorrectable Error Severity This register indicates the severity of the uncorrectable errors.
PCI Express Non-Transparent Bridge 3.19.4.6 CORERRSTS: Correctable Error Status This register identifies the status of the correctable errors that have been detected by the PCI Express port. Register:CORERRSTS Bus:0 Device:3 Function:0 Offset:114h 3.19.4.
PCI Express Non-Transparent Bridge 3.19.4.8 ERRCAP: Advanced Error Capabilities and Control Register Register:ERRCAP Bus:0 Device:3 Function:0 Offset:11Ch Bit Attr Default 31:9 RV 0h 8 RO 0 ECRC Check Enable: N/A to IIO 7 RO 0 ECRC Check Capable: N/A to IIO 6 RO 0 ECRC Generation Enable: N/A to IIO 5 RO 0 ECRC Generation Capable: N/A to IIO 4:0 3.19.4.
PCI Express Non-Transparent Bridge 3.19.4.10 RPERRCMD: Root Port Error Command Register This register controls behavior upon detection of errors. Register:ERRCMD Bus:0 Device:3 Function:0 Offset:130h 3.19.4.11 Bit Attr Default Description 31:3 RV 0h 2 RW 0 FATAL Error Reporting Enable Enable MSI/MSI-X interrupt on fatal errors when set. See Section 11.6, “IIO Errors Handling Summary” (IOH Platform Architecture Specification) for details of MSI/MSI-X generation for PCI Express error events.
PCI Express Non-Transparent Bridge Register:RPERRSTS Bus:0 Device:3 Function:0 Offset:134h Bit Attr Default Description Advanced Error Interrupt Message Number Advanced Error Interrupt Message Number offset between base message data an the MSI/MSI-X message if assigned more than one message number. IIO hardware automatically updates this register to 0x1h if the number of messages allocated to the RP is 2. See bit 6:4 in Section 3.19.3.
PCI Express Non-Transparent Bridge 3.19.4.12 ERRSID: Error Source Identification Register Register:ERRSID Bus:0 Device:3 Function:0 Offset:138h Bit 31:16 15:0 3.19.4.13 Attr ROS ROS Default Description 0h Fatal Non Fatal Error Source ID Requestor ID of the source when an Fatal or Non Fatal error message is received and the Error Fatal/Nonfatal Received bit is not already set. i.e log ID of the first Fatal or Non Fatal error message.
PCI Express Non-Transparent Bridge 3.19.4.14 APICBASE: APIC Base Register BDF 030 Offset 140H. This register exist in both RP and NTB modes. It is documented in RP Section 3.4.5.13, “APICBASE: APIC Base Register”. See Volume 2 of the Datasheet. 3.19.4.15 APICLIMIT: APIC Limit Register BDF 030 Offset 142H. This register exist in both RP and NTB modes. It is documented in RP Section 3.4.5.14, “APICLIMIT: APIC Limit Register”. See Volume 2 of the Datasheet. 3.19.4.
PCI Express Non-Transparent Bridge 3.19.4.18 ACSCTRL: Access Control Services Control Register This register identifies the Access Control Services (ACS) control bits. Register:ACSCTRL Bus:0 Device:3 Function:0 Offset:156h Bit Attr Default 15:7 RO 0 Reserved. 6 RO 0 ACS Direct Translated P2P Enable (T) This is hardwired to 0b as the component does not implement ACS Direct Translated P2P.
PCI Express Non-Transparent Bridge 3.19.4.21 PCIE_IOU0_BIF_CTRL: PCIE IOU0 Bifurcation Control Register BDF 030 Offset 190H. This register exist in both RP and NTB modes. It is documented in RP Section 3.4.5.21, “PCIE_IOU0_BIF_CTRL: PCIE IOU0 Bifurcation Control Register” in Volume 2 of the Datasheet. 3.19.4.22 NTBDEVCAP: PCI Express Device Capabilities Register The PCI Express Device Capabilities register identifies device specific information for the device.
PCI Express Non-Transparent Bridge Register:NTBDEVCAP Bus:0 Device:3 Function:0 Offset:194h Bit 11:9 8:6 Attr RWO RWO Default Description 110b Endpoint L1 Acceptable Latency This field indicates the acceptable latency that an Endpoint can withstand due to the transition from L1 state to the L0 state. It is essentially an indirect measure of the Endpoint’s internal buffering.
PCI Express Non-Transparent Bridge 3.19.4.23 LNKCAP: PCI Express Link Capabilities Register The Link Capabilities register identifies the PCI Express specific link capabilities The link capabilities register needs some default values setup by the local host. This register has been created to provide a back door path to program the link capabilities from the primary side. The link capabilities register on the secondary side of the NTB is located at Section 3.20.3.
PCI Express Non-Transparent Bridge Register:LNKCAP Bus:0 Device:3 Function:0 Offset:19Ch Bit 14:12 11:10 9:4 3:0 Attr RWO RWO RWO RO Default 011 11 001000b See description Description L0s Exit Latency This field indicates the L0s exit latency (i.e L0s to L0) for the PCI-Express port.
PCI Express Non-Transparent Bridge 3.19.4.24 LNKCON: PCI Express Link Control Register The PCI Express Link Control register controls the PCI Express Link specific parameters. The link control register needs some default values setup by the local host. This register has been created to provide a back door path to program the link control register from the primary side. The link control register on the secondary side of the NTB is located at Section 3.20.3.
PCI Express Non-Transparent Bridge Register:LNKCON Bus:0 Device:3 Function:0 Offset:1A0h Bit Attr Default Description 04 RWL 0b Link Disable This bit is not applicable and is reserved for Endpoints Note: Appears to SW as RO 03 RO 0b Read Completion Boundary Set to zero to indicate IIO could return read completions at 64B boundaries Note: NTB is not PCIE compliant in this respect. NTB is only capable of 64B RCB.
PCI Express Non-Transparent Bridge 3.19.4.25 LNKSTS: PCI Express Link Status Register The PCI Express Link Status register provides information on the status of the PCI Express Link such as negotiated width, training etc. The link status register needs some default values setup by the local host. This register has been created to provide a back door path to program the link status from the primary side. The link status register on the secondary side of the NTB is located at Section 3.20.3.
PCI Express Non-Transparent Bridge Register:LNKSTS Bus:0 Device:3 Function:0 Offset:1A2h Bit Attr Default Description 11 RO 0 Link Training This field indicates the status of an ongoing link training session in the PCI Express port 0: LTSSM has exited the recovery/configuration state 1: LTSSM is in recovery/configuration state or the Retrain Link was set but training has not yet begun. The IIO hardware clears this bit once LTSSM has exited the recovery/ configuration state.
PCI Express Non-Transparent Bridge 3.19.4.26 SLTCAP: PCI Express Slot Capabilities Register The Slot Capabilities register identifies the PCI Express specific slot capabilities. Register:SLTCAP Bus:0 Device:3 Function:0 Offset:1A4h Bit Attr Default 31:19 RWO 0h Physical Slot Number This field indicates the physical slot number of the slot connected to the PCI Express port and is initialized by BIOS. 18 RO 0h Command Complete Not Capable: IIO is capable of command complete interrupt.
PCI Express Non-Transparent Bridge Register:SLTCAP Bus:0 Device:3 Function:0 Offset:1A4h Bit 4 3 2 1 0 Attr RWO RWO RWO RWO RWO Default Description 0h Power Indicator Present This bit indicates that a Power Indicator is implemented for this slot and is electrically controlled by the chassis.
PCI Express Non-Transparent Bridge 3.19.4.27 SLTCON: PCI Express Slot Control Register The Slot Control register identifies the PCI Express specific slot control parameters for operations such as Hot-plug and Power Management.
PCI Express Non-Transparent Bridge Register:SLTCON Bus:0 Device:3 Function:0 Offset:1A8h Bit 5 4 3 2 1 0 Attr RW RW RW RW RW RW Default Description 0h Hot-plug Interrupt Enable When set to 1b, this bit enables generation of Hot-Plug MSI interrupt (and not wake event) on enabled Hot-Plug events, provided ACPI mode for hotplug is disabled.
PCI Express Non-Transparent Bridge 3.19.4.28 SLTSTS: PCI Express Slot Status Register The PCI Express Slot Status register defines important status information for operations such as Hot-plug and Power Management. Register:SLTSTS Bus:0 Device:3 Function:0 Offset:1AAh Bit Attr Default 15:9 RsvdZ 0h Reserved. 0h Data Link Layer State Changed This bit is set (if it is not already set) when the state of the Data Link Layer Link Active bit in the Link Status register changes.
PCI Express Non-Transparent Bridge Register:SLTSTS Bus:0 Device:3 Function:0 Offset:1AAh Bit Attr Default Description Presence Detect Changed 3 RW1C 0h This bit is set by the IIO when a Presence Detect Changed event is detected. It is subsequently cleared by software after the field has been read and processed. On-board logic per slot must set the VPP signal corresponding this bit inactive if the FF/system does not support out-of-band presence detect.
PCI Express Non-Transparent Bridge 3.19.4.29 ROOTCON: PCI Express Root Control Register The PCI Express Root Control register specifies parameters specific to the root complex port. Note: Since this PCI Express port can be configured as RP or NTB when configured as NTB register is moved from standard location and used to unable error reporting for upstream notification to the local host that is physically attached to the NTB.
PCI Express Non-Transparent Bridge Register:ROOTCON Bus:0 Device:3 Function:0 Offset:1ACh Bit Attr Default Description 0 RW 0b System Error on Correctable Error Enable This field controls notifying the internal core error logic of the occurrence of a correctable error in the device. The internal core error logic of IIO then decides if/how to escalate the error further (pins/message etc.). See Section 11.
PCI Express Non-Transparent Bridge 3.19.4.30 DEVCAP2: PCI Express Device Capabilities 2 Register NTB Primary is a RCiEP but needs to have some capabilities associated with a RP so that transactions are guaranteed to complete. This register controls transactions sent from local CPU to an external device through the PCIE NTB port.
PCI Express Non-Transparent Bridge 3.19.4.31 DEVCTRL2: PCI Express Device Control 2 Register Register:DEVCTRL2 Bus:0 Device:3 Function:0 Offset:1B8h Bit Attr Default 15:6 RO 0h 5 RW 0 Alternative RID Interpretation (ARI) Enable - When set to 1b, ARI is enabled for the NTB EP. Note: The BIOS must leave this bit at its default value.
PCI Express Non-Transparent Bridge 3.19.4.32 LNKCON2: PCI Express Link Control Register 2 Register:LNKCON2 Bus:0 Device:3 Function: 0 Offset:1C0h Bit Attr Default Description 15:13 RO 0 Reserved 12 RWS 0 Compliance De-emphasis – This bit sets the de-emphasis level in Polling.Compliance state if the entry occurred due to the Enter Compliance bit being 1b. Encodings: 1b -3.
PCI Express Non-Transparent Bridge 3.19.4.33 LNKSTS2: PCI Express Link Status 2 Register The PCI Express Link Status 2 register provides information on the status of the PCI Express Link current De-emphasis level and other definition is currently reserved. Register:LNKSTS2 Bus:0 Device:3 Function:0 Offset:1C2h Bit Attr Default 15:01 RO 0h Reserved 0b Current De-emphasis Level: When the Link is operating at 5 GT/s speed, this bit reflects the level of de-emphasis. Encodings: 1b -3.
PCI Express Non-Transparent Bridge 3.19.4.40 XPUNCERRSEV - XP Uncorrectable Error Severity Register BDF 030 Offset 210H. This register exist in both RP and NTB modes. It is documented in RP Section 3.4.5.26, “XPUNCERRSEV - XP Uncorrectable Error Severity Register”. See Volume 2 of the Datasheet. 3.19.4.41 XPUNCERRPTR - XP Uncorrectable Error Pointer Register BDF 030 Offset 214H. This register exist in both RP and NTB modes. It is documented in RP Section 3.4.5.
PCI Express Non-Transparent Bridge 3.20 PCI Express Configuration Registers (NTB Secondary Side) 3.20.1 Configuration Register Map (NTB Secondary Side) This section covers the NTB secondary side configuration space registers. When configured as an NTB there are two sides to discuss for configuration registers.
PCI Express Non-Transparent Bridge DID VID 00h PCISTS PCICMD 04h TABLEOFF_BIR RID 08h PBAOFF_BIR CLSR 0Ch CCR BIST HDR PLAT 10h MSIXMSGCTRL MSIXNTPTR MSIXCAPID 80h 84h 88h 8Ch PXPCAP PXPNXTPTR PXPCAPID 90h SB01BASE 14h 18h DEVCAP DEVSTS 94h DEVCTRL 98h SB23BASE 1Ch 20h LNKCAP LNKSTS 9Ch LNKCON A0h SB45BASE A4h 24h SID SUBVID CAPPTR 28h A8h 2Ch ACh 30h B0h 34h DEVCAP2 38h MAXLAT MINGNT INTPIN INTL DEVCTRL2 BCh 40h C0h 44h C4h 48h C8h 4Ch CCh 50h D0
PCI Express Non-Transparent Bridge 3.20.2 Standard PCI Configuration Space (0x0 to 0x3F) - Type 0 Common Configuration Space This section covers the secondary side registers in the 0x0 to 0x3F region that are common to Bus M, Device 0. The Primary side of the NTB was discussed in the previous section and is located on NTB Bus 0, Device 3. Comments at the top of the table indicate what devices/functions the description applies to.
PCI Express Non-Transparent Bridge 3.20.2.3 PCICMD: PCI Command Register (Dev#N, PCIE NTB Sec Mode) This register defines the PCI 3.0 compatible command register values applicable to PCI Express space. Register:PCICMD Bar:PB01BASE + 500h + Offset, SB01BASE + 500h + Offset Bus:M Device:0 Function: 0 Offset:04h Bit Attr Default 15:11 RV 00h Description Reserved. (by PCI SIG) 10 RW 0 INTxDisable: Interrupt Disable Controls the ability of the PCI-Express port to generate INTx messages.
PCI Express Non-Transparent Bridge Register:PCICMD Bar:PB01BASE + 500h + Offset, SB01BASE + 500h + Offset Bus:M Device:0 Function: 0 Offset:04h Bit Attr Default 3 RO 0 Special Cycle Enable Not applicable to PCI Express must be hardwired to 0. 0 Bus Master Enable 1: When this bit is Set, the PCIE NTB will forward Memory Requests that it receives on its primary internal interface to its secondary external link interface.
PCI Express Non-Transparent Bridge 3.20.2.4 PCISTS: PCI Status Register The PCI Status register is a 16-bit status register that reports the occurrence of various events associated with the primary side of the “virtual” PCI-PCI bridge embedded in PCI Express ports and also primary side of the other devices on the internal IIO bus.
PCI Express Non-Transparent Bridge Register:PCISTS Bar:PB01BASE + 500h + Offset, SB01BASE + 500h + Offset Bus:M Device:0 Function: 0 Offset:06h Bit Attr Default 11 RW1C 0 10:9 RO 0h Description Signaled Target Abort This bit is set when the NTB port forwards a completer abort (CA) completion status from the primary interface to the secondary interface. DEVSEL# Timing Not applicable to PCI Express. Hardwired to 0.
PCI Express Non-Transparent Bridge 3.20.2.5 RID: Revision Identification Register This register contains the revision number of the IIO. The revision number steps the same across all devices and functions i.e. individual devices do not step their RID independently. IIO supports the CRID feature where in this register’s value can be changed by BIOS. See Section 3.2.2, “Compatibility Revision ID” in Volume 2 of the Datasheet for details.
PCI Express Non-Transparent Bridge 3.20.2.7 CLSR: Cacheline Size Register Register:CLSR Bar:PB01BASE + 500h + Offset, SB01BASE + 500h + Offset Bus:M Device:0 Function: 0 Offset:0Ch 3.20.2.8 Bit Attr Default Description 7:0 RW 0h Cacheline Size This register is set as RW for compatibility reasons only. Cacheline size for IIO is always 64B. IIO hardware ignore this setting. PLAT: Primary Latency Timer This register denotes the maximum time slice for a burst transaction in legacy PCI 2.
PCI Express Non-Transparent Bridge 3.20.2.10 BIST: Built-In Self Test This register is used for reporting control and status information of BIST checks within a PCI Express port. It is not supported in Intel® Xeon® processor C5500/C3500 series. Register:BIST Bar:PB01BASE + 500h + Offset, SB01BASE + 500h + Offset Bus:M Device:0 Function: 0 Offset:0Fh 3.20.2.11 Bit Attr Default 7:0 RO 0h Description BIST_TST: BIST Tests Not supported.
PCI Express Non-Transparent Bridge 3.20.2.12 SB23BASE: Secondary BAR 2/3 Base Address (PCIE NTB Mode) This register is BAR 2/3 for the secondary side of the NTB. This configuration register can be modified via configuration transaction from the secondary side of the NTB and can also be modified from the primary side of the NTB via MMIO transaction to Section 3.21.1.10, “SBAR2BASE: Secondary BAR 2/3 Base Address” Note: SW must program upper DW first and then lower DW.
PCI Express Non-Transparent Bridge 3.20.2.13 SB45BASE: Secondary BAR 4/5 Base Address This register is BAR 4/5 for the secondary side of the NTB. This configuration register can be modified via configuration transaction from the secondary side of the NTB and can also be modified from the primary side of the NTB via MMIO transaction to Section 3.21.1.11, “SBAR4BASE: Secondary BAR 4/5 Base Address” Note: SW must program upper DW first and then lower DW.
PCI Express Non-Transparent Bridge 3.20.2.14 SUBVID: Subsystem Vendor ID (Dev#3, PCIE NTB Sec Mode) This register identifies the vendor of the subsystem. Register:SUBVID Bar:PB01BASE + 500h + Offset, SB01BASE + 500h + Offset Bus:M Device:0 Function: 0 Offset:2Ch 3.20.2.15 Bit Attr Default Description 15:0 RWO 0000h Subsystem Vendor ID: This field must be programmed during boot-up to indicate the vendor of the system board.
PCI Express Non-Transparent Bridge 3.20.2.17 INTL: Interrupt Line Register The Interrupt Line register is used to communicate interrupt line routing information between initialization code and the device driver. This register is not used in newer OSes and is just kept as RW for compatibility purposes only. Register:INTL Bar:PB01BASE + 500h + Offset, SB01BASE + 500h + Offset Bus:M Device:0 Function: 0 Offset:3Ch 3.20.2.
PCI Express Non-Transparent Bridge 3.20.2.19 MINGNT: Minimum Grant Register . Register:INTPIN Bar:PB01BASE + 500h + Offset, SB01BASE + 500h + Offset Bus:M Device:0 Function: 0 Offset:3Eh 3.20.2.20 Bit Attr Default Description 7:0 RO 00h Minimum Grant: This register does not apply to PCI Express. It is hard-coded to “00”h. MAXLAT: Maximum Latency Register .
PCI Express Non-Transparent Bridge 3.20.3.3 MSICTRL: MSI Control Register Register:MSICTRL Bar:PB01BASE + 500h + Offset, SB01BASE + 500h + Offset Bus:M Device:0 Function:0 Offset:62h Bit Attr Default 15:9 RV 00h 8 RO 1b Per-vector masking capable This bit indicates that PCI Express ports support MSI per-vector masking.
PCI Express Non-Transparent Bridge Register:MSICTRL Bar:PB01BASE + 500h + Offset, SB01BASE + 500h + Offset Bus:M Device:0 Function:0 Offset:62h Bit 0 3.20.3.4 Attr RW Default Description 0b MSI Enable The software sets this bit to select platform-specific interrupts or transmit MSI messages. 0: Disables MSI from being generated. 1: Enables the PCI Express port to use MSI messages for RAS, provided bit 4 in Section 3.19.4.20, “MISCCTRLSTS: Misc.
PCI Express Non-Transparent Bridge Register:MSIUAR Bar:PB01BASE + 500h + Offset, SB01BASE + 500h + Offset Bus:M Device:0 Function:0 Offset:68h Bit 31:00 3.20.3.6 Attr RW Default Description 00000000h Upper Address MSB If the MSI Enable bit (bit 0 of the MSICTRL) is set, the contents of this register (if non-zero) specify the upper 32-bits of a 64-bit message address (AD[63::32]).
PCI Express Non-Transparent Bridge Table 92. MSI Vector Handling and Processing by IIO on Secondary Side Number of Messages enabled by Software Events IV[7:0] 1 PD[15:00] xxxxxxxx1 1. The term “xxxxxx” in the Interrupt vector denotes that software initializes them and IIO will not modify any of the “x” bits except the LSB as indicated in the table as a function of MMEN 3.20.3.7 MSIMSK: MSI Mask Bit Register The Mask Bit register enables software to disable message sending on a per-vector basis.
PCI Express Non-Transparent Bridge 3.20.3.9 MSIXCAPID: MSI-X Capability ID Register:MSIXCAPID Bar:PB01BASE + 500h + Offset, SB01BASE + 500h + Offset Bus:M Device:0 Function:0 Offset:80h 3.20.3.10 Bit Attr Default 7:0 RO 11h Description Capability ID Assigned by PCI-SIG for MSI-X. MSIXNXTPTR: MSI-X Next Pointer Register:MSIXNXTPTR Bar:PB01BASE + 500h + Offset, SB01BASE + 500h + Offset Bus:M Device:0 Function:0 Offset:81h 3.20.3.
PCI Express Non-Transparent Bridge Register:MSIXMSGCTRL Bar:PB01BASE + 500h + Offset, SB01BASE + 500h + Offset Bus:M Device:0 Function:0 Offset:82h Bit 10:00 3.20.3.12 Attr RO Default Description 003h Table Size System software reads this field to determine the MSI-X Table Size N, which is encoded as N-1. For example, a returned value of “00000000011” indicates a table size of 4. NTB table size is 4, encoded as a value of 003h The value in this field depends on the setting of Section 3.20.3.
PCI Express Non-Transparent Bridge 3.20.3.13 PBAOFF_BIR: MSI-X Pending Bit Array Offset and BAR Indicator Register default: 00005000h Register:PBAOFF_BIR Bar:PB01BASE + 500h + Offset, SB01BASE + 500h + Offset Bus:M Device:0 Function:0 Offset:88h Bit 31:03 02:00 3.20.3.14 Attr RO RO Default Description 00000A00h Table Offset MSI-X PBA Structure is at offset 20K from the SB01BASE BAR address. See Section 3.21.3.4, “SMSIXPBA: Secondary MSI-X Pending Bit Array Register” for details.
PCI Express Non-Transparent Bridge 3.20.3.15 PXPNXTPTR: PCI Express Next Pointer Register The PCI Express Capability List register enumerates the PCI Express Capability structure in the PCI 3.0 configuration space. Register:PXPNXTPTR Bar:PB01BASE + 500h + Offset, SB01BASE + 500h + Offset Bus:M Device:0 Function: 0 Offset:91h 3.20.3.16 Bit Attr Default 7:0 RWO E0h Description Next Ptr This field is set to the PCI PM capability.
PCI Express Non-Transparent Bridge 3.20.3.17 DEVCAP: PCI Express Device Capabilities Register The PCI Express Device Capabilities register identifies device specific information for the device. Register:DEVCAP Bar:PB01BASE + 500h + Offset, SB01BASE + 500h + Offset Bus:M Device:0 Function: 0 Offset:94h Bit Attr Default 31:29 Rsvd P 0h Reserved 28 RO 0b Function Level Reset Capability A value of 1b indicates the Function supports the optional Function Level Reset mechanism.
PCI Express Non-Transparent Bridge Register:DEVCAP Bar:PB01BASE + 500h + Offset, SB01BASE + 500h + Offset Bus:M Device:0 Function: 0 Offset:94h Bit 11:9 8:6 Attr RWO RWO Default Description 110b Endpoint L1 Acceptable Latency This field indicates the acceptable latency that an Endpoint can withstand due to the transition from L1 state to the L0 state. It is essentially an indirect measure of the Endpoint’s internal buffering.
PCI Express Non-Transparent Bridge 3.20.3.18 DEVCTRL: PCI Express Device Control Register (PCIE NTB Secondary) The PCI Express Device Control register controls PCI Express specific capabilities parameters associated with the device. Register:DEVCTRL Bar:PB01BASE + 500h + Offset, SB01BASE + 500h + Offset Bus:M Device:0 Function: 0 Offset:98h PCIE_ONLY Bit Attr Default 15 RsvdP 0h Description Reserved.
PCI Express Non-Transparent Bridge Register:DEVCTRL Bar:PB01BASE + 500h + Offset, SB01BASE + 500h + Offset Bus:M Device:0 Function: 0 Offset:98h PCIE_ONLY Bit 4 3 2 1 0 Attr RO RO RO RO RO Default Description 0 Enable Relaxed Ordering Not applicable since the NTB is never the originator of a TLP. This bit has no impact on forwarding of relaxed ordering attribute on peer requests. 0 Unsupported Request Reporting Enable Applies only to the PCI Express/DMI ports.
PCI Express Non-Transparent Bridge 3.20.3.19 DEVSTS: PCI Express Device Status Register The PCI Express Device Status register provides information about PCI Express device specific parameters associated with the device. Register:DEVSTS Bar:PB01BASE + 500h + Offset, SB01BASE + 500h + Offset Bus:M Device:0 Function: 0 Offset: 9Ah Bit Attr Default 15:6 RsvdZ 000h Description Reserved. Transactions Pending: Does not apply.
PCI Express Non-Transparent Bridge 3.20.3.20 LNKCAP: PCI Express Link Capabilities Register The Link Capabilities register identifies the PCI Express specific link capabilities Note: This register is a secondary view into the LNKCAP register. BIOS must set some RWO configuration bits prior to use. See Section 3.19.4.23, “LNKCAP: PCI Express Link Capabilities Register” .
PCI Express Non-Transparent Bridge Register:LNKCAP Bar:PB01BASE + 500h + Offset, SB01BASE + 500h + Offset Bus:M Device:0 Function:0 Offset:9Ch Bit Attr 14:12 RO 11:10 RO 9:4 RO 3:0 1. RO Default 011 11 001000b 0010b Description L0s Exit Latency This field indicates the L0s exit latency (i.e L0s to L0) for the PCI-Express port.
PCI Express Non-Transparent Bridge 3.20.3.21 LNKCON: PCI Express Link Control Register The PCI Express Link Control register controls the PCI Express Link specific parameters Note: This register is a secondary view into the LNKCAP register. Some additional controllability is available through the primary side equivalent register. See Section 3.19.4.24, “LNKCON: PCI Express Link Control Register” Note: In NTB/RP mode RP will program this register.
PCI Express Non-Transparent Bridge Register:LNKCON Bar:PB01BASE + 500h + Offset, SB01BASE + 500h + Offset Bus:M Device:0 Function:0 Offset:A0h Bit Attr Default Description 04 RsvdP 0b Link Disable This bit is not applicable and is reserved for Endpoints 03 RO 0b Read Completion Boundary Set to zero to indicate IIO could return read completions at 64B boundaries Note: NTB is not PCIE compliant in this respect. NTB is only capable of 64B RCB.
PCI Express Non-Transparent Bridge 3.20.3.22 LNKSTS: PCI Express Link Status Register Note: This register is a secondary view into the LNKSTS register. BIOS must set some registers prior to use. See Section 3.19.4.25, “LNKSTS: PCI Express Link Status Register” . The PCI Express Link Status register provides information on the status of the PCI Express Link such as negotiated width, training etc.
PCI Express Non-Transparent Bridge Register:LNKSTS Bar:PB01BASE + 500h + Offset, SB01BASE + 500h + Offset Bus:M Device:0 Function:0 Offset:A2h Bit 11 Attr RO Default 0 Description Link Training This field indicates the status of an ongoing link training session in the PCI Express port 0: LTSSM has exited the recovery/configuration state 1: LTSSM is in recovery/configuration state or the Retrain Link was set but training has not yet begun.
PCI Express Non-Transparent Bridge 3.20.3.23 DEVCAP2: PCI Express Device Capabilities Register 2 Register:DEVCAP2 Bar:PB01BASE + 500h + Offset, SB01BASE + 500h + Offset Bus:M Device:0 Function:0 Offset:B4h Bit Attr Default 31:6 RO 0h 5 RO 0 Alternative RID Interpretation (ARI) Capable - This bit is set to 1b indicating Root Port supports this capability.
PCI Express Non-Transparent Bridge Register:DEVCTRL2 Bar:PB01BASE + 500h + Offset, SB01BASE + 500h + Offset Bus:M Device:0 Function:0 Offset:B8h Bit Attr Default 15:5 RO 0h 4 3:0 February 2010 Order Number: 323103-001 RW RW Description Reserved 0 Completion Timeout Disable – When set to 1b, this bit disables the Completion Timeout mechanism for all NP tx that IIO issues on the PCIE/DMI link and in the case of Intel® QuickData Technology DMA, for all NP tx that DMA issues upstream.
PCI Express Non-Transparent Bridge 3.20.3.25 SSCNTL: Secondary Side Control This register provides secondary side control of NTB functions. . Register:SSCNTL Bar:PB01BASE + 500h + Offset, SB01BASE + 500h + Offset Bus:M Device:0 Function:0 Offset:D4h 3.20.3.26 Bit Attr Default Description 15:01 RO 0h Reserved 00 RW 0b NTB Secondary side - MSI-X Single Message Vector: This bit when set, causes only a single MSI-X message to be generated if MSI-X is enabled.
PCI Express Non-Transparent Bridge Register:PMCAP Bar:PB01BASE + 500h + Offset, SB01BASE + 500h + Offset Bus:M Device:0 Function:0 Offset:E0h 3.20.3.27 Bit Attr Default Description 19 RO 0b 18:16 RO 011b Version This field is set to 3h (PM 1.2 compliant) as version number for all PCI Express ports. 15:8 RO 00h Next Capability Pointer This is the last capability in the chain and hence set to 0. 7:0 RO 01h Capability ID Provides the PM capability ID assigned by PCI-SIG.
PCI Express Non-Transparent Bridge Register:PMCSR Bar:PB01BASE + 500h + Offset, SB01BASE + 500h + Offset Bus:M Device:0 Function:0 Offset:E4h Bit Attr Default 7:4 RsvdP 0h Reserved. No Soft Reset Indicates IIO does not reset its registers when transitioning from D3hot to D0. Note: This bit must be written by BIOS to a ‘1’ so that this register bit cannot be cleared. 3 RWO 1 2 RsvdP 0h Reserved.
PCI Express Non-Transparent Bridge 3.21 NTB MMIO Space NTB MMIO space consists of a shared set of MMIO registers (shadowed), primary side MMIO registers and secondary side MMIO registers. 3.21.1 NTB Shadowed MMIO Space All shadow registers are visible from the primary side of the NTB. Only some of the shadow registers are visible from the secondary side of the NTB. See each register description for visibility. Table 93.
PCI Express Non-Transparent Bridge Table 93. NTB MMIO Shadow Registers 74h F4h 78h F8h 7Ch FCh Secondary Link State - 1 bit (trained or untrained) (change generates interrupt) Table 94.
PCI Express Non-Transparent Bridge 3.21.1.1 PBAR2LMT: Primary BAR 2/3 Limit This register contains a value used to limit the size of the window exposed by 64-bit BAR 2/3 to a size less than the power-of-two expressed in the Primary BAR 2/3 pair. This register is written by the NTB device driver and will contain the formulated sum of the base address plus the size of the BAR. This final value equates to the highest address that will be accepted through this port.
PCI Express Non-Transparent Bridge 3.21.1.2 PBAR4LMT: Primary BAR 4/5 Limit This register contains a value used to limit the size of the window exposed by 64-bit BAR 4/5 to a size less than the power-of-two expressed in the Primary BAR 4/5 pair. This register is written by the NTB device driver and will contain the formulated sum of the base address plus the size of the BAR. This final value equates to the highest address that will be accepted through this port.
PCI Express Non-Transparent Bridge 3.21.1.3 PBAR2XLAT: Primary BAR 2/3 Translate This register contains a value used to direct accesses into the memory located on the Secondary side of the NTB made from the Primary side of the NTB through the window claimed by BAR 2/3 on the primary side. The register contains the base address of the Secondary side memory window.
PCI Express Non-Transparent Bridge 3.21.1.5 SBAR2LMT: Secondary BAR 2/3 Limit This register contains a value used to limit the size of the window exposed by 64-bit BAR 2/3 to a size less than the power-of-two expressed in the Secondary BAR 2/3 pair. This register is written by the NTB device driver and will contain the formulated sum of the base address plus the size of the BAR. This final value equates to the highest address that will be accepted through this port.
PCI Express Non-Transparent Bridge 3.21.1.6 SBAR4LMT: Secondary BAR 4/5 Limit This register contains a value used to limit the size of the window exposed by 64-bit BAR 4/5 to a size less than the power-of-two expressed in the Secondary BAR 4/5 pair. This register is written by the NTB device driver and will contain the formulated sum of the base address plus the size of the BAR. This final value equates to the highest address that will be accepted through this port.
PCI Express Non-Transparent Bridge 3.21.1.7 SBAR2XLAT: Secondary BAR 2/3 Translate This register contains a value used to direct accesses into the memory located on the Primary side of the NTB made from the Secondary side of the NTB through the window claimed by BAR 2/3 on the secondary side. The register contains the base address of the Primary side memory window. Note: NTB will translate full 64b range. Switch logic will perform address range checks for both normal and VT-d flows.
PCI Express Non-Transparent Bridge 3.21.1.8 SBAR4XLAT: Secondary BAR 4/5 Translate This register contains a value used to direct accesses into the memory located on the Primary side of the NTB made from the Secondary side of the NTB through the window claimed by BAR 4/5 on the secondary side. The register contains the base address of the Primary side memory window. Note: NTB will translate full 64b range. Switch logic will perform address range checks for both normal and VT-d flows.
PCI Express Non-Transparent Bridge 3.21.1.10 SBAR2BASE: Secondary BAR 2/3 Base Address This register is mirrored from the BAR 2/3 register pair in the Configuration Space of the Secondary side of the NTB. The register is used by the processor on the primary side of the NTB to examine and load the BAR 2/3 register pair on the Secondary side of the NTB.
PCI Express Non-Transparent Bridge 3.21.1.11 SBAR4BASE: Secondary BAR 4/5 Base Address This register is mirrored from the BAR 4/5 register pair in the Configuration Space of the Secondary side of the NTB. The register is used by the processor on the primary side of the NTB to examine and load the BAR 4/5 register pair on the Secondary side of the NTB.
PCI Express Non-Transparent Bridge 3.21.1.12 NTBCNTL: NTB Control This register contains Control bits for the Non-transparent Bridge device. Register:NTBCNTL Bar:PB01BASE, SB01BASE Offset:58h Bit Attr Default 31:11 RO 00h 10 09:08 07:06 05:04 RW RW Bar: Attr PB01BASE: RW else: RO RW 0b Description Reserved Crosslink SBDF Disable Increment This bit is only valid in NTB/NTB mode This bit determines if SBDF value on the DSD is incremented or not.
PCI Express Non-Transparent Bridge 03:02 Bar: Attr PB01BASE: RW else: RO 01 Bar: Attr PB01BASE: RW else: RO 00 Bar: Attr PB01BASE: RW else: RO February 2010 Order Number: 323103-001 00b BAR 2/3 Secondary to Primary Snoop Override Control This bit controls the ability to force all transactions within the Secondary BAR 2/3 window going from the Secondary side to the Primary side to be snoop/no-snoop independent of the ATTR field in the TLP header.
PCI Express Non-Transparent Bridge 3.21.1.13 SBDF: Secondary Bus, Device and Function This register contains the Bus, Device and Function for the secondary side of the NTB when PPD.Port Definition is configured as NTB/NTB Section 3.19.3.23, “PPD: PCIE Port Definition” . Note: The region between the two NTBs is in no mans land and does not matter what value BDF is set to, but the same value must be programmed in both NTBs on each side of the link.
PCI Express Non-Transparent Bridge Register:CBDF Bar:PB01BASE, SB01BASE Offset:5Eh 3.21.1.15 Bit Attr Default 15:8 RO 00h 7:3 RO 00000b 2:0 RO 000b Description Secondary Bus Value to be used for the Bus number for ID-based routing. Secondary Device Value to be used for the Device number for ID-based routing. Secondary Function Value to be used for the Function number for ID-based routing.
PCI Express Non-Transparent Bridge 3.21.1.16 PDBMSK: Primary Doorbell Mask This register is used to mask the generation of interrupts to the Primary side of the NTB. Register:PDBMSK Bar:PB01BASE, SB01BASE Offset:62h 3.21.1.17 Bit Attr 15:0 Bar: Attr PB01BASE: RW else: RO Default FFFFh Description Primary Doorbell Mask This register will allow software to mask the generation of interrupts to the processor on the Primary side of the NTB.
PCI Express Non-Transparent Bridge Register:USMEMMISS Bar:PB01BASE, SB01BASE Offset:70h 3.21.1.20 Bit Attr Default 15:0 RW 00h Description Upstream Memory Miss This register keeps a running count of misses to any of the 3 upstream memory windows on the secondary side of the NTB. The counter does not freeze at max count, it rolls over. SPAD[0 - 15]: Scratchpad Registers 0 - 15 This set of 16 registers, SPAD0 through SPAD15, are shared to both sides of the NTB.
PCI Express Non-Transparent Bridge 3.21.1.21 SPADSEMA4: Scratchpad Semaphore This register will allow software to share the Scratchpad registers. Register:SPADSEMA4 Bar:PB01BASE, SB01BASE Offset:C0h Bit Attr Default 31:01 RO 00h 00 R0TS W1TC Intel® Xeon® Processor C5500/C3500 Series Datasheet, Volume 1 294 0b Description Reserved Scratchpad Semaphore This bit will allow software to synchronize write ownership of the scratchpad register set. The processor will read the register.
PCI Express Non-Transparent Bridge 3.21.1.22 RSDBMSIXV70: Route Secondary Doorbell MSI-X Vector 7 to 0 This register is used to allow flexibility in the SDOORBELL Section 3.21.1.17, “SDOORBELL: Secondary Doorbell” bits 7 to 0 assignments to one of 4 MSI-X vectors.
PCI Express Non-Transparent Bridge 3.21.1.23 RSDBMSIXV158: Route Secondary Doorbell MSI-X Vector 15 to 8 This register is used to allow flexibility in the SDOORBELL Section 3.21.1.17, “SDOORBELL: Secondary Doorbell” bits 15 to 8 assignments to one of 4 MSI-X vectors.
PCI Express Non-Transparent Bridge 3.21.1.24 WCCNTRL: Write Cache Control Register This register is used for IIO write cache controlability Register:WCCNTRL Bar:PB01BASE, SB01BASE Offset:E0h Bit Attr Default 31:01 RO 0h Description Reserved WCFLUSH When set forces snap shot flush of the IIO write cache. This register can be set either by host write or inbound MMIO write. Note: 00 RW1S 0b This bit is cleared by hardware upon completion of write cache flush. Software cannot clear this register.
PCI Express Non-Transparent Bridge 3.21.1.26 B2BDOORBELL: Back-to-Back Doorbell This register is valid when in NTB/NTB configuration. This register is used by the processor on the primary side of the NTB to generate accesses to the PDOORBELL register on a second NTB whose Secondary side is connected to the Secondary side of this NTB.
PCI Express Non-Transparent Bridge 3.21.1.27 B2BBAR0XLAT: Back-to-Back BAR 0/1 Translate This register is valid when in NTB/NTB configuration. This register is used to set the base address where the back-to-back doorbell and scratchpad packets will be sent. This register must match the base address loaded into the BAR 0/1 pair on the opposite NTB, whose Secondary side in linked to the Secondary side of this NTB.
PCI Express Non-Transparent Bridge 3.21.2 MSI-X MMIO Registers (NTB Primary side) Primary side MSI-X MMIO registers reached via PB01BASE Table 95.
PCI Express Non-Transparent Bridge 3.21.2.1 PMSIXTBL[0-3]: Primary MSI-X Table Address Register 0 - 3 . Register:PMSIXTBLn Bar:PB01BASE, SB01BASE Offset:00002000h, 00002010h, 00002020h, 00002030h 3.21.2.2 Bit Attr Default Description 63:32 RW 00000000h MSI-X Upper Address Upper address bits used when generating an MSI-X. 31:02 RW 00000000h MSI-X Address System-specified message lower address.
PCI Express Non-Transparent Bridge 3.21.2.4 PMSIXPBA: Primary MSI-X Pending Bit Array Register Register:PMSIXPBA Bar:PB01BASE, SB01BASE Offset:00003000h Bit Attr Default 31:04 RO 0000h 03 RO 0b MSI-X Table Entry 03 (NTB) has a Pending Message. 02 RO 0b MSI-X Table Entry 02 (NTB) has a Pending Message. 01 RO 0b MSI-X Table Entry 01 (NTB) has a Pending Message. 00 RO 0b MSI-X Table Entry 00 (NTB) has a Pending Message.
PCI Express Non-Transparent Bridge 3.21.3 MSI-X MMIO registers (NTB Secondary Side) Secondary side MSI-X MMIO registers reached via PB01BASE (debug) and SB01BASE. These registers are valid when in NTB/RP configuration. Table 97.
PCI Express Non-Transparent Bridge 3.21.3.1 SMSIXTBL[0-3]: Secondary MSI-X Table Address Register 0 - 3 . Register:SMSIXTBLn Bar:PB01BASE, SB01BASE Offset:00004000h, 00004010h, 00004020h, 00004030h 3.21.3.2 Bit Attr Default Description 63:32 RW 00000000h MSI-X Upper Address Upper address bits used when generating an MSI-X. 31:02 RW 00000000h MSI-X Address System-specified message lower address.
PCI Express Non-Transparent Bridge 3.21.3.3 SMSIXVECCNTL[0-3]: Secondary MSI-X Vector Control Register 0 - 3 Register:SMSIXVECCNTLn Bar:PB01BASE, SB01BASE Offset:0000400Ch, 0000401Ch, 0000402Ch, 0000403Ch Bit Attr Default 31:01 RO 00000000h 00 3.21.3.4 RW 1b Description Reserved MSI-X Mask: When this bit is set, the NTB is prohibited from sending a message using this MSI-X Table entry.
Technologies 4.0 Technologies 4.1 Intel® Virtualization Technology (Intel® VT) Intel® VT comprises technology components to support virtualization of platforms based on Intel architecture microprocessors and chipsets. Intel® Virtualization Technology (Intel® VT-x) added hardware support in the processor to improve the virtualization performance and robustness.
Technologies 4.1.2 Intel® VT-x Features The processor core supports the following Intel® VT-x features: • Extended Page Tables (EPT) — Hardware-assisted page table virtualization. — Eliminates VM exits from guest OS to the VMM for shadow page-table maintenance. • Virtual Processor IDs (VPID) — Ability to assign a VM ID to tag processor core hardware structures (e.g. TLBs). — Avoids flushes on VM transitions to give a lower-cost VM transition time and an overall reduction in virtualization overhead.
Technologies 4.1.4 Intel® VT-d Features The processor supports the following Intel® VT-d features: • The Intel® Xeon® processor C5500/C3500 series also supports Intel® VT-d2, which is a superset of VT-d that provides improved performance.
Technologies 4.2.1 Intel® QuickData Technology Intel® QuickData Technology makes Intel® chipsets excel with Intel network controllers. The Intel® Xeon® processor C5500/C3500 series uses the third generation of the Intel® QuickData Technology. The Intel® Xeon® processor C5500/C3500 series supports Intel® QuickData Technology.
Technologies • Effective move BW of 2.5 GB/s (2.5 GB/s effective read + 2.5 GB/s effective write), calculated assuming descriptor batch size of 2 and a data payload size of 1460 B. • Raw BW of 5 GB/s read + 5 GB/s write. • Eight independent DMA Channels where each channel is compliant with Intel® QuickData Technology versions 3 and 2, but not compatible with version 1. • Data transfer between two system memory locations, or from system memory to MMIO. • CRC-32 Generation and Check. • Flow through CRC.
Technologies 4.3 Simultaneous Multi Threading (SMT) The Intel® Xeon® processor C5500/C3500 series supports SMT, which allows a single core to function as two logical processors. While some execution resources such as caches, execution units, and buses are shared, each logical processor has its own architectural state with its own set of general-purpose registers and control registers. This feature must be enabled via the BIOS and requires operating system support. 4.
IIO Ordering Model 5.0 IIO Ordering Model 5.1 Introduction The IIO spans two different ordering domains: one that adheres to producer-consumer ordering (PCI Express*) and one that is unordered (Intel® QPI). One of the primary functions of the IIO is to ensure that the producer-consumer ordering model is maintained in the unordered, Intel® QPI domain. This section describes the rules that are required to ensure that both PCI Express and Intel® QPI ordering is preserved.
IIO Ordering Model Table 99. Ordering Term Definitions (Sheet 2 of 2) Term 5.2 Definition Inbound Read Completion The completion for an inbound read. For example, the read data which results in a PCI Express device read to main memory. While the data flows outbound, the completion is still for an inbound read. Inbound Write A write issued toward an Intel® QPI component. This can be a write issued by a PCI Express device. An obvious example is a PCI Express device writing main memory.
IIO Ordering Model RULE 7: If an inbound read completes with multiple sub-completions (e.g. a cacheline at a time), those sub-completions must be returned on PCI Express in linearly increasing address order. The above rules apply whether the transaction is coherent or non-coherent. Some regions of memory space are considered non-coherent (e.g. the No Snoop attribute is set). The IIO will order all transactions regardless of its destination.
IIO Ordering Model 5.2.3 Inbound Ordering Rules Summary Table 100 indicates an ordering relationship between two inbound transactions as implemented in the IIO and summarizes the inbound ordering rules described in previous sections. Yes The second transaction (row) allowed to pass the first (column). No The second transaction not be allowed to pass the first transaction. This is may be required to satisfy the Producer-Consumer strong ordering model or may be the implementation choice for the IIO.
IIO Ordering Model RULE 1: Inbound non-posted completions must be allowed to progress past stalled outbound non-posted requests. RULE 2: Outbound posted requests must be allowed to progress past stalled outbound non-posted requests. This rule prevents deadlocks by guaranteeing forward progress. Consider the case when the outbound queues are entirely filled with read requests and likewise, the inbound queues are also filled with read requests.
IIO Ordering Model Table 101. Outbound Data Flow Ordering Rules Outbound Write or Message Request Outbound Read Request Outbound Configuration Write Request Inbound Read Completion Outbound Write or Message Request No1 Yes Yes Yes Outbound Read Request No No No Yes Outbound Configuration or I/O Write Request No No No Yes Inbound Read Completion No Yes Yes Row Pass Column? 1. 2. Yes2 No3 1. A Memory Write or Message Request may not pass any other Memory Write or Message Request.
IIO Ordering Model 5.4.3 Remote Peer-to-Peer In the initiating IIO, a remote peer-to-peer transaction follows the same ordering rules as inbound transactions destined to main memory. In the target IIO, a remote peer-topeer transaction follows the same ordering rules as outbound transactions destined to an I/O device. RULE 1: Similar to peer to peer write requests, the IIO must serialize remote peer-topeer read completions. 5.
IIO Ordering Model 5.6 Configuration Register Ordering Rules The IIO implements legacy PCI configuration space registers. Legacy PCI configuration registers are accessed with NcCfgRd and NcCfgWr transactions (using PCI Bus, Device Function) received on the Intel® QPI interface. For PCI configuration space, the ordering requirements are the same as standard, nonposted configuration cycles on PCI. See Section 5.2.1 and Section 5.3.1 for details.
System Address Map 6.0 System Address Map This chapter provides a basic overview of the system address map and describes how the IIO comprehends and decodes the various regions in the system address map. The term “IIO” in this chapter refers to the integrated IO module of Intel® Xeon® processor C5500/C3500 series. This chapter does not provide the full details of the platform system address space as viewed by software and it also does not provide the details of processor address decoding.
System Address Map 6.1 Memory Address Space Figure 63 shows the system memory address space. There are three basic regions of memory address space in the system: address below 1 MB, address between 1 MB and 4 GB, and address above 4 GB. These regions are described in the following sections. Throughout this section, there will be references to the subtractive decode port. It refers to the port of the IIO that is attached to the PCH or provides a path towards the PCH.
System Address Map 6.1.1 System DRAM Memory Regions Address Region From To 640KB DOS Memory 000_0000_0000 000_0009_FFFF 1MB to Top-of-low-memory 000_0010_0000 TOLM Bottom-of-high-memory to Top-of-high-memory 4 GB TOHM These address ranges are always mapped to system DRAM memory, regardless of the system configuration. The top of main memory below 4 G is defined by the Top of Low Memory (TOLM). Memory between 4 GB and TOHM is extended system memory.
System Address Map 6.1.2.1 VGA/SMM Memory Space Address Region From To VGA 000_000A_0000 000_000B_FFFF This legacy address range is used by video cards to map a frame buffer or a characterbased video buffer. By default, accesses to this region are forwarded to main memory by the processor. However, once firmware figures out where the VGA device is in the system, it sets up the processor’s source address decoders to forward these accesses to the appropriate IIO.
System Address Map 6.1.3 Address Region Between 1 MB and TOLM This region is always allocated to system DRAM memory. Software must set up one of the coarse memory decode ranges that IIO uses for inbound system memory decoding to include this address range. The IIO will forward inbound accesses to this region to system memory (unless any of these access addresses fall within a protected dram range).
System Address Map 6.1.6 Memory Address Range TOLM – 4 GB 6.1.6.1 PCI Express Memory Mapped Configuration Space (PCI MMCFG) This is the system address region that is allocated for software to access the PCI Express Configuration Space. This region is relocatable below 4 GB by BIOS/firmware. 6.1.6.2 MMIOL Address Region From To MMIOL GMMIOL.Base GMMIOL.Limit This region is used for PCIE device memory addressing below 4 GB.
System Address Map 6.1.6.4 HPET/Others Address Region From To HPET/Others FED0_0000 FEDF_FFFF This region covers the High performance event timers in the PCH. All inbound/peer-topeer accesses to this region are completer aborted by the IIO. Outbound non-locked Intel® QPI accesses (that is, accesses that happen when Intel® QPI quiescence is not established) to the FED4_0xxx region are converted by IIO before forwarding to legacy DMI port.
System Address Map 6.1.7 Address Regions above 4 GB 6.1.7.1 High System Memory Address Region From To High System Memory 4 GB TOHM This region is used to describe the address range of system memory above the 4GB boundary. The IIO forwards all inbound accesses to this region to DRAM, unless any of these access addresses are also marked protected. See GENPROTRANGE1.BASE and GENPROTRANGE2.BASE registers.
System Address Map 6.2 IO Address Space There are four classes of I/O addresses that are specifically decoded by the platform: • I/O addresses used for VGA controllers. • I/O addresses used for ISA aliasing. • I/O addresses used for the PCI Configuration protocol - CFC/CF8. • I/O addresses used by downstream PCI/PCIE IO devices, typically legacy devices. This space is divided amongst the IIOs in the system. Each IIO can be associated with an IO range.
System Address Map 6.3 IIO Address Map Notes 6.3.1 Memory Recovery When software recovers an underlying DRAM memory region that resides below the 4 GB address line that is used for system resources like firmware, local APIC, and IOAPIC, etc. (the gap below 4 GB address line), it needs to make sure that it does not create system memory holes whereby all the system memory cannot be decoded with two contiguous ranges.
System Address Map non legacy IIO, the Intel® QPI port is the subtractive decode port. Thus all subtractively decoded transactions will eventually target the PCH. — The SUBDECEN bit in the IIO Miscellaneous Control Register (IIOMISCCTRL) sets the subtractive port of the IIO. — Virtual peer-to-peer bridge decoding related registers with their associated control bits (e.g.
System Address Map 6.4.1.2 FWH Decoding FWH accesses are allowed only from a CPU. Accesses from SMBus or PCIe are not supported. All FWH addresses (4 GB:4 GB-16 MB) and (1 MB:1 MB-128 K) that do not positively decode to the IIO’s PCIe ports, are subtractively forwarded to its legacy decode port.
System Address Map Table 102. Outbound Target Decoder Entries Target Decoder Entry Address Region Comments VGA (Memory space 0xA_0000 - 0xB_FFFF and IO space 0x3B0 - 0x3BB and 0x3C0 - 0x3DF) 4+11 Fixed. TPM/LT/FW ranges (E/F segs and 4G-16M to 4G) 1 Fixed. MMIOL 4 Variable. From P2P Bridge Configuration Register Space I/OxAPIC 4 Variable. From P2P Bridge Configuration Register Space MMIOH 4 Variable.
System Address Map 6.4.1.6 Summary of Outbound Memory/IO/Configuration Decoding Throughout the tables in this section, a reference to a PCIe port generically refers to a standard PCIe port or a DMI port. Note: Intel® Xeon® processor C5500/C3500 series will support configurations cycles that originate only from the CPU. For Intel® Xeon® processor C5500/C3500 series’s NTB, inbound CFG is support for access to the Secondary configuration registers. Table 103.
System Address Map Table 104. Decoding of Outbound Configuration Requests from Intel® QPI and Decoding of Outbound Peer-to-Peer Completions from Intel® QPI Address Range Bus 0 Bus 1-255 IIO Behavior Conditions Bus 0 and legacy IIO and device number matches one of internal device numbers Forward to that internal device.
System Address Map 6.4.2 Inbound Address Decoding This section covers the decoding that is done on any transaction that is received on a PCIE or DMI port or any transaction that originates from the Intel®QuickData Technology DMA port. 6.4.2.1 Overview • All inbound addresses that fall above the top of Intel® QPI physical address limit are flagged as errors by the IIO. Top of Intel® QPI physical address limit is dependent on the Intel® QPI profile.
System Address Map Figure 65. Intel® Xeon® Processor C5500/C3500 Series Only: Peer-to-Peer Illustration Peer-to-Peer (DP System) Intel Xeon Processor C5500/C300 Series QPI CPU CPU Internal QPI Internal QPI IIO (Legacy IIO) IIO Remote P2P IO..x IO..
System Address Map 6.4.2.2 Summary of Inbound Address Decoding Table 106, “Inbound Memory Address Decoding” summarizes the IIO behavior on inbound memory transactions from any PCIe port. This table is only intended to show the routing of transactions based on the address. It is not intended to show the details of several control bits that govern forwarding of memory requests from a given PCI Express port. See the PCI Express Base Specification, Revision 2.
System Address Map Table 106. Inbound Memory Address Decoding (Sheet 2 of 2) Address Range Conditions DRAM Memory holes and other non-existent regions • {4G <= Address <= TOHM (OR) 0 <= Address <= TOLM } AND address does not decode to any socket in Intel® QPI source decoder • Address > TOCM • When VT-d translation enabled, and guest address greater than 2^GPA_LIMIT IIO Behavior Master Abort Forward to subtractive decode port for legacy Intel® Xeon® processor C5500/C3500 series.
Interrupts 7.0 Interrupts 7.1 Overview This chapter describes how interrupts are handled in the IIO module. See the Software Developers Manual for details on how the CPUs process interrupts. The IIO module supports both MSI and legacy PCI interrupts from its PCI Express* ports. MSI interrupts received from PCI Express are forwarded directly to the processor socket.
Interrupts An IIO is not always guaranteed to have its DMI port enabled for legacy. When an IIO’s DMI port is disabled for legacy in non-legacy IIOs, then it has to route the INTx messages it receives from its downstream PCI Express ports, to its coherent interface, provided they are not serviced via the integrated I/OxAPIC. 7.2.1 Integrated I/OxAPIC The integrated I/OxAPIC in IIO converts legacy PCI Express interrupt messages into MSI interrupts.
Interrupts Table 108. I/OxAPIC Table Mapping to PCI Express Interrupts (Sheet 2 of 2) I/OxAPIC Table Entry# Interrupt Source # in Table 107 PCI Express Virtual Wire Type1 10 2 INTD 11 3 INTA 12 3 INTB 13 3 INTD 14 4 INTA 15 4 INTB 16 4 INTC 17 5 INTB 18 5 INTC 19 5 INTD 20 6 INTA 21 6 INTC 22 1 INTC 23 1 INTB 1.
Interrupts determines further action. This guarantees that any MSI generated from the integrated I/OxAPIC, or from the I/OxAPIC in PCH, if the integrated I/OxAPIC is disabled, will be ordered behind the memory write A, guaranteeing producer/consumer sanity. 7.2.3 INTR_Ack/INTR_Ack_Reply Messages INTR_Ack and INTR_Ack_Reply messages on DMI and IntAck on Intel® QuickPath Interconnect support legacy 8259-style interrupts required for system boot operations.
Interrupts Table 109. MSI Address Format when Remapping Disabled (Sheet 2 of 2) Bits Description 3 Redirection Hint: This bit allows the interrupt message to be directed to one among many targets, based on chipset redirection algorithm. 0 = The message will be delivered to the agent (CPU) listed in bits 19:4 1 = The message will be delivered to an agent based on the IIO redirection algorithm and the scope the interrupt as specified in the interrupt address.
Interrupts Table 112. MSI Data Format when Remapping is Enabled Bits 31:16 15:0 Description Reserved - IIO hardware checks for this field to be 0 (note that this checking is done only when remapping is enabled Sub Handle All PCI Express devices are required to support MSI. The IIO converts memory writes to this address (both PCI Express and internal sources) as a IntLogical, IntPhysical transactions on the Intel® QuickPath Interconnect.
Interrupts Figure 66. Interrupt Transformation Table Entry (IRTE) The Destination ID shown in the above illustration becomes the APICID on the Intel® QuickPath Interconnect interrupt packet. 7.3.2 MSI Forwarding: IA32 Processor-based Platform IA-32 interrupts have two modes: legacy mode and extended mode. Legacy mode has been supported in all chipsets to date. Extended mode is a new mode that allows for scaling beyond 60/255 threads in logical/physical mode operation.
Interrupts • The IIO adds a value of 2 to the original selected APICs address bit location. If the APIC corresponding to modulo 8 of that value is also not a valid target, then the IIO adds a value of 4 to the previous value and takes the modulo 8 of the resulting value. If that corresponding APIC is also not a valid target, then, • The IIO adds a value of 3 to the original selected APICs address bit location.
Interrupts APICs that were not specifically addressed will drop the message. There are two per core and one per thread, yielding eight local APICs in Intel® Xeon® processor C5500/ C3500 series when all four cores with SMT are enabled. 7.5 Platform Interrupts General Purpose Event (GPE) interrupts are generated as a result of hot plug and power management events. GPE interrupts are conveyed as VLW messages routed to the IOxAPIC within the PCH.
Interrupts 7.6.1 Legacy Interrupt Handled By IIO Module IOxAPIC When an INTx interrupt is enabled within the IIO module IOxAPIC, then the IIO module IOxAPIC may be programmed to deliver the legacy interrupt depending on the mask : Mask DRTPCH Behavior 0 X Convert to MSI 1 0 Forward INTx to PCH 1 1 Pend INTx in IOxAPIC There is no mode in which the integrated IOxAPIC delivers a legacy interrupt directly to the CPU.
Power Management 8.0 Power Management 8.1 Introduction Intel® Xeon® processor C5500/C3500 series power management is compatible with the PCI Bus Power Management Interface Specification, Revision 1.1 (referenced as PCI-PM). It is also compatible with the Advanced Configuration and Power Interface (ACPI) Specification, Revision 2.0b. This chapter provides information on the following power management topics: 8.1.
Power Management Figure 67. ACPI Power States in G0, G1, and G2 States System States Stop Grant S1 Soft Off S5 Idle Time Suspend to RAM S3 Wake Event S0 S4 Suspend to Disk Voltage/Frequency Combination P0 C-States Higher Power C0 P1 Processor States Lower Power G0 State: System State S0. Core State can be C0...Cx. In C0 state, P states can be P0...Pn G1 State: System State can be S1, S3 or S4 G2 State: System state will be S5 G3 State: Power Off 8.1.
Power Management Table 113. Platform System States (Sheet 2 of 2) System State Description S3 Suspend to RAM (STR) [Supported] This is also known as Standby. CPU, and PCI reset. All context can be lost except memory. This state is commonly known as “Suspend”. S4 Suspend to Disk (STD) [Supported] CPU, PCI and Memory reset. The S4 state is similar to S3 except that the system context is saved to disk rather that main memory. This state is commonly known as “Hibernate”. Self Refresh is not required.
Power Management 8.1.6 DMI States Table 116. DMI States State Description L0 Full on – Active transfer state. L0s First Active Power Management low power state – Low exit latency. L1a L1a is active state L1 in the DMI Specification. L3 Lowest power state (power-off) – Longest exit latency. 8.1.7 Intel® QPI States Table 117. Intel® QPI States State Description L0s First Active Power Management low power state – Low exit latency. L1 Lowest Active Power Management - Longer exit latency.
Power Management 8.1.10 Supported DMI Power States The transitions to and from the following power management states are supported on the DMI link: Table 120.
Power Management • Because there is low transition latency between P-states, a significant number of transitions per second are possible. • The highest frequency/voltage operating point is known as the highest frequency mode (HFM). • The lowest frequency/voltage operating point is known as the lowest frequency mode (LFM). 8.2.2 Low-Power Idle States When the processor is idle, low-power idle states (C-states) are used to save power. More power savings actions are taken for numerically higher C-states.
Power Management Entry and exit of the C-States at the thread and core level are shown in Figure 69. Figure 69. Thread and Core C-State Entry and Exit C0 MWAIT(C1), HLT (C1E Enabled) MWAIT(C6), P_LVL3 I/O Read MWAIT(C3), P_LVL2 I/O Read C1E C3 C6 While individual threads can request low power C-states, power-saving actions only take place after the core C-state is resolved. The processor automatically resolves Core C-states.
Power Management Note: The P_LVLx I/O Monitor address needs to be set up before using the P_LVLx I/O read interface. Each P-LVLx is mapped to the supported MWAIT(Cx) instruction as follows. Table 122. P_LVLx to MWAIT Conversion P_LVLx MWAIT(Cx) Notes P_LVL2 MWAIT(C3) The P_LVL2 base address is defined in the PMG_IO_CAPTURE MSR. P_LVL3 MWAIT(C6) C6.
Power Management While a core is in C1E state, it processes bus snoops and snoops from other threads. For more information on C1E, see Section 8.2.5.2. 8.2.4.3 Core C3 State Individual threads of a core can enter the C3 state by initiating a P_LVL2 I/O read to the P_BLK or an MWAIT (C3) instruction. A core in C3 state flushes the contents of its instruction cache, data cache, and Mid-Level Cache (MLC) to the Last Level Cache (LLC), while maintaining its architectural state.
Power Management The processor exits a package C-state when a break event is detected. If DRAM was allowed to go into self-refresh in package C3 or C6 state, it will be taken out of selfrefresh. Depending on the type of break event, the processor does the following: • If a core break event is received, the target core is activated and the break event message is forwarded to the target core.
Power Management Figure 70. Package C-State Entry and Exit C0 MWAIT C1E C3 C6 MWAIT Note: The package C state resolves to the highest power dissipation C state of the cores. 8.2.5.1 Package C0 This is the normal operating state for the processor. The processor remains in the normal state when at least one of its cores is in the C0 or C1E state or when the platform has not granted permission to the processor to go into a low power state.
Power Management 8.2.5.4 Package C6 State A processor enters the package C6 low power state when: • All cores are in C6 and the processor has been granted permission by the platform. In the package C6 state, all cores save their architectural state and have their core voltages reduced. The LLC is still powered and snoopable in this state. The processor remains in package C6 state as long as any part of the LLC is still active. 8.
Power Management The target behavior is to enter self-refresh for C3 and above states as long as there are no memory requests to service. The target usage is shown in Table 124. Table 124. Targeted Memory State Conditions Mode 8.3.2.3 Memory State C0, C1E Dynamic memory rank power down based on idle conditions. C3, C6 Dynamic memory rank power down based on idle conditions If there are no memory requests, then enter self-refresh. Otherwise use dynamic memory rank power down based on idle conditions.
Power Management The Intel® Xeon® processor C5500/C3500 series contains the following integrated ADR feature elements: • Level-sensitive pin, DDR_ADR, that triggers DDR3 self-refresh entry. • BIOS re-initialization of the memory controller triggers exit from DDR3 self-refresh. A complete and robust memory backup implementation involves many areas of the platform, e.g. hardware, BIOS, firmware OS, application, etc. The ADR mechanism described in this section does not provide such a solution by itself.
Power Management 8.3.2.5.2 Pin-Triggered Self-Refresh Entry ADR provides an external pin, DDR_ADR, that places the DDR3 into self-refresh. The critical data, now all in the DDR3, can be preserved as long as power is maintained to the DIMMs in self-refresh. The interface and sequence for placing DDR3 in self-refresh is part of the existing JEDEC DDR3 specification.
Power Management — The BIOS initializes the memory controller. Just before enabling CKE the BIOS redefines (pre-selected by the platform implementation) one of the Intel® 3420 chipset GPIO pins as an output and drives it active. The BIOS also writes the H[2:0]_REFRESH_THROTTLE_SUPPORT registers to arm the DDR_ADR pin to trigger the ADR entry sequence. (The arming BIOS writes must be timed such that the DDR3 is fully active - ~200 clocks after CKE rising.
Power Management Table 125. ADR Self-Refresh Entry Timing - AC Characteristics (CMOS 1.5 V) Symbol Parameter Min. Typ. Max. Unit Figure Notes TRFSH Time required to sample DDR_ADR input as active 8 Clock Figure 71 1 TDSR Time required to complete DIMM self-refresh activation from DDR_ADR input assertion (last CKE falling) 20 µs Figure 71 2 Notes: 1. Input is synchronized internally; no setup and hold times are required relative to clocks. 2.
Power Management entered an S-state. It must be cleared by SW. If SW were to leave this bit asserted, then the CPU will attempt to go to Sx by writing the Sleep Enable bit, do the RET, read the Wake Status bit as '1' and continue through the code before the PMReq(S1) had been delivered. When the PMReq(S1) is delivered the CPU will be executing some code and get halted in the middle. e. There will never be a C-state and S-state transition simultaneously.
Power Management 8.5 PCIe Power Management The IIO module supports the following link/device states and events: • L0s as receiver and transmitter. • L1 link state. • ASPM L1 link state. • L3 link state. • MSI or GPE event on power manage events internally generated (on a PCI Express port hotplug event) or received from PCI Express. • D0 and D3 hot states on a PCI Express port. • Wake from D3-hot on a hot plug event at a PCI Express port.
Power Management 8.7 Intel® QPI Power Management • L0 – Full performance, full power. • L1 – Turn off the link, longer latency back to L0. Note: There is no L0s support in the internal Intel® QPI link. 8.8 Intel® QuickData Technology Power Management The Intel® Xeon® processor C5500/C3500 series implements with Intel® QuickData Technology support different device power states.
Thermal Management 9.0 Thermal Management For thermal specifications and design guidelines, see the Intel® Xeon® Processor C5500/C3500 Series Thermal Mechanical Design Guide.
Reset 10.0 Reset 10.1 Introduction This chapter describes specific aspects of various hardware resets. 10.1.1 Types of Reset These are the types of reset: • Power Good Reset Power good reset is invoked by the de-assertion of the VCCPWRGOOD signal and is part of power-up reset. This reset clears sticky bits, clears all system states, and downloads fuses. Power-good reset destroys program state, can corrupt memory contents, and destroys error logs.
Reset Table 126. Core Trigger, Type, Domain Association 10.2 Misc. State Machines PCI Express Logic QPI Link Logic Layer DMI Logic Internal CPU Reset (RESETO_N) Signal CPU Warm Array Initialization Engines SMBus IIO.SYRE.CPURESET Fuse Downloader SMBus protocol Tri-statable Outputs PCI Express x SYRE.SAVCFG, QPILCL.1, Configuration Bit Link QPI IIO.BCTRL.
Reset The Intel® Xeon® processor C5500/C3500 series will support either the legacy or the non-legacy CPU being the boot processor. Selection of the boot processor is controlled by the BIOS. The Legacy IIO is always the firmware agent and either of the processors can fetch code from the flash. The processors may then use a semaphore register in the IIO to determine which processor is designated as the boot processor. 10.
Reset 10.4 Reset Timing Diagrams For clarification, the different voltages used in the system are: • VCC = Ungated power to core. • VTT = Ungated power to uncore, IIO. • VDD = Dram power. See the following figure. Intel® Xeon® Processor C5500/C3500 Series System Diagram Figure 72. Pwr Sply PS_ON_N VR 11.
Reset (from Intel® QPI, DMI, or PCI-Express) may hang interfaces that are not cleared by a warm reset. • System activity is initiated by a request from a processor link. No I/O devices will initiate requests until configured by a processor to do so. The requirements for DDR_DRAMPWROK assertion are: • Signal must be monotonic. • 100 ns minimum delay between VDDQ @ 1.425 V to DDR_DRAMPWROK @ Vihminspec (0.627 V). • DDR_DRAMPWROK must be asserted no later than VCCPWRGOOD assertion.
Reliability, Availability, Serviceability (RAS) 11.0 Reliability, Availability, Serviceability (RAS) 11.1 IIO RAS Overview This chapter describes the features provided by the Intel® Xeon® processor C5500/C3500 series IIO module for the development of high RAS (Reliability, Availability, Serviceability) systems. RAS refers to three main features associated with system’s robustness.
Reliability, Availability, Serviceability (RAS) 11.2 System Level RAS 11.2.1 Inband System Management Inband system management is accomplished by firmware running in high privileged mode (SMM) and accessing system configuration registers for system event services. In the event of error, fault, or hot add/remove, firmware is required to determine the system condition and service the event accordingly.
Reliability, Availability, Serviceability (RAS) 11.3.1 Error Severity Classification Errors are classified into three severities in the IIO: Correctable, Uncorrectable, and Fatal. This classification separates those errors resulting in functional failures from those errors resulting in degraded performance. In the IIO, each severity can trigger a system event according to the mapping defined by the error severity register.
Reliability, Availability, Serviceability (RAS) — System interface is compromised. — Inband reporting may be possible. e.g. Uncorrectable tag error in cache, or Permanent PCIe link failure. — Requires immediate logging and reporting of the error to CPU or legacy IIO. 11.3.2 Inband Error Reporting Inband error reporting signals the system of a detected error via inband cycles. There are two complementary inband mechanisms in the IIO.
Reliability, Availability, Serviceability (RAS) 11.3.2.1.1 Completion/Response Status A Non-posted Request requires the return of the completion cycle. This provides an opportunity for the responder to communicate to the requester the success or failure of the request. A status field can be attached to the completion cycle and sent back to the requester. A successful status signifies the request was completed without an error.
Reliability, Availability, Serviceability (RAS) 11.3.2.2.1 NMI (Non-Maskable Interrupt) In past platforms, NMI reported fatal error conditions, typically through PCH component SERR mapping. Since the IIO provides direct mapping of an error to NMI, SERR reporting is obsolete. When an error triggers an NMI, the IIO broadcasts an NMI virtual legacy wire cycle to the CPUs. The PCH reports the NMI through assertion of the NMI pin.
Reliability, Availability, Serviceability (RAS) 11.3.2.2.7 PCIe/DMI “Stop and Scream” There is a enable bit per PCIe port that controls “stop and scream” mode. In this mode the desire is to disallow sending poisoned data onto PCIe and instead disable the PCIe port that was the target of poisoned data. This is done because in the past there have been PCIe/DMI devices that have ignored the poison bit and committed the data that can corrupt the I/O device. 11.3.2.2.
Reliability, Availability, Serviceability (RAS) Figure 73. IIO Error Registers Local Error Log Register IIO Core Local Error Status Control Reg Global Error Log Reg Local Error Severity Reg CPEI PCI-E Intel® QPI Error Control/Status PCI- E Error Control/Status NMI Global Error Status Control Reg IIO Local Error Registers Intel® QPI System Event Reg Intel® QPI Error Severity PCI- E Error Severity SMI Error Pin IIO Global Error Registers MSI Per PCI- E Specification 11.3.3.
Reliability, Availability, Serviceability (RAS) Figure 74.
Reliability, Availability, Serviceability (RAS) Local clusters, maps detected errors to three error severities and report them to global error logic. These errors are sorted into Fatal and Non-fatal and reported to respective global error status register, with severity 2 as fatal, and 0 & 1 as non-fatal. When an error is reported by the local cluster, the corresponding bit in the global fatal or non-fatal error status register is set. Software clears the error bit by writing 1 to the bit.
Reliability, Availability, Serviceability (RAS) then an SMI is dispatched to the CPU and IIO error[2] is asserted. The CPU or BMC can read the Global and Local Error Log register to determine where the error came from and how it should handle the error. At power-on reset, these register are initialized to their default values. The default mapping of severity and system event is set to be consistent with Table 129.
Reliability, Availability, Serviceability (RAS) Figure 77.
Reliability, Availability, Serviceability (RAS) Figure 78 shows the logic diagram of the IIO local and global error registers. Figure 78.
Reliability, Availability, Serviceability (RAS) 11.3.3.3 First and Next Error Log Registers This section describes local error logging for Intel® QuickPath Interconnect and IIO core errors, and it describes global error logging. The log registers are named *FERR and *NERR in the IIO Register Specification. PCIe specifies its own error logging mechanism, This will not be described here. See the PCIe specification for details.
Reliability, Availability, Serviceability (RAS) Figure 79. IIO Error Logging Flow Local Error Local Error Masked ? Yes Done Global Error Masked ? No Set Global Error Status for The Error Severity Map Error to Programmed Severity. Separate logging to Local Fatal and Local Non-Fatal Map Error Severity to Programmed System Event.
Reliability, Availability, Serviceability (RAS) 3. The local FERR and NERR logging events are forwarded to the global FERR and NERR registers. The report of local FERR/NERR sets the corresponding global error bit if the global error is enabled; otherwise the global error bit is not set and the error is forgotten. The global FERR logs the first occurrence of local FERR/NERR event in the IIO and the global NERR logs the subsequent local FERR/NERR events. 4.
Reliability, Availability, Serviceability (RAS) 11.3.3.7 Error Counters This feature allows the system management controller to monitor the component’s health by periodically reporting the correctable error count. The error RAS structure already provides a first error status and a second error status. Because the response time of system management is on the order of milliseconds it is not possible to read and clear the error logs in time to detect short bursts of errors across the chip.
Reliability, Availability, Serviceability (RAS) 11.4.1 Intel® QuickPath Interconnect Error Detection, Logging, and Reporting The IIO implements Intel® QuickPath Interconnect error detection and logging that follows the IIO local and global error reporting mechanism described in this chapter. These registers provide the control and logging of the errors detected on the Intel® QuickPath Interconnect interface.
Reliability, Availability, Serviceability (RAS) to the IIO error severity and reports it to the global error status register. PCIe errors can be classified as two types: Uncorrectable errors and Correctable errors. Uncorrectable errors can further be classified as Fatal or Non-Fatal. This classification is compatible and mapped with the IIO’s error classification: Correctable as Correctable, Non-Fatal as Recoverable, and Fatal as Fatal. 11.5.3.
Reliability, Availability, Serviceability (RAS) Table 128. IIO Default Error Severity Map Error Severity Table 129. ID Intel® QPI IIO Inband Error Reporting (Programmable) PCIe 0 Hardware Correctable Error Hardware Correctable Error Correctable Error. NMI/SMI/CPEI IIO default: CPEI 1 Recoverable Error Recoverable Error Non-Fatal Error.
Reliability, Availability, Serviceability (RAS) Table 129. ID IIO Error Summary (Sheet 2 of 15) Error Default Error Severity Default Error Logging1 Transaction Response FERR/NERR is logged in IIO Core and Global NonFatal Error Log Registers: IIONFERRST IIONNERRST 25 Core header queue parity error 2 Undefined GFERRST GFFERRST GFFERRTIME GFNERRST No Header logging for this errors.
Reliability, Availability, Serviceability (RAS) Table 129. ID IIO Error Summary (Sheet 3 of 15) Error Default Error Severity Default Error Logging1 Transaction Response FERR/NERR is logged in Miscellaneous and Global Fatal Error Log Registers: 20 IIO Configuration Register Parity Error (not including Intel® QPI, PCIe or DMA registers which are covered elsewhere) 21 Persistent SMBus retry failure. 22 Reserved 23 Virtual Pin Port Error. (IIO encountered persistent VPP failure.
Reliability, Availability, Serviceability (RAS) Table 129. ID IIO Error Summary (Sheet 4 of 15) Error Default Error Severity Default Error Logging1 Transaction Response DMA Errors2 40 DMA Transfer Source Address Error 41 DMA Transfer Destination Address Error 42 DMA Next Descriptor Address Error 43 DMA Descriptor Error 44 DMA Chain Address Value Error 45 DMA CHANCMD Error 46 DMA Chipset Uncorrectable Data Integrity error (i.e.
Reliability, Availability, Serviceability (RAS) Table 129. IIO Error Summary (Sheet 5 of 15) ID Error 62 DMA configuration register parity error 63 DMA miscellaneous fatal errors (lock sequence error etc.) Default Error Severity 2 Default Error Logging1 Transaction Response N/A since the error is not associated with a specific transaction. Log the error in corresponding DMAUNCERRSTS/ DMAUNCERRPTR registers and also DMAGLBERRPTR register.
Reliability, Availability, Serviceability (RAS) Table 129. IIO Error Summary (Sheet 6 of 15) Default Error Severity Transaction Response Default Error Logging1 80 Received ‘Unsupported Request’ completion status from downstream device Intel® QPI to PCIe read: IIO returns all ‘1s’ and normal response to Intel® QPI to indicate master abort. Intel® QPI to PCIe NP write: IIO returns normal response. PCIe to PCIe read/NP-write: ‘Unsupported request’ is returned3 to original PCIe requester.
Reliability, Availability, Serviceability (RAS) Table 129. ID 84 IIO Error Summary (Sheet 7 of 15) Error Default Error Severity Completion timeout on NP transactions outstanding on PCI Express/DMI Intel® QPI to PCIe read: IIO returns normal response to Intel® QPI and all 1’s for read data. Intel® QPI to PCIe non-posted write: IIO returns normal response to Intel® QPI. PCIe to PCIe read/non-posted write: UR3 is returned on PCIe. SMBus reads: IIO returns a UR status on SMbus.
Reliability, Availability, Serviceability (RAS) Table 129. ID IIO Error Summary (Sheet 8 of 15) Error 86 Received PCIe unexpected Completion 87 PCIe Flow Control Protocol Error7 88 Received ERR_NONFATAL Message from downstream device 89 PCIe ECC Uncorrectable data Error (PCIe cluster detected internal ECC Uncorrectable data error) 90 PCIe Malformed TLP7 91 PCIe Data Link Protocol Error7 92 PCIe Receiver Overflow 93 Surprise Down 94 Received ERR_FATAL message from downstream device.
Reliability, Availability, Serviceability (RAS) Table 129. ID IIO Error Summary (Sheet 9 of 15) Error Default Error Severity XP header queue parity error. 97 Note: XP Cluster is PCIe/DMI 98 Undefined. Log in XPUNCERRSTS register. Log in XPGLBERRSTS, XPGLBERRPTR registers. If error is forwarded to the global error registers, it is logged in global non-fatal log registers - GFERRST, GFFERRST, GFNERRST, GFFERRTIME. Drop the transaction. Log in XPUNCERRSTS register.
Reliability, Availability, Serviceability (RAS) Table 129. ID A4 A4 IIO Error Summary (Sheet 10 of 15) Error Data parity error while doing a L2 lookup Data parity error while doing a L3 lookup Default Error Severity Default Error Logging1 Transaction Response Log in VTUNCERRSTS and VTUNCERRPTR registers. These errors can also be routed to the IIO global error logic and logged in the global fatal registers. 2 GFERRST GFFERRST GFFERRTIME GFNERRST Log in VTUNCERRSTS and VTUNCERRPTR registers.
Reliability, Availability, Serviceability (RAS) Table 129.
Reliability, Availability, Serviceability (RAS) Table 129.
Reliability, Availability, Serviceability (RAS) Table 129.
Reliability, Availability, Serviceability (RAS) Table 129. ID IIO Error Summary (Sheet 14 of 15) Error D7 Intel® QPI Protocol Layer Received Unexpected or Illegal Response/ Completion D8 Intel® QPI Protocol Layer Received illegal packet field or incorrect target Node ID or poisoned in LL Rx (outbound) when poison is disabled Default Error Severity FERR/NERR is logged in Intel® QPI and Global NonFatal Error Log Registers: 2 Drop Transaction, No Response. This will cause time-out in the requester.
Reliability, Availability, Serviceability (RAS) Table 129. ID IIO Error Summary (Sheet 15 of 15) Error DC IIO SAD illegal or non-existent memory for outbound snoop DE IIO Routing Table pointed to a disabled Intel® QPI port DF Default Error Severity FERR/NERR is logged in Intel® QPI and Global Fatal Error Log Registers: 2 Illegal inbound request (includes VCp/VC1 request when they are disabled) Drop Transaction, No Response. This will cause time-out in the requester for non-posted requests. (e.
Reliability, Availability, Serviceability (RAS) Hot add/remove is the ability to add or remove a component without requiring the system to reboot. There are two types of hot add/remove in Intel® Xeon® processor C5500/C3500 series: • Physical Hot add/remove This is the conventional hot plug of a physical component in the system. • Logical Hot add/remove Logical hot add/remove differs from physical hot add/remove by not requiring physical removal or addition of a component.
Reliability, Availability, Serviceability (RAS) Figure 80. IIO PCI Express Hog Plug Serial Interface CPU + Uncore IIO PCH PEX Root Port GPE MSI (P2P bridge, HPC) VPP 100 KHz SM Bus A2 A1 A0 A2 A1 A0 IO Extender 0 8 8 8 8 11.7.2.
Reliability, Availability, Serviceability (RAS) Table 130. Hot Plug Interface (Sheet 2 of 2) Signal Name Description Action PRSNT# Input signal that indicates if a hot pluggable PCIe card/module is currently plugged into the slot. When a change is detected in this signal, the Presence Detect Event Status register is set and either an interrupt or a general-purpose event message Assert/Deassert_HPGPE is sent to the PCH.
Reliability, Availability, Serviceability (RAS) Figure 81.
Reliability, Availability, Serviceability (RAS) Figure 82.
Reliability, Availability, Serviceability (RAS) The IIO VPP only supports SMBus devices with the command sequence shown Table 131. Each PCIe port is associated with one of these 8-bit ports. The mapping is defined by a Virtual Pin Port register field for each PCIe slot. The VPP register holds the SMBus address and Port (0 or 1) of the I/O port associated with the PCIe. A[1:0] pins on each I/O extender (i.e. PCA9555) connected to the IIO must be strapped uniquely. Table 131. 11.7.2.
Reliability, Availability, Serviceability (RAS) Table 132.
Reliability, Availability, Serviceability (RAS) 11.7.2.5 Miscellaneous Notes Table 134. Read Command Bits IIO Drives 1 Start 7 Address[6:0] 1 0 1 8 [6:3] = 0100 [2:0] = “VPPCTL: VPP Control” indicates write ACK Start 7 Address[6:0] 1 1 8 1 If NACK is received, IIO completes with stop and sets in “VPPSTS: VPP Status Register”.
Reliability, Availability, Serviceability (RAS) 11.7.2.5.2 Attention Button The IIO implements the attention button signal as an edge triggered signal, i.e. the attention button status bit in the Slot Status register is set when an asserting edge on the signal is detected. If an asserting edge on attention button is seen in the same clock, then software clears the attention button status bit, the bit should remain set and if MSI is enabled, another MSI message should be generated.
Packaging and Signal Information 12.0 Packaging and Signal Information 12.1 Signal Descriptions This chapter describes the processor signals. They are arranged in functional groups according to their associated interface or category. All straps are only a weak pulldown needed if a Vss is desired. The following notations describe the signal types: Notations Signal Type I Input pin O Output pin I/O Bi-directional input/output pin Analog Analog reference or output 12.1.
Packaging and Signal Information 12.1.2 System Memory Interface 12.1.2.1 DDR Channel A Signals Table 136. DDR Channel A Signals Signal Names I/O Type DDRA_BA[2:0] O Bank Address Select: These signals define which banks are selected within each SDRAM rank. DDRA_CAS# O CAS Control Signal: Used with DDRA_RAS# and DDRA_WE# (along with DDRA_CS#) to define the SDRAM commands. DDRA_CKE[3:0] O Clock Enable: (one per rank) used to: • Initialize the SDRAMs during power-up. • Power-down SDRAM ranks.
Packaging and Signal Information 12.1.2.2 DDR Channel B Signals Table 137. DDR Channel B Signals Signal Names I/O Type DDRB_BA[2:0] O Bank Address Select: These signals define which banks are selected within each SDRAM rank. DDRB_CAS# O CAS Control Signal: Used with DDRB_RAS# and DDRB_WE# (along with DDRB_CS#) to define the SDRAM Commands. DDRB_CKE[3:0] O Clock Enable: (one per rank) used to: • Initialize the SDRAMs during power-up. • Power-down SDRAM ranks.
Packaging and Signal Information 12.1.2.3 DDR Channel C Signals Table 138. DDR Channel C Signals Signal Names I/O Type DDRC_BA[2:0] O Bank Address Select: These signals define which banks are selected within each SDRAM rank. DDRC_CAS# O CAS Control Signal: Used with DDRC_RAS# and DDRC_WE# (along with DDRC_CS#) to define the SDRAM commands. DDRC_CKE[3:0] O Clock Enable: (one per rank) used to: • Initialize the SDRAMs during power-up. • Power-down SDRAM ranks.
Packaging and Signal Information 12.1.3 PCI Express* Signals Table 140. PCI Express Signals Signal Names I/O Type Description PE_CFG[2:0] I/O PCI Express* Port Bifurcation Configuration: 111 = One x16 PCI Express I/O. 110 = Two x8 PCI Express I/O. 101 = Four x4 PCI Express I/O. 100 = Wait for BIOS to configure PCI Express I/O. 011 = One x8 (port 1-2) and two x4 PCI Express I/O. 010 = Two x4 and one x8 (port 3-4) PCI Express I/O. 001 = Reserved. 000 = Reserved.
Packaging and Signal Information 12.1.5 DMI / ESI Signals Table 142. DMI / ESI Signals Signal Names I/O Type Description I/O DMI/ESI Configuration: Pulled to Vss = ESI (AC coupling required). Pulled to processor Vtt = DMI (DC coupling required). Note: The processor and the PCH must both be configured appropriately to support the same mode of operation. DMI_PE_CFG# I/O DMI/ESI or PCI Express Configuration: No Connect = x4 interface set as DMI/ESI for the legacy (boot) processor.
Packaging and Signal Information 12.1.7 Reset and Miscellaneous Signals Table 144. Miscellaneous Signals Signal Names I/O Type Description DP_SYNCRST# I/O Dual Processor Synchronous Reset: signal driven from the legacy (boot) processor to the non-legacy (application) processor. This signal is only needed for a dual socket configuration. COMP0 I EKEY_NC EXTSYSTRG Must be termianted on the system board using precision resistor.
Packaging and Signal Information Table 145. Thermal Signals (Sheet 2 of 2) I/O Type Description PSI# O Processor Power Status Indicator: This signal is asserted when maximum possible processor core current consumption is less than 20 A. Assertion of this signal is an indication that the VR controller does not currently need to be able to provide ICC above 20 A, and the VR controller can use this information to move to more efficient operating point. This signal will de-assert at least 3.
Packaging and Signal Information Table 146. Power Signals (Sheet 2 of 2) Signal Names I/O Type VTTD Analog Processor power for the memory controller, shared cache and I/O (1.1 V). VTTD_SENSE Analog VTTD_SENSE and VSS_SENSE_VTT provide an isolated, low impedance connection to the processor VTT voltage and ground. They can be used to sense or measure voltage near the silicon. VTT_VID[4:2] O VTT_VID[4:2] is used to support automatic selection of power supply voltages (VTT).
Packaging and Signal Information 12.1.12 ITP Signals Table 149. ITP Signals Signal Names I/O Type Description BPM[7:0]# I/O Breakpoint and Performance Monitor Signals: Outputs from the processor that indicate the status of breakpoints and programmable counters used for monitoring processor performance. PRDY# O PRDY# is a processor output used by debug tools to determine processor debug readiness. PREQ# I PREQ# is used by debug tools to request debug operation of the processor.
Packaging and Signal Information Table 150.
Packaging and Signal Information Table 150.
Packaging and Signal Information Table 150.
Packaging and Signal Information Table 151.
Packaging and Signal Information Table 151.
Packaging and Signal Information Table 151.
Packaging and Signal Information Table 152.
Packaging and Signal Information Table 152.
Packaging and Signal Information Table 152.
Packaging and Signal Information Table 153.
Packaging and Signal Information XY Coord Signal XY Coord XY Coord Signal Signal B12 VDDQ C12 DDRA_CAS# D10 DDRC_ODT[3] B13 DDRA_WE# C13 DDRA_CS[2]# D11 DDRB_ODT[0] B14 DDRB_MA[13] C14 DDRB_CS[6]# D12 DDRB_CS[0]# B15 DDRA_CS[4]# C15 VDDQ D13 VDDQ B16 DDRA_BA[0] C16 DDRC_WE# D14 DDRB_ODT[2] B17 VDDQ C17 DDRB_CS[4]# D15 DDRC_ODT[2] B18 DDRC_MA_PAR C18 DDRB_BA[0] D16 DDRC_CS[2]# B19 DDRA_MA[10] C19 DDRA_CLK_DN[1] D17 DDRC_RAS# B20 DDRA_MA_PAR C20 VDDQ
Packaging and Signal Information XY Coord Signal XY Coord Signal XY Coord Signal E8 DDRB_DQ[33] F6 DDRB_DQ[39] G4 DDRB_DQ[42] E9 DDRB_DQ[32] F7 DDRB_DQS_DN[13] G5 DDRB_DQ[46] E10 DDRB_CS[5]# F8 DDRB_DQS_DP[13] G6 DDRB_DQS_DN[5] E11 VDDQ F9 VSS G7 VSS E12 DDRB_CS[7]# F10 DDRB_DQ[36] G8 DDRB_DQ[37] E13 DDRB_CS[3]# F11 DDRB_ODT[3] G9 DDRB_DQ[44] E14 DDRB_CAS# F12 DDRA_ODT[0] G10 DDRC_DQ[37] E15 DDRB_CS[2]# F13 DDRC_ODT[1] G11 DDRC_DQ[36] E16 VDDQ F14 VD
Packaging and Signal Information XY Coord Signal XY Coord XY Coord Signal Signal H2 DDRA_DQ[40] H43 DDRA_DQ[17] J41 DDRA_DQ[21] H3 DDRA_DQ[45] J1 DDRA_DQS_DN[14] J42 DDRA_DQ[20] H4 DDRB_DQ[43] J2 DDRA_DQS_DP[14] J43 VSS H5 VSS J3 VSS K1 VSS H6 DDRB_DQS_DP[5] J4 DDRB_DQ[52] K2 DDRA_DQS_DP[5] H7 DDRB_DQS_DP[14] J5 DDRB_DQ[47] K3 DDRA_DQS_DN[5] H8 DDRB_DQ[40] J6 DDRB_DQ[41] K4 DDRB_DQ[48] H9 DDRB_DQ[45] J7 DDRB_DQS_DN[14] K5 DDRB_DQ[49] H10 VSS J8 VSS
Packaging and Signal Information XY Coord Signal XY Coord Signal XY Coord Signal K39 DDRC_DQS_DN[2] L37 DDRB_DQS_DP[11] M35 DDRB_DQ[16] K40 DDRC_DQS_DP[2] L38 DDRC_DQS_DN[11] M36 DDRB_DQ[21] K41 VSS L39 VSS M37 VSS K42 DDRA_DQ[10] L40 DDRC_DQ[22] M38 DDRC_DQS_DP[11] K43 DDRA_DQ[11] L41 DDRA_DQS_DP[1] M39 DDRC_DQ[16] L1 DDRA_DQ[42] L42 DDRA_DQ[15] M40 DDRC_DQ[17] L2 DDRA_DQ[47] L43 DDRA_DQ[14] M41 DDRA_DQS_DN[1] L3 DDRA_DQ[46] M1 DDRA_DQ[43] M42 VSS L4
Packaging and Signal Information XY Coord Signal XY Coord XY Coord Signal Signal P11 VSS T8 DDRC_DQS_DN[7] V5 VSS P33 VSS T9 VSS V6 DDRC_DQS_DP[16] P34 DDRB_DQ[8] T10 DDRC_DQ[58] V7 DDRC_DQS_DN[16] P35 DDRB_DQ[9] T11 VCC V8 DDRC_DQ[62] P36 DDRB_DQS_DP[10] T33 VCC V9 DDRB_DQ[60] P37 DDRB_DQS_DN[10] T34 VSS V10 VSS P38 VSS T35 DDRC_DQS_DN[9] V11 NC_V11 P39 DDRB_DQ[10] T36 DDRC_DQ[11] V33 VCCPLL P40 DDRC_DQ[20] T37 DDRC_DQS_DP[1] V34 DDRC_DQ[5] P41
Packaging and Signal Information XY Coord Signal XY Coord Signal XY Coord Signal Y2 DDRA_DQ[58] AB7 VSS AD4 QPI_TX_DN[15] Y3 DDRA_DQ[59] AB8 VTTD AD5 QPI_TX_DP[18] Y4 DDRB_DQS_DP[16] AB9 VTTD AD6 QPI_TX_DP[17] Y5 DDRB_DQS_DN[16] AB10 VTTD AD7 QPI_TX_DN[17] Y6 VSS AB11 VTTD AD8 QPI_TX_DN[19] Y7 DDR_COMP[1] AB33 VTTD AD9 VTTD Y8 DDRB_DQS_DP[7] AB34 VTTD AD10 VTTA Y9 DDRB_DQS_DN[7] AB35 DDRB_DQ[4] AD11 VSS Y10 DDRB_DQ[58] AB36 DDRB_DQ[5] AD33 VSS Y11
Packaging and Signal Information XY Coord Signal XY Coord Signal XY Coord Signal AF1 BCLK_BUF_DN AG41 PE_RX_DN[10] AJ38 PE_RX_DP[0] AF2 QPI_TX_DP[10] AG42 PE_RX_DN[11] AJ39 VSS AF3 QPI_TX_DN[10] AG43 PE_RX_DP[11] AJ40 PE_RX_DP[5] AF4 RSVD_AF4 AH1 VSS AJ41 PE_RX_DP[7] AF5 VSS AH2 QPI_TX_DP[9] AJ42 PE_RX_DN[7] AF6 QPI_CLKTX_DP AH3 QPI_TX_DP[8] AJ43 PE_RX_DP[9] AF7 VTT_VID[3] AH4 QPI_TX_DN[8] AK1 QPI_TX_DP[7] AF8 VTTD AH5 RSVD_AH5 AK2 RSVD_AK2 AF9 VTTD A
Packaging and Signal Information XY Coord Signal XY Coord Signal XY Coord Signal AK36 VSS AL34 SYS_ERR_STAT[2]# AM32 VSS AK37 PE_RX_DP[3] AL35 PROCHOT# AM33 EXTSYSTRG AK38 PE_RX_DN[3] AL36 RSVD_AL36 AM34 DDR_ADR AK39 PE_RX_DN[0] AL37 VSS AM35 SYS_ERR_STAT[0]# AK40 PE_RX_DN[5] AL38 PE_RX_DP[2] AM36 SKTOCC# AK41 VSS AL39 VSS AM37 PE_RX_DP[1] AK42 PE_RX_DN[6] AL40 PE_RX_DP[4] AM38 PE_RX_DN[2] AK43 VSS AL41 PE_RX_DN[4] AM39 VSS AL1 VSS AL42 PE_RX_DP[6]
Packaging and Signal Information XY Coord Signal XY Coord XY Coord Signal Signal AN30 VCC AP28 VCC AR26 VSS AN31 VCC AP29 VSS AR27 VCC AN32 VSS AP30 VCC AR28 VCC AN33 RSVD_AN33 AP31 VCC AR29 VSS AN34 VSS AP32 VSS AR30 VCC AN35 VSS AP33 PE_HP_CLK AR31 VCC AN36 PM_SYNC AP34 PE_HP_DATA AR32 VSS AN37 PE_RX_DN[1] AP35 RSVD_AP35 AR33 VSS AN38 RSVD_AN38 AP36 SYS_ERR_STAT[1]# AR34 RSVD_AR34 AN39 PE_TX_DP[15] AP37 VSS AR35 SMB_DATA AN40 RSVD_AN40 A
Packaging and Signal Information XY Coord Signal XY Coord Signal XY Coord Signal AT24 VCC AU22 VSS AV20 VSS AT25 VCC AU23 VSS AV21 VCC AT26 VSS AU24 VCC AV22 VSS AT27 VCC AU25 VCC AV23 VSS AT28 VCC AU26 VSS AV24 VCC AT29 VSS AU27 VCC AV25 VCC AT30 VCC AU28 VCC AV26 VSS AT31 VCC AU29 VSS AV27 VCC AT32 VSS AU30 VCC AV28 VCC AT33 PE_NTBXL AU31 VCC AV29 VSS AT34 VSS AU32 VSS AV30 VCC AT35 PE_TX_DP[0] AU33 NC_AU33 AV31 VCC AT36 DP_SY
Packaging and Signal Information XY Coord Signal XY Coord Signal XY Coord Signal AW18 VCC AY18 VCC BA19 VCC AW19 VCC AY19 VCC BA20 VSS AW20 VSS AY20 VSS BA24 VCC AW21 VCC AY21 VCC BA25 VCC AW22 VSS AY22 VSS BA26 VSS AW23 VSS AY23 VSS BA27 VCC AW24 VCC AY24 VCC BA28 VCC AW25 VCC AY25 VCC BA29 VSS AW26 VSS AY26 VSS BA30 VCC AW27 VCC AY27 VCC BA35 DMI_PE_CFG# AW28 VCC AY28 VCC BA36 PE_TX_DP[3] AW29 VSS AY29 VSS BA37 PE_TX_DN[3] AW30
Packaging and Signal Information Table 154.
Packaging and Signal Information Signal February 2010 Order Number: 323103-001 XY Coord DDRA_CS[2]# C13 DDRA_CS[3]# B9 DDRA_CS[4]# B15 DDRA_CS[5]# A7 DDRA_CS[6]# C11 DDRA_CS[7]# B8 DDRA_DQ[0] W41 DDRA_DQ[1] V41 DDRA_DQ[2] R43 DDRA_DQ[3] R42 DDRA_DQ[4] W40 DDRA_DQ[5] W42 DDRA_DQ[6] U41 DDRA_DQ[7] T42 DDRA_DQ[8] N41 DDRA_DQ[9] N43 DDRA_DQ[10] K42 DDRA_DQ[11] K43 DDRA_DQ[12] P42 DDRA_DQ[13] P41 DDRA_DQ[14] L43 DDRA_DQ[15] L42 DDRA_DQ[16] H41 DDRA_DQ[17] H43 D
Packaging and Signal Information Signal XY Coord DDRA_DQ[35] G3 DDRA_DQ[36] B6 DDRA_DQ[37] C6 DDRA_DQ[38] F3 DDRA_DQ[39] F2 DDRA_DQ[40] H2 DDRA_DQ[41] H1 DDRA_DQ[42] L1 DDRA_DQ[43] M1 DDRA_DQ[44] G1 DDRA_DQ[45] H3 DDRA_DQ[46] L3 DDRA_DQ[47] L2 DDRA_DQ[48] N1 DDRA_DQ[49] N2 DDRA_DQ[50] T1 DDRA_DQ[51] T2 DDRA_DQ[52] M3 DDRA_DQ[53] N3 DDRA_DQ[54] R4 DDRA_DQ[55] T3 DDRA_DQ[56] U4 DDRA_DQ[57] V1 DDRA_DQ[58] Y2 DDRA_DQ[59] Y3 DDRA_DQ[60] U1 DDRA_DQ[61] U3 D
Packaging and Signal Information Signal February 2010 Order Number: 323103-001 XY Coord DDRA_DQS_DN[12] C39 DDRA_DQS_DN[13] D4 DDRA_DQS_DN[14] J1 DDRA_DQS_DN[15] P1 DDRA_DQS_DN[16] V3 DDRA_DQS_DN[17] B35 DDRA_DQS_DP[0] T43 DDRA_DQS_DP[1] L41 DDRA_DQS_DP[2] F41 DDRA_DQS_DP[3] B39 DDRA_DQS_DP[4] E3 DDRA_DQS_DP[5] K2 DDRA_DQS_DP[6] R2 DDRA_DQS_DP[7] W2 DDRA_DQS_DP[8] D34 DDRA_DQS_DP[9] V43 DDRA_DQS_DP[10] N42 DDRA_DQS_DP[11] H42 DDRA_DQS_DP[12] D39 DDRA_DQS_DP[13] D5
Packaging and Signal Information Signal XY Coord DDRA_MA[9] C26 DDRA_MA[10] B19 DDRA_MA[11] A26 DDRA_MA[12] B26 DDRA_MA[13] A10 DDRA_MA[14] A28 DDRA_MA[15] B29 DDRA_MA_PAR B20 DDRA_ODT[0] F12 DDRA_ODT[1] C9 DDRA_ODT[2] B11 DDRA_ODT[3] C7 DDRA_PAR_ERR[0]# D25 DDRA_PAR_ERR[1]# B28 DDRA_PAR_ERR[2]# A27 DDRA_RAS# A15 DDRA_RESET# D32 DDRA_WE# B13 DDRB_BA[0] C18 DDRB_BA[1] K13 DDRB_BA[2] H27 DDRB_CAS# E14 DDRB_CKE[0] H28 DDRB_CKE[1] E27 DDRB_CKE[2] D27 DDRB_CKE
Packaging and Signal Information Signal February 2010 Order Number: 323103-001 XY Coord DDRB_CS[7]# E12 DDRB_DQ[0] AA36 DDRB_DQ[1] AA35 DDRB_DQ[2] Y35 DDRB_DQ[3] Y34 DDRB_DQ[4] AB35 DDRB_DQ[5] AB36 DDRB_DQ[6] Y40 DDRB_DQ[7] Y39 DDRB_DQ[8] P34 DDRB_DQ[9] P35 DDRB_DQ[10] P39 DDRB_DQ[11] N39 DDRB_DQ[12] R34 DDRB_DQ[13] R35 DDRB_DQ[14] N37 DDRB_DQ[15] N38 DDRB_DQ[16] M35 DDRB_DQ[17] M34 DDRB_DQ[18] K35 DDRB_DQ[19] J35 DDRB_DQ[20] N34 DDRB_DQ[21] M36 DDRB_DQ[22]
Packaging and Signal Information Signal XY Coord DDRB_DQ[40] H8 DDRB_DQ[41] J6 DDRB_DQ[42] G4 DDRB_DQ[43] H4 DDRB_DQ[44] G9 DDRB_DQ[45] H9 DDRB_DQ[46] G5 DDRB_DQ[47] J5 DDRB_DQ[48] K4 DDRB_DQ[49] K5 DDRB_DQ[50] R5 DDRB_DQ[51] T5 DDRB_DQ[52] J4 DDRB_DQ[53] M6 DDRB_DQ[54] R8 DDRB_DQ[55] R7 DDRB_DQ[56] W6 DDRB_DQ[57] W7 DDRB_DQ[58] Y10 DDRB_DQ[59] W10 DDRB_DQ[60] V9 DDRB_DQ[61] W5 DDRB_DQ[62] AA7 DDRB_DQ[63] W9 DDRB_DQS_DN[0] Y37 DDRB_DQS_DN[1] R37 DDRB_DQ
Packaging and Signal Information Signal February 2010 Order Number: 323103-001 XY Coord DDRB_DQS_DN[17] E35 DDRB_DQS_DP[0] Y38 DDRB_DQS_DP[1] R38 DDRB_DQS_DP[2] L35 DDRB_DQS_DP[3] L30 DDRB_DQS_DP[4] E7 DDRB_DQS_DP[5] H6 DDRB_DQS_DP[6] L6 DDRB_DQS_DP[7] Y8 DDRB_DQS_DP[8] G33 DDRB_DQS_DP[9] AA37 DDRB_DQS_DP[10] P36 DDRB_DQS_DP[11] L37 DDRB_DQS_DP[12] K34 DDRB_DQS_DP[13] F8 DDRB_DQS_DP[14] H7 DDRB_DQS_DP[15] M5 DDRB_DQS_DP[16] Y4 DDRB_DQS_DP[17] F35 DDRB_ECC[0] D36 D
Packaging and Signal Information Signal XY Coord DDRB_MA[14] H26 DDRB_MA[15] F26 DDRB_MA_PAR D20 DDRB_ODT[0] D11 DDRB_ODT[1] C8 DDRB_ODT[2] D14 DDRB_ODT[3] F11 DDRB_PAR_ERR[0]# C22 DDRB_PAR_ERR[1]# E25 DDRB_PAR_ERR[2]# F25 DDRB_RAS# G14 DDRB_RESET# D29 DDRB_WE# G13 DDRC_BA[0] A17 DDRC_BA[1] F17 DDRC_BA[2] L26 DDRC_CAS# F16 DDRC_CKE[0] J26 DDRC_CKE[1] G26 DDRC_CKE[2] D26 DDRC_CKE[3] L27 DDRC_CLK_DN[0] J21 DDRC_CLK_DN[1] K20 DDRC_CLK_DN[2] G21 DDRC_CLK_DN[3]
Packaging and Signal Information Signal February 2010 Order Number: 323103-001 XY Coord DDRC_DQ[4] U34 DDRC_DQ[5] V34 DDRC_DQ[6] V37 DDRC_DQ[7] V38 DDRC_DQ[8] U38 DDRC_DQ[9] U39 DDRC_DQ[10] R39 DDRC_DQ[11] T36 DDRC_DQ[12] W39 DDRC_DQ[13] V39 DDRC_DQ[14] T41 DDRC_DQ[15] R40 DDRC_DQ[16] M39 DDRC_DQ[17] M40 DDRC_DQ[18] J40 DDRC_DQ[19] J39 DDRC_DQ[20] P40 DDRC_DQ[21] N36 DDRC_DQ[22] L40 DDRC_DQ[23] K38 DDRC_DQ[24] G40 DDRC_DQ[25] F40 DDRC_DQ[26] J37 DDRC_DQ[27]
Packaging and Signal Information Signal XY Coord DDRC_DQ[45] M10 DDRC_DQ[46] L8 DDRC_DQ[47] M8 DDRC_DQ[48] P7 DDRC_DQ[49] N6 DDRC_DQ[50] P9 DDRC_DQ[51] P10 DDRC_DQ[52] N8 DDRC_DQ[53] N7 DDRC_DQ[54] R10 DDRC_DQ[55] R9 DDRC_DQ[56] U5 DDRC_DQ[57] U6 DDRC_DQ[58] T10 DDRC_DQ[59] U10 DDRC_DQ[60] T6 DDRC_DQ[61] T7 DDRC_DQ[62] V8 DDRC_DQ[63] U9 DDRC_DQS_DN[0] W36 DDRC_DQS_DN[1] T38 DDRC_DQS_DN[2] K39 DDRC_DQS_DN[3] E40 DDRC_DQS_DN[4] J9 DDRC_DQS_DN[5] K7 DDRC_DQS
Packaging and Signal Information Signal February 2010 Order Number: 323103-001 XY Coord DDRC_DQS_DP[4] J10 DDRC_DQS_DP[5] L7 DDRC_DQS_DP[6] P6 DDRC_DQS_DP[7] U8 DDRC_DQS_DP[8] G29 DDRC_DQS_DP[9] U35 DDRC_DQS_DP[10] U40 DDRC_DQS_DP[11] M38 DDRC_DQS_DP[12] H38 DDRC_DQS_DP[13] H11 DDRC_DQS_DP[14] K9 DDRC_DQS_DP[15] N4 DDRC_DQS_DP[16] V6 DDRC_DQS_DP[17] H31 DDRC_ECC[0] H32 DDRC_ECC[1] F33 DDRC_ECC[2] E29 DDRC_ECC[3] E30 DDRC_ECC[4] J31 DDRC_ECC[5] J30 DDRC_ECC[6] F31
Packaging and Signal Information Signal XY Coord DDRC_ODT[2] D15 DDRC_ODT[3] D10 DDRC_PAR_ERR[0]# F21 DDRC_PAR_ERR[1]# J25 DDRC_PAR_ERR[2]# F23 DDRC_RAS# D17 DDRC_RESET# E32 DDRC_WE# C16 DMI_COMP AW33 DMI_PE_CFG# BA35 DMI_PE_RX_DN[0] AA41 DMI_PE_RX_DN[1] AB39 DMI_PE_RX_DN[2] AB38 DMI_PE_RX_DN[3] AC38 DMI_PE_RX_DP[0] AB41 DMI_PE_RX_DP[1] AB40 DMI_PE_RX_DP[2] AC39 DMI_PE_RX_DP[3] AC37 DMI_PE_TX_DN[0] AE42 DMI_PE_TX_DN[1] AD40 DMI_PE_TX_DN[2] AC42 DMI_PE_TX_DN[3] A
Packaging and Signal Information Signal XY Coord PE_CFG[0] AY33 PE_CFG[1] AV34 PE_CFG[2] AY34 PE_CLK_DN AR40 PE_CLK_DP AT40 PE_GEN2_DISABLE# AV33 PE_HP_CLK AP33 PE_HP_DATA AP34 PE_ICOMPI AN43 PE_ICOMPO AM43 PE_NTBXL AT33 PE_RBIAS AU41 PE_RCOMPO AL43 PE_RX_DN[0] AK39 PE_RX_DN[1] AN37 PE_RX_DN[2] AM38 PE_RX_DN[3] AK38 PE_RX_DN[4] AL41 PE_RX_DN[5] AK40 PE_RX_DN[6] AK42 PE_RX_DN[7] AJ42 PE_RX_DN[8] AG39 PE_RX_DN[9] AH43 PE_RX_DN[10] AG41 PE_RX_DN[11] AG42 PE_
Packaging and Signal Information Signal February 2010 Order Number: 323103-001 XY Coord PE_RX_DP[12] AF42 PE_RX_DP[13] AG40 PE_RX_DP[14] AE40 PE_RX_DP[15] AF38 PE_TX_DN[0] AU35 PE_TX_DN[1] AV37 PE_TX_DN[2] AY36 PE_TX_DN[3] BA37 PE_TX_DN[4] AW38 PE_TX_DN[5] AY38 PE_TX_DN[6] AW39 PE_TX_DN[7] AV40 PE_TX_DN[8] AU38 PE_TX_DN[9] AT39 PE_TX_DN[10] AR43 PE_TX_DN[11] AR42 PE_TX_DN[12] AN42 PE_TX_DN[13] AP41 PE_TX_DN[14] AP38 PE_TX_DN[15] AP39 PE_TX_DP[0] AT35 PE_TX_DP[1]
Packaging and Signal Information Signal XY Coord PRDY# B41 PREQ# C42 PROCHOT# AL35 PSI# AP7 QPI_CLKRX_DN AR6 QPI_CLKRX_DP AT6 QPI_CLKTX_DN AE6 QPI_CLKTX_DP AF6 QPI_COMP[0] AL6 QPI_COMP[1] AU34 QPI_RX_DN[0] AV8 QPI_RX_DN[1] AW7 QPI_RX_DN[2] BA8 QPI_RX_DN[3] AW5 QPI_RX_DN[4] BA6 QPI_RX_DN[5] AY5 QPI_RX_DN[6] AU6 QPI_RX_DN[7] AW3 QPI_RX_DN[8] AU3 QPI_RX_DN[9] AT2 QPI_RX_DN[10] AR1 QPI_RX_DN[11] AR5 QPI_RX_DN[12] AN2 QPI_RX_DN[13] AM1 QPI_RX_DN[14] AM3 QPI_RX
Packaging and Signal Information Signal February 2010 Order Number: 323103-001 XY Coord QPI_RX_DP[11] AR4 QPI_RX_DP[12] AP2 QPI_RX_DP[13] AN1 QPI_RX_DP[14] AM2 QPI_RX_DP[15] AP3 QPI_RX_DP[16] AM4 QPI_RX_DP[17] AN5 QPI_RX_DP[18] AM6 QPI_RX_DP[19] AM8 QPI_TX_DN[0] AH8 QPI_TX_DN[1] AJ7 QPI_TX_DN[2] AJ6 QPI_TX_DN[3] AK5 QPI_TX_DN[4] AK4 QPI_TX_DN[5] AG6 QPI_TX_DN[6] AJ2 QPI_TX_DN[7] AJ1 QPI_TX_DN[8] AH4 QPI_TX_DN[9] AG2 QPI_TX_DN[10] AF3 QPI_TX_DN[11] AD1 QPI_TX_DN[
Packaging and Signal Information Signal XY Coord QPI_TX_DP[12] AD2 QPI_TX_DP[13] AC3 QPI_TX_DP[14] AE3 QPI_TX_DP[15] AC4 QPI_TX_DP[16] AB6 QPI_TX_DP[17] AD6 QPI_TX_DP[18] AD5 QPI_TX_DP[19] AC8 RSTIN# AJ36 RSVD_A40 A40 RSVD_AG4 AG4 RSVD_AG5 AG5 RSVD_AH5 AH5 RSVD_AK2 AK2 RSVD_AL4 AL4 RSVD_AL5 AL5 RSVD_AL36 AL36 RSVD_AM40 AM40 RSVD_AM41 AM41 RSVD_AN33 AN33 RSVD_AN38 AN38 RSVD_AN40 AN40 RSVD_AP35 AP35 RSVD_AR34 AR34 RSVD_AR37 AR37 RSVD_AT4 AT4 RSVD_AT5 AT5
Packaging and Signal Information Signal February 2010 Order Number: 323103-001 XY Coord RSVD_AY41 AY41 RSVD_BA4 BA4 RSVD_BA40 BA40 RSVD_K15 K15 RSVD_K24 K24 RSVD_L15 L15 RSVD_L23 L23 SKTOCC# AM36 SMB_CLK AR36 SMB_DATA AR35 SYS_ERR_STAT[0]# AM35 SYS_ERR_STAT[1]# AP36 SYS_ERR_STAT[2]# AL34 TCLK AH10 TDI AJ9 TDI_M AH33 TDO AJ10 TDO_M AL33 THERMTRIP# AG37 TMS AG10 TRST# AH9 VCC M11 VCC M13 VCC M15 VCC M19 VCC M21 VCC M23 VCC M25 VCC M29 VCC M31 VCC
Packaging and Signal Information Signal XY Coord VCC AJ34 VCC AK11 VCC AK12 VCC AK13 VCC AK15 VCC AK16 VCC AK18 VCC AK19 VCC AK21 VCC AK24 VCC AK25 VCC AK27 VCC AK28 VCC AK30 VCC AK31 VCC AK33 VCC AL12 VCC AL13 VCC AL15 VCC AL16 VCC AL18 VCC AL19 VCC AL21 VCC AL24 VCC AL25 VCC AL27 VCC AL28 VCC AL30 VCC AL31 VCC AM12 VCC AM13 VCC AM15 VCC AM16 VCC AM18 VCC AM19 VCC AM21 VCC AM24 VCC AM25 VCC AM27 VCC AM28 VCC AM30 Intel® Xe
Packaging and Signal Information Signal February 2010 Order Number: 323103-001 XY Coord VCC AM31 VCC AN12 VCC AN13 VCC AN15 VCC AN16 VCC AN18 VCC AN19 VCC AN21 VCC AN24 VCC AN25 VCC AN27 VCC AN28 VCC AN30 VCC AN31 VCC AP12 VCC AP13 VCC AP15 VCC AP16 VCC AP18 VCC AP19 VCC AP21 VCC AP24 VCC AP25 VCC AP27 VCC AP28 VCC AP30 VCC AP31 VCC AR10 VCC AR12 VCC AR13 VCC AR15 VCC AR16 VCC AR18 VCC AR19 VCC AR21 VCC AR24 VCC AR25 VCC AR27 VC
Packaging and Signal Information Signal XY Coord VCC AT9 VCC AT10 VCC AT12 VCC AT13 VCC AT15 VCC AT16 VCC AT18 VCC AT19 VCC AT21 VCC AT24 VCC AT25 VCC AT27 VCC AT28 VCC AT30 VCC AT31 VCC AU9 VCC AU10 VCC AU12 VCC AU13 VCC AU15 VCC AU16 VCC AU18 VCC AU19 VCC AU21 VCC AU24 VCC AU25 VCC AU27 VCC AU28 VCC AU30 VCC AU31 VCC AV9 VCC AV10 VCC AV12 VCC AV13 VCC AV15 VCC AV16 VCC AV18 VCC AV19 VCC AV21 VCC AV24 VCC AV25 Intel® Xeon®
Packaging and Signal Information Signal February 2010 Order Number: 323103-001 XY Coord VCC AV27 VCC AV28 VCC AV30 VCC AV31 VCC AW9 VCC AW10 VCC AW12 VCC AW13 VCC AW15 VCC AW16 VCC AW18 VCC AW19 VCC AW21 VCC AW24 VCC AW25 VCC AW27 VCC AW28 VCC AW30 VCC AW31 VCC AY9 VCC AY10 VCC AY12 VCC AY13 VCC AY15 VCC AY16 VCC AY18 VCC AY19 VCC AY21 VCC AY24 VCC AY25 VCC AY27 VCC AY28 VCC AY30 VCC AY31 VCC BA9 VCC BA10 VCC BA12 VCC BA13 VCC
Packaging and Signal Information Signal XY Coord VCC BA19 VCC BA24 VCC BA25 VCC BA27 VCC BA28 VCC BA30 VCC_SENSE AR9 VCCPLL U33 VCCPLL V33 VCCPLL W33 VCCPWRGOOD AR7 VDDQ A9 VDDQ A14 VDDQ A19 VDDQ A24 VDDQ A29 VDDQ B7 VDDQ B12 VDDQ B17 VDDQ B22 VDDQ B27 VDDQ B32 VDDQ C10 VDDQ C15 VDDQ C20 VDDQ C25 VDDQ C30 VDDQ D13 VDDQ D18 VDDQ D23 VDDQ D28 VDDQ E11 VDDQ E16 VDDQ E21 VDDQ E26 VDDQ E31 VDDQ F14 VDDQ F19 VDDQ F24 VDDQ G17 VDDQ
Packaging and Signal Information Signal February 2010 Order Number: 323103-001 XY Coord VDDQ G27 VDDQ H15 VDDQ H20 VDDQ H25 VDDQ J18 VDDQ J23 VDDQ J28 VDDQ K16 VDDQ K21 VDDQ K26 VDDQ L14 VDDQ L19 VDDQ L24 VDDQ M17 VDDQ M27 VID[0] AL10 VID[1] AL9 VID[2] AN9 VID[3] AM10 VID[4] AN10 VID[5] AP9 VID[6] AP8 VID[7] AN8 VSS A4 VSS A6 VSS A31 VSS A35 VSS A39 VSS A41 VSS B2 VSS B37 VSS B42 VSS C5 VSS C31 VSS C32 VSS C35 VSS C40 VSS C43 VS
Packaging and Signal Information Signal XY Coord VSS D31 VSS D33 VSS D38 VSS D43 VSS E1 VSS E6 VSS E28 VSS E36 VSS E41 VSS F4 VSS F9 VSS F28 VSS F29 VSS F34 VSS F39 VSS G2 VSS G7 VSS G12 VSS G28 VSS G32 VSS G37 VSS G42 VSS H5 VSS H10 VSS H29 VSS H30 VSS H35 VSS H40 VSS J3 VSS J8 VSS J13 VSS J29 VSS J33 VSS J38 VSS J43 VSS K1 VSS K6 VSS K11 VSS K27 VSS K29 VSS K31 Intel® Xeon® Processor C5500/C3500 Series Datasheet, Volume 1
Packaging and Signal Information Signal February 2010 Order Number: 323103-001 XY Coord VSS K36 VSS K41 VSS L4 VSS L9 VSS L29 VSS L34 VSS L39 VSS M2 VSS M7 VSS M12 VSS M14 VSS M16 VSS M18 VSS M20 VSS M22 VSS M24 VSS M26 VSS M28 VSS M30 VSS M32 VSS M37 VSS M42 VSS N5 VSS N10 VSS N35 VSS N40 VSS P3 VSS P8 VSS P11 VSS P33 VSS P38 VSS P43 VSS R1 VSS R6 VSS R36 VSS R41 VSS T4 VSS T9 VSS T34 VSS T39 VSS U2 Intel® Xeon® Processor
Packaging and Signal Information Signal XY Coord VSS U7 VSS U37 VSS U42 VSS V5 VSS V10 VSS V35 VSS V40 VSS W3 VSS W8 VSS W38 VSS W43 VSS Y1 VSS Y6 VSS Y11 VSS Y33 VSS Y36 VSS Y41 VSS AA3 VSS AA9 VSS AA34 VSS AA39 VSS AA40 VSS AB4 VSS AB7 VSS AB37 VSS AB42 VSS AC2 VSS AC5 VSS AC7 VSS AC9 VSS AC36 VSS AC40 VSS AD11 VSS AD33 VSS AD37 VSS AD39 VSS AD43 VSS AE2 VSS AE7 VSS AE41 VSS AF5 Intel® Xeon® Processor C5500/C3500 Series D
Packaging and Signal Information Signal February 2010 Order Number: 323103-001 XY Coord VSS AF35 VSS AF39 VSS AF43 VSS AG3 VSS AG9 VSS AG11 VSS AG33 VSS AG35 VSS AG38 VSS AH1 VSS AH7 VSS AH34 VSS AH38 VSS AH40 VSS AH42 VSS AJ5 VSS AJ39 VSS AK3 VSS AK7 VSS AK9 VSS AK10 VSS AK14 VSS AK17 VSS AK20 VSS AK22 VSS AK23 VSS AK26 VSS AK29 VSS AK32 VSS AK34 VSS AK36 VSS AK41 VSS AK43 VSS AL1 VSS AL2 VSS AL7 VSS AL11 VSS AL14 VSS AL17 VS
Packaging and Signal Information Signal XY Coord VSS AL23 VSS AL26 VSS AL29 VSS AL32 VSS AL37 VSS AL39 VSS AM5 VSS AM9 VSS AM11 VSS AM14 VSS AM17 VSS AM20 VSS AM22 VSS AM23 VSS AM26 VSS AM29 VSS AM32 VSS AM39 VSS AM42 VSS AN3 VSS AN7 VSS AN11 VSS AN14 VSS AN17 VSS AN20 VSS AN22 VSS AN23 VSS AN26 VSS AN29 VSS AN32 VSS AN34 VSS AN35 VSS AN41 VSS AP1 VSS AP5 VSS AP6 VSS AP10 VSS AP11 VSS AP14 VSS AP17 VSS AP20 Intel® Xeon® Pro
Packaging and Signal Information Signal February 2010 Order Number: 323103-001 XY Coord VSS AP22 VSS AP23 VSS AP26 VSS AP29 VSS AP32 VSS AP37 VSS AP43 VSS AR2 VSS AR3 VSS AR11 VSS AR14 VSS AR17 VSS AR20 VSS AR22 VSS AR23 VSS AR26 VSS AR29 VSS AR32 VSS AR33 VSS AR39 VSS AT7 VSS AT8 VSS AT11 VSS AT14 VSS AT17 VSS AT20 VSS AT22 VSS AT23 VSS AT26 VSS AT29 VSS AT32 VSS AT34 VSS AT37 VSS AT38 VSS AT41 VSS AU1 VSS AU5 VSS AU11 VSS AU1
Packaging and Signal Information Signal XY Coord VSS AU22 VSS AU23 VSS AU26 VSS AU29 VSS AU32 VSS AU36 VSS AU40 VSS AU43 VSS AV4 VSS AV11 VSS AV14 VSS AV17 VSS AV20 VSS AV22 VSS AV23 VSS AV26 VSS AV29 VSS AV32 VSS AV35 VSS AV39 VSS AV41 VSS AW1 VSS AW6 VSS AW8 VSS AW11 VSS AW14 VSS AW17 VSS AW20 VSS AW22 VSS AW23 VSS AW26 VSS AW29 VSS AW32 VSS AW35 VSS AY2 VSS AY7 VSS AY11 VSS AY14 VSS AY17 VSS AY20 VSS AY22 Intel® Xeon® Pr
Packaging and Signal Information Signal February 2010 Order Number: 323103-001 XY Coord VSS AY23 VSS AY26 VSS AY29 VSS AY32 VSS AY37 VSS AY42 VSS BA3 VSS BA5 VSS BA11 VSS BA14 VSS BA17 VSS BA20 VSS BA26 VSS BA29 VSS BA39 VSS_SENSE AR8 VSS_SENSE_VTT AE37 VTT_VID[2] AV3 VTT_VID[3] AF7 VTT_VID[4] AV6 VTTA AD10 VTTA AE10 VTTA AE11 VTTA AE33 VTTA AF11 VTTA AF33 VTTA AF34 VTTA AG34 VTTD AA10 VTTD AA11 VTTD AA33 VTTD AB8 VTTD AB9 VTTD AB10 VTT
Packaging and Signal Information Signal XY Coord VTTD AC35 VTTD AD9 VTTD AD34 VTTD AD35 VTTD AD36 VTTD AE8 VTTD AE9 VTTD AE34 VTTD AE35 VTTD AF8 VTTD AF9 VTTD AF36 VTTD AF37 VTTD_SENSE AE36 VTTPWRGOOD AH37 §§ Intel® Xeon® Processor C5500/C3500 Series Datasheet, Volume 1 482 February 2010 Order Number: 323103-001
Electrical Specifications 13.0 Electrical Specifications 13.1 Processor Signaling The Intel® Xeon® processor C5500/C3500 series includes 1366 lands that utilize various signaling technologies. Signals are grouped by electrical characteristics and buffer type into various signal groups.
Electrical Specifications termination resistance for each DQ and DQS/DQS# signal via the ODT control pin. The ODT feature improves signal integrity of the memory channel by allowing the DRAM controller to independently turn on or off the termination resistance for any or all DRAM devices themselves instead of on the motherboard. 13.1.
Electrical Specifications — x16 lanes can be bifurcated to support Gen 1/2 combinations of x8 and x4 links • Intel® Xeon® processor C5500/C3500 series x4 port: 2.5 GT/s.
Electrical Specifications 13.1.6 Clock Signals The processor core, processor uncore, Intel® QuickPath Interconnect link, and DDR3 memory interface frequencies are generated from BCLK_DP and BCLK_DN signals. There is no direct link between core frequency and Intel® QuickPath Interconnect link frequency (e.g., no core frequency to Intel® QuickPath Interconnect multiplier).
Electrical Specifications Table 155 outlines the required voltage supplies necessary to support Intel® Xeon® processor C5500/C3500 series. Table 155. Processor Power Supply Voltages1 Power Rail Nominal Voltage Notes VCC See Table 163; Figure 86 Each processor includes a dedicated VR11.1 regulator. VCCPLL 1.80 V Each processor includes dedicated VCCPLL and PLL circuits. VDDQ 1.50 V Each processor and DDR3 stack shares a dedicated voltage regulator.
Electrical Specifications Individual processor VID values may be calibrated during manufacturing such that two processor units with the same core frequency may have different default VID settings. The processor uses eight voltage identification signals, VID[7:0], to support automatic selection of core power supply voltages. Table 156 specifies the voltage level corresponding to the state of VID[7:0]. A ‘1’ in this table refers to a high voltage level and a ‘0’ refers to a low voltage level.
Electrical Specifications Table 156. Voltage Identification Definition (Sheet 2 of 5) VID7 VID6 VID5 VID4 VID3 VID2 VID1 VID0 VCC_MAX 0 0 0 1 0 1 0 1 1.48125 0 0 0 1 0 1 1 0 1.47500 0 0 0 1 0 1 1 1 1.46875 0 0 0 1 1 0 0 0 1.46250 0 0 0 1 1 0 0 1 1.45625 0 0 0 1 1 0 1 0 1.45000 0 0 0 1 1 0 1 1 1.44375 0 0 0 1 1 1 0 0 1.43750 0 0 0 1 1 1 0 1 1.43125 0 0 0 1 1 1 1 0 1.42500 0 0 0 1 1 1 1 1 1.
Electrical Specifications Table 156. Voltage Identification Definition (Sheet 3 of 5) VID7 VID6 VID5 VID4 VID3 VID2 VID1 VID0 VCC_MAX 0 0 1 1 1 1 0 1 1.23125 0 0 1 1 1 1 1 0 1.22500 0 0 1 1 1 1 1 1 1.21875 0 1 0 0 0 0 0 0 1.21250 0 1 0 0 0 0 0 1 1.20625 0 1 0 0 0 0 1 0 1.20000 0 1 0 0 0 0 1 1 1.19375 0 1 0 0 0 1 0 0 1.18750 0 1 0 0 0 1 0 1 1.18125 0 1 0 0 0 1 1 0 1.17500 0 1 0 0 0 1 1 1 1.
Electrical Specifications Table 156. Voltage Identification Definition (Sheet 4 of 5) VID7 VID6 VID5 VID4 VID3 VID2 VID1 VID0 VCC_MAX 0 1 1 0 0 1 0 1 0.98125 0 1 1 0 0 1 1 0 0.97500 0 1 1 0 0 1 1 1 0.96875 0 1 1 0 1 0 0 0 0.96250 0 1 1 0 1 0 0 1 0.95625 0 1 1 0 1 0 1 0 0.95000 0 1 1 0 1 0 1 1 0.94375 0 1 1 0 1 1 0 0 0.93750 0 1 1 0 1 1 0 1 0.93125 0 1 1 0 1 1 1 0 0.92500 0 1 1 0 1 1 1 1 0.
Electrical Specifications Table 156. Voltage Identification Definition (Sheet 5 of 5) VID7 VID6 VID5 VID4 VID3 VID2 VID1 VID0 VCC_MAX 1 0 0 0 1 1 0 1 0.73125 1 0 0 0 1 1 1 0 0.72500 1 0 0 0 1 1 1 1 0.71875 1 0 0 1 0 0 0 0 0.71250 1 0 0 1 0 0 0 1 0.70625 1 0 0 1 0 0 1 0 0.70000 1 0 0 1 0 0 1 1 0.69375 1 0 0 1 0 1 0 0 0.68750 1 0 0 1 0 1 0 1 0.68125 1 0 0 1 0 1 1 0 0.67500 1 0 0 1 0 1 1 1 0.
Electrical Specifications Note: • The expected voltage range is 1.35-0.75v. • When the “11111111” VID pattern is observed, or when the SKTOCC# pin is high, the voltage regulator output should be disabled. • Shading denotes the expected VID range of the processor. • The VID range includes VID transitions that may be initiated by thermal events, Extended HALT state transitions (see Section 8.2, “Processor Core Power Management”), higher C-States (see Section 8.2.
Electrical Specifications 13.1.10.3.2 Power-On Configuration (POC) Several configuration options can be configured by hardware. Power-on configuration (POC) functionality is either MUx’ed onto VID signals (see Section 13.1.10.3) or sampled on the active-to-inactive transition of RSTIN#. For specifics on these options, See Table 158. Requests to execute Built-In Self Test (BIST) are not selected by hardware, but rather passed across the Intel® QuickPath Interconnect link during initialization.
Electrical Specifications Table 158. VTT Voltage Identification Definition VID7 13.1.11 VID6 VID5 VID4 VID3 VID2 VID1 VID0 VR 11.0 Voltage VTT_TYP (Voltage + Offset) 0 1 0 0 0 0 1 0 1.200V 1.220V 0 1 0 0 0 1 1 0 1.175V 1.195V 0 1 0 0 1 0 1 0 1.150V 1.170V 0 1 0 0 1 1 1 0 1.125V 1.145V 0 1 0 1 0 0 1 0 1.100V 1.120V 0 1 0 1 0 1 1 0 1.075V 1.095V 0 1 0 1 1 0 1 0 1.050V 1.070V 0 1 0 1 1 1 1 0 1.025V 1.
Electrical Specifications Table 159.
Electrical Specifications Table 159.
Electrical Specifications Table 159.
Electrical Specifications Table 159. Signal Groups (Sheet 5 of 5) Signal Group Power / Ground Signals1 Buffer Type Analog VSS Analog Input ISENSE Analog VCCSENSE Analog VSSSENSE Analog VSS_SENSE_VTTD Analog VTTD_SENSE Single ended CMOS (Push-Pull) Output (CMOS Input during power up for POC straps) VID[7:6], VID[5:3]/CSC[2:0], VID[2:0]/MSID[2:0] Single ended CMOS (Push-Pull) Output VTT_VID[4:2] No Connect & Reserved Signals NC_x RSV_x 1. See Section 6.
Electrical Specifications 13.3 Mixing Processors Intel supports and validates dual processor configurations only in which both processors operate with the same Intel® QuickPath Interconnect frequency, core frequency, power segment, and have the same internal cache sizes. Mixing components operating at different internal clock frequencies is not supported and will not be validated by Intel. Combining processors from different power segments is also not supported.
Electrical Specifications Table 161. Processor Absolute Minimum and Maximum Ratings Symbol Min VCC Processor core voltage with respect to VSS -0.300 VCCPLL Processor PLL voltage with respect to VSS 1.800 V 4 VDDQ Processor I/O supply voltage for DDR3 with respect to VSS 1.500 V 4 VTTA Processor uncore analog voltage with respect to VSS 1.155 V 3 VTTD Processor uncore digital voltage with respect to VSS 1.
Electrical Specifications Table 162. Voltage and Current Specifications (Sheet 1 of 3) Symbol Parameter VID VCC VID Range VCC Core Voltage (Launch - FMB) VVID_STEP VID step size during a transition VCCPLL PLL Voltage (DC + AC specification) VDDQ I/O Voltage for DDR3 (DC + AC specification) VTT_VID VTT Voltage Plane Min - 0.750 VCC Typ Unit 1.350 V 2,3 V 3,4,6,7, 11 ± 6.250 mV 9 See Table 163 and Figure 86 - Notes1 Max VCCPLL 0.95*VCCPLL (Typ) 1.800 1.
Electrical Specifications Table 162. Voltage and Current Specifications (Sheet 2 of 3) Symbol Unit Notes1 ECC5549: TDP = 85W VCC VCCPLL VDDQ VTTA VTTD 100 1.5 9 6 22 A 11 ECC5509: TDP = 85W VCC VCCPLL VDDQ VTTA VTTD 75 1.5 7.5 6 27 A 11 ECC3539: TDP = 65W VCC VCCPLL VDDQ VTTA VTTD 55 1.5 7.5 4 20 A 11 LC5528: TDP = 60W VCC VCCPLL VDDQ VTTA VTTD 70 1.5 7.5 6 17 A 11 EC5539: TDP = 65W VCC VCCPLL VDDQ VTTA VTTD 42 1.5 9 6 27 A 11 LC5518: TDP = 48W VCC VCCPLL VDDQ VTTA VTTD 54 1.
Electrical Specifications Table 162. Voltage and Current Specifications (Sheet 3 of 3) Symbol ICC_MAX ICCPLL_MAX IDDQ_MAX ITT_MAX IDDQ_S3 Voltage Plane Max Unit Notes1 P1053: TDP = 50W VCC VCCPLL VDDQ VTTA VTTD 13 1.5 4.5 4.2 22 A 11 LC3528: TDP = 32W VCC VCCPLL VDDQ VTTA VTTD 24 1.5 5.5 4 15 A 11 LC3518: TDP = 23W VCC VCCPLL VDDQ VTTA VTTD 11 1.5 4.5 4 13 A 11 Parameter DDR3 System Memory Interface Supply Current in Standby State VDDQ Min Typ A Notes: 1.
Electrical Specifications Table 163. VCC Static and Transient Tolerance ICC (A) VCC_MAX (V) VCC_TYP (V) VCC_MIN (V) 0 VID - 0.000 VID - 0.015 VID - 0.030 5 VID - 0.004 VID - 0.019 VID - 0.034 10 VID - 0.008 VID - 0.023 VID - 0.038 15 VID - 0.012 VID - 0.027 VID - 0.042 20 VID - 0.016 VID - 0.031 VID - 0.046 25 VID - 0.020 VID - 0.035 VID - 0.050 30 VID - 0.024 VID - 0.039 VID - 0.054 35 VID - 0.028 VID - 0.043 VID - 0.058 40 VID - 0.032 VID - 0.047 VID - 0.
Electrical Specifications Figure 86. VCC Static and Transient Tolerance Loadlines1,2,3,4 Icc [A] 0 10 20 30 40 50 60 70 80 90 100 110 120 130 140 150 VID - 0.000 VID - 0.020 VID - 0.040 Vcc [V] VID - 0.060 VID - 0.080 VID - 0.100 VID - 0.120 VID - 0.140 VID - 0.160 VID - 0.180 Notes: 1. The VCC_MIN and VCC_MAX loadlines represent static and transient limits. See Section 13.6.1 for VCC overshoot specifications. 2. See Table 163 for VCC Static and Transient Tolerance. 3.
Electrical Specifications 13.6.1 VCC Overshoot Specifications The processor can tolerate short transient overshoot events where VCC exceeds the VID voltage when transitioning from a high-to-low current load condition. This overshoot cannot exceed VID + VOS_MAX (VOS_MAX is the maximum allowable overshoot above VID). These specifications apply to the processor die voltage as measured across the VCC_SENSE and VSS_SENSE lands. Table 164. VCC Overshoot Specifications Symbol Figure 87.
Electrical Specifications 13.6.2 Die Voltage Validation Core voltage (VCC) overshoot events at the processor must meet the specifications in Table 164 when measured across the VCC_SENSE and VSS_SENSE lands. Overshoot events that are < 10 ns in duration may be ignored. These measurements of processor die level overshoot should be taken with a 100 MHz bandwidth limited oscilloscope. Table 165. ICC Max and ICC TDP by SKU SKU 13.6.
Electrical Specifications Table 166. DDR3 Signal Group DC Specifications (Sheet 2 of 2) Symbol Parameter Min Typ Max Units Notes1 RON DDR3 Data Buffer On Resistance 21 31 Ohms 5 Data ODT On-Die Termination for Data Signals 45 90 55 110 Ohms 7 ParErr ODT On-Die Termination for Parity Error bits 60 80 Ohms ILI Input Leakage Current ± 500 mA DDR_COMP0 COMP Resistance 99 100 101 Ohms 8 DDR_COMP1 COMP Resistance 24.65 24.9 25.15 Ohms 8 DDR_COMP2 COMP Resistance 128.
Electrical Specifications 13.6.4 PCI Express Signal DC Specifications The following table defines the parameters for transmitters and receivers. Table 167. PCI Express/DMI Interface -- 2.5 and 5.0 GT/s Transmitter DC Specifications Symbol Parameter 2.5 GT/s 5.0 GT/s Units Comments 80 (min) 120 (max) 120 (max) Ohms Low impedance defined during signaling. Parameter is captured for 5.0 GHz by RLTX-DIFF.
Electrical Specifications Table 168. PCI Express Interface -- 2.5 and 5.0 GT/s Recevier DC Specifications Symbol Parameter 2.5 GT/s 5.0 GT/s Units Comments ZRX-DC Receiver DC common mode impedance 40 (min) 60 (max) 40 (min) 60 (max) Ohms DC impedance limits are needed to guarantee Receiver detect. ZRX-DIFF-DC DC differential impedance 80 (min) 120 (max) Not specified Ohms For 5.0 GT/s covered under RLRX-DIFF parameter.
Electrical Specifications 13.6.6 PECI Signal DC Specifications The following table defines the parameters for PECI. Table 170. PECI DC Electrical Limits Symbol Definition and Conditions Min Max -0.150 VTTD + 0.150 Units Notes1 VIn Input Voltage Range VHysteresis Hysteresis VN Negative-edge threshold voltage 0.275 * VTTD 0.500 * VTTD V 2,6 VP Positive-edge threshold voltage 0.550 * VTTD 0.725 * VTTD V 2,6 RPullup Pullup Resistance (VOH = 0.
Electrical Specifications 13.6.8 Reset and Micscellaneous DC Specifications Table 172. Reset and Miscellaneous Signal Group DC Specifications Symbol Parameter VIL Input Low Voltage VIH Input High Voltage VOL Output Low Voltage Min Typ Max Units Notes1 0.64 * VTTA V 2,3 V 2 V 2,4 0.76 * VTTA VTTA * RON / (RON + RSYS_TERM) VOH Output High Voltage VTTA ODT On-Die Termination 45 55 RON Buffer On Resistance 10 18 Notes: 1. 2. 3. 4. 5. Thermal DC Specification Table 173.
Electrical Specifications 13.6.10 Test Access Port (TAP) DC Specification Table 174. Test Access Port (TAP) Signal Group DC Specification Symbol Parameter VIL Input Low Voltage VIH Input High Voltage Min Max Units Notes1 0.40 * VTTA V 2,3 V 2 V 2,4 V 2 Typ 0.60 * VTTA VTTA * RON / (RON + RSYS_TERM) VOL Output Low Voltage VOH Output High Voltage ODT On-Die Termination 45 55 RON Buffer On Resistance 10 18 Notes: 1. 2. 3. 4. 5.
Testability 14.0 Testability The processor includes boundary-scan for board and system level testability. 14.1 Boundary-Scan The processor is compatible with the IEEE 1149.1-2001 (Standard Test Access Port and Boundary-Scan Architecture) specification. See the specification for functionality.
Testability Figure 88.
Testability 14.3 TAP Instructions and Opcodes The TAP controllers support the boundary-scan instructions listed in: • Table 176, “Processor Core TAP Controller Supported Boundary-Scan Instruction Opcodes” on page 517. • Table 177, “Processor Un-Core TAP Controller Supported Boundary-Scan Instruction Opcodes” on page 517. • Table 178, “Processor Integrated I/O TAP Controller Supported Boundary-Scan Instruction Opcodes” on page 518. 14.3.
Testability Table 178. Processor Integrated I/O TAP Controller Supported Boundary-Scan Instruction Opcodes Opcode (binary) 14.3.
Testability Figure 90. Processor TAP Connections TRST# TCK TDO PROCESSOR TDO_M TMS TDI TDI_M Note: TDI_M must be connected to TDO_M for correct operation The processor uses seven dedicated pins to access the TAP as shown in Figure 90 and as described in Table 179. Power must be applied and VCCPWRGOOD_0, VCCPWRGOOD_1 and VTTPWRGOOD must be driven high prior to using the TAP. Table 179.
Testability 14.4 TAP Port Timings The TAP port timings are shown in Figure 91 and Table 180. Figure 91. Boundary-Scan Port Timing Waveforms TJRL TRST# TMS TJSU TJH TJSU TJH TDI TJC TJCL TJCH TCK TJCO TJCO TJVZ TDO Table 180. Boundary-Scan Signal Timings Symbol 14.5 Parameter Min TJC Boundary-scan TCK clock period 31.25 Max Unit ns TJCL Boundary-scan TCK clock low time 0.4 * TJC TJCH Boundary-scan TCK clock high time 0.
