Intel 64 and IA-32 Architectures Software Developers Manual Volume 3B, System Programming Guide Part 2

ManualsBrandsIntel ManualsOtherIntel Pentium 4 Processor 2.80 GHz, 512K Cache, 533 MHz FSB

371

372

373

374

375

376

377

378

379

380

Table Of Contents

Chapter 18 Debugging and Performance Monitoring
- 18.1 Overview of Debug Support Facilities
- 18.2 Debug Registers
  - 18.2.1 Debug Address Registers (DR0-DR3)
  - 18.2.2 Debug Registers DR4 and DR5
  - 18.2.3 Debug Status Register (DR6)
  - 18.2.4 Debug Control Register (DR7)
  - 18.2.5 Breakpoint Field Recognition
  - 18.2.6 Debug Registers and Intel® 64 Processors
- 18.3 Debug Exceptions
  - 18.3.1 Debug Exception (#DB)-Interrupt Vector 1
  - 18.3.2 Breakpoint Exception (#BP)-Interrupt Vector 3
- 18.4 Last Branch Recording Overview
- 18.5 Last Branch, Interrupt, and Exception Recording (Intel® Core™2 Duo and Intel® Atom™ Processor Family)
  - 18.5.1 IA32_DEBUGCTL MSR
  - 18.5.2 LBR Stack
  - 18.5.3 BTS and Related Facilities
    - 18.5.3.1 Freezing LBR and Performance Counters on PMI
    - 18.5.3.2 Debug Store (DS) Mechanism
- 18.6 Last Branch, Interrupt, and Exception Recording (Intel® Core™i7 Processor Family)
  - 18.6.1 LBR Stack
  - 18.6.2 Filtering of Last Branch Records
- 18.7 Last Branch, Interrupt, and Exception Recording (Processors based on Intel NetBurst® Microarchitecture)
  - 18.7.1 CPL-Qualified Branch Trace Mechanism
  - 18.7.2 MSR_DEBUGCTLA MSR
  - 18.7.3 LBR Stack for Processors Based on Intel NetBurst Microarchitecture
    - 18.7.3.1 LBR Stack and Intel® 64 Processors
  - 18.7.4 Monitoring Branches, Exceptions, and Interrupts
  - 18.7.5 Single-Stepping on Branches, Exceptions, and Interrupts
  - 18.7.6 Branch Trace Messages
  - 18.7.7 Last Exception Records
    - 18.7.7.1 Last Exception Records and Intel 64 Architecture
  - 18.7.8 Branch Trace Store (BTS)
- 18.8 Last Branch, Interrupt, and Exception Recording (Intel® Core™ Solo and Intel® Core™ Duo Processors)
- 18.9 Last Branch, Interrupt, and Exception Recording (Pentium M Processors)
- 18.10 Last Branch, Interrupt, and Exception Recording (P6 Family Processors)
  - 18.10.1 DEBUGCTLMSR Register
  - 18.10.2 Last Branch and Last Exception MSRs
  - 18.10.3 Monitoring Branches, Exceptions, and Interrupts
- 18.11 Time-Stamp Counter
  - 18.11.1 Invariant TSC
  - 18.11.2 IA32_TSC_AUX Register and RDTSCP Support
- 18.12 Performance Monitoring Overview
- 18.13 Architectural Performance Monitoring
  - 18.13.1 Architectural Performance Monitoring Version 1
    - 18.13.1.1 Architectural Performance Monitoring Version 1 Facilities
  - 18.13.2 Additional Architectural Performance Monitoring Extensions
    - 18.13.2.1 Architectural Performance Monitoring Version 2 Facilities
    - 18.13.2.2 Architectural Performance Monitoring Version 3 Facilities
  - 18.13.3 Pre-defined Architectural Performance Events
- 18.14 Performance Monitoring (Intel® Core™ Solo and Intel® Core™ Duo Processors)
- 18.15 Performance Monitoring (Processors based on Intel® Core™ Microarchitecture)
  - 18.15.1 Fixed-function Performance Counters
  - 18.15.2 Global Counter Control Facilities
  - 18.15.3 At-Retirement Events
  - 18.15.4 Precise Event Based Sampling (PEBS)
- 18.16 Performance Monitoring (Processors based on Intel® Atom™ Microarchitecture)
- 18.17 Performance Monitoring for Processors based on Intel® Microarchitecture (Nehalem)
  - 18.17.1 Enhancements of Performance Monitoring in the Processor Core
  - 18.17.2 Performance Monitoring Facility in the Uncore
- 18.18 Performance Monitoring (Processors Based on Intel NetBurst microarchitecture)
  - 18.18.1 ESCR MSRs
  - 18.18.2 Performance Counters
  - 18.18.3 CCCR MSRs
  - 18.18.4 Debug Store (DS) Mechanism
  - 18.18.5 DS Save Area
    - 18.18.5.1 DS Save Area and IA-32e Mode Operation
  - 18.18.6 Programming the Performance Counters for Non-Retirement Events
  - 18.18.7 At-Retirement Counting
  - 18.18.8 Precise Event-Based Sampling (PEBS)
  - 18.18.9 Operating System Implications
- 18.19 Performance Monitoring and Intel Hyper- Threading Technology in Processors Based on Intel NetBurst Microarchitecture
  - 18.19.1 ESCR MSRs
  - 18.19.2 CCCR MSRs
  - 18.19.3 IA32_PEBS_ENABLE MSR
  - 18.19.4 Performance Monitoring Events
- 18.20 Counting Clocks
  - 18.20.1 Non-Halted Clockticks
  - 18.20.2 Non-Sleep Clockticks
  - 18.20.3 Incrementing the Time-Stamp Counter
  - 18.20.4 Non-Halted Reference Clockticks
  - 18.20.5 Cycle Counting and Opportunistic Processor Operation
- 18.21 Performance Monitoring, Branch Profiling and System Events
- 18.22 Performance Monitoring and Dual-Core Technology
- 18.23 Performance Monitoring on 64-bit Intel Xeon Processor MP with Up to 8-MByte L3 Cache
- 18.24 Performance Monitoring on L3 and Caching Bus Controller sub-systems
  - 18.24.1 Overview of Performance Monitoring with L3/Caching Bus Controller
  - 18.24.2 GBSQ Event Interface
  - 18.24.3 GSNPQ Event Interface
  - 18.24.4 FSB Event Interface
    - 18.24.4.1 FSB Sub-Event Mask Interface
  - 18.24.5 Common Event Control Interface
- 18.25 Performance Monitoring (P6 Family Processor)
  - 18.25.1 PerfEvtSel0 and PerfEvtSel1 MSRs
  - 18.25.2 PerfCtr0 and PerfCtr1 MSRs
  - 18.25.3 Starting and Stopping the Performance-Monitoring Counters
  - 18.25.4 Event and Time-Stamp Monitoring Software
  - 18.25.5 Monitoring Counter Overflow
- 18.26 Performance Monitoring (Pentium Processors)
  - 18.26.1 Control and Event Select Register (CESR)
  - 18.26.2 Use of the Performance-Monitoring Pins
  - 18.26.3 Events Counted
Chapter 19 Introduction to Virtual-Machine Extensions
- 19.1 Overview
- 19.2 Virtual Machine Architecture
- 19.3 Introduction to VMX Operation
- 19.4 Life Cycle of VMM Software
- 19.5 Virtual-Machine Control Structure
- 19.6 Discovering Support for VMX
- 19.7 Enabling and Entering VMX Operation
- 19.8 Restrictions on VMX Operation
Chapter 20 Virtual-Machine Control Structures
- 20.1 Overview
- 20.2 Format of the VMCS Region
- 20.3 Organization of VMCS Data
- 20.4 Guest-State Area
  - 20.4.1 Guest Register State
  - 20.4.2 Guest Non-Register State
- 20.5 Host-State Area
- 20.6 VM-Execution Control Fields
- 20.7 VM-Exit Control Fields
  - 20.7.1 VM-Exit Controls
  - 20.7.2 VM-Exit Controls for MSRs
- 20.8 VM-Entry Control Fields
- 20.9 VM-Exit Information Fields
- 20.10 Software Access to the VMCS and Related Structures
- 20.11 Using VMCLEAR to Initialize a VMCS Region
Chapter 21 VMX Non-Root Operation
- 21.1 Instructions That Cause VM Exits
- 21.2 APIC-Access VM Exits
- 21.3 Other Causes of VM Exits
- 21.4 Changes to Instruction Behavior in VMX Non- Root Operation
- 21.5 APIC Accesses That Do Not Cause VM Exits
- 21.6 Other Changes in VMX Non-Root Operation
  - 21.6.1 Event Blocking
  - 21.6.2 Treatment of Task Switches
- 21.7 Features Specific to VMX Non-Root Operation
Chapter 22 VM Entries
- 22.1 Basic VM-Entry Checks
- 22.2 Checks on VMX Controls and Host-State Area
- 22.3 Checking and Loading Guest State
- 22.4 Loading MSRs
- 22.5 Event Injection
  - 22.5.1 Vectored-Event Injection
    - 22.5.1.1 Details of Vectored-Event Injection
    - 22.5.1.2 VM Exits During Event Injection
  - 22.5.2 Injection of Pending MTF VM Exits
- 22.6 Special Features of VM Entry
- 22.7 VM-Entry Failures During or After Loading Guest State
- 22.8 Machine Checks During VM Entry
Chapter 23 VM Exits
- 23.1 Architectural State Before a VM Exit
- 23.2 Recording VM-Exit Information and Updating VM-Entry Control Fields
- 23.3 Saving Guest State
- 23.4 Saving MSRs
- 23.5 Loading Host State
- 23.6 Loading MSRs
- 23.7 VMX Aborts
- 23.8 Machine Check During VM Exit
Chapter 24 Support for Address Translation
- 24.1 Virtual Processor Identifiers (VPIDs)
- 24.2 Extended Page Tables (EPT)
- 24.3 Caching Translation Information
Chapter 25 System Management
- 25.1 System Management Mode Overview
  - 25.1.1 System Management Mode and VMX Operation
- 25.2 System Management Interrupt (SMI)
- 25.3 Switching Between SMM and the Other Processor Operating Modes
  - 25.3.1 Entering SMM
  - 25.3.2 Exiting From SMM
- 25.4 SMRAM
  - 25.4.1 SMRAM State Save Map
    - 25.4.1.1 SMRAM State Save Map and Intel 64 Architecture
  - 25.4.2 SMRAM Caching
- 25.5 SMI Handler Execution Environment
- 25.6 Exceptions and Interrupts Within SMM
- 25.7 Managing Synchronous and Asynchronous System Management Interrupts
  - 25.7.1 I/O State Implementation
- 25.8 NMI Handling While in SMM
- 25.9 SMM Revision Identifier
- 25.10 Auto HALT Restart
  - 25.10.1 Executing the HLT Instruction in SMM
- 25.11 SMBASE Relocation
  - 25.11.1 Relocating SMRAM to an Address Above 1 MByte
- 25.12 I/O Instruction Restart
  - 25.12.1 Back-to-Back SMI Interrupts When I/O Instruction Restart Is Being Used
- 25.13 SMM Multiple-Processor Considerations
- 25.14 Default Treatment of SMIs and SMM with VMX Operation and SMX Operation
- 25.15 Dual-Monitor Treatment of SMIs and SMM
- 25.16 SMI and Processor Extended State Management
Chapter 26 Virtual-Machine Monitor Programming Considerations
- 26.1 VMX System Programming Overview
- 26.2 Supporting Processor Operating Modes in Guest Environments
  - 26.2.1 Emulating Guest Execution
- 26.3 Managing VMCS Regions and Pointers
- 26.4 Using VMX Instructions
- 26.5 VMM Setup & Tear Down
  - 26.5.1 Algorithms for Determining VMX Capabilities
- 26.6 Preparation and Launching a Virtual Machine
- 26.7 Handling of VM Exits
  - 26.7.1 Handling VM Exits Due to Exceptions
    - 26.7.1.1 Reflecting Exceptions to Guest Software
    - 26.7.1.2 Resuming Guest Software after Handling an Exception
- 26.8 Multi-Processor Considerations
- 26.9 32-Bit and 64-Bit Guest Environments
- 26.10 Handling Model Specific Registers
- 26.11 Handling Accesses to Control Registers
- 26.12 Performance Considerations
Chapter 27 Virtualization of System Resources
- 27.1 Overview
- 27.2 Virtualization Support for Debugging Facilities
  - 27.2.1 Debug Exceptions
- 27.3 Memory Virtualization
- 27.4 Microcode Update Facility
  - 27.4.1 Early Load of Microcode Updates
  - 27.4.2 Late Load of Microcode Updates
Chapter 28 Handling Boundary Conditions in a Virtual Machine Monitor
- 28.1 Overview
- 28.2 Interrupt Handling in VMX Operation
- 28.3 External Interrupt Virtualization
- 28.4 Error Handling by VMM
  - 28.4.1 VM-Exit Failures
  - 28.4.2 Machine Check Considerations
- 28.5 Handling Activity States by VMM
Appendix A Performance-Monitoring Events
- A.1 Architectural Performance-Monitoring Events
- A.2 Performance Monitoring Events for Intel® Intel® Core™i7 Processor Family
- A.3 Performance Monitoring Events for Intel® Xeon® Processor 5200, 5400 Series and Intel® Core™2 Extreme ProcessorS QX 9000 Series
- A.4 Performance Monitoring Events for Intel® Xeon® Processor 3000, 3200, 5100, 5300 Series and Intel® Core™2 Duo ProcessorS
- A.5 Performance Monitoring Events for Intel® Atom™ ProcessorS
- A.6 Performance Monitoring Events for Intel® Core™ Solo and Intel® Core™ Duo ProcessorS
- A.7 Pentium 4 and Intel Xeon Processor Performance-Monitoring Events
- A.8 Performance Monitoring Events for Intel® Pentium® M ProcessorS
- A.9 P6 Family Processor Performance- Monitoring Events
- A.10 Pentium Processor Performance- Monitoring Events
Appendix B Model-Specific Registers (MSRs)
- B.1 Architectural MSRs
- B.2 MSRs In the Intel® Core™ 2 Processor Family
- B.3 MSRs In the Intel® Atom™ Processor Family
- B.4 MSRs In the Intel® Microarchitecture (Nehalem)
- B.5 MSRs In the Pentium® 4 and Intel® Xeon® Processors
  - B.5.1 MSRs Unique to Intel Xeon Processor MP with L3 Cache
- B.6 MSRs In Intel® Core™ Solo and Intel® Core™ Duo Processors
- B.7 MSRs In the Pentium M Processor
- B.8 MSRs In the P6 Family Processors
- B.9 MSRs in Pentium Processors
Appendix C MP Initialization For P6 Family Processors
- C.1 Overview of the MP Initialization Process For P6 Family Processors
- C.2 MP Initialization Protocol Algorithm
  - C.2.1 Error Detection and Handling During the MP Initialization Protocol
Appendix D Programming the LINT0 and LINT1 Inputs
- D.1 Constants
- D.2 LINT[0:1] Pins Programming Procedure
Appendix E Interpreting Machine-Check Error Codes
- E.1 Incremental Decoding Information: Processor Family 06H Machine Error Codes For Machine Check
- E.2 Incremental Decoding Information: Intel Core 2 Processor Family Machine Error Codes For Machine Check
  - E.2.1 Model-Specific Machine Check Error Codes for Intel Xeon Processor 7400 Series
    - E.2.1.1 Processor Machine Check Status Register Incremental MCA Error Code Definition
  - E.2.2 Intel Xeon Processor 7400 Model Specific Error Code Field
    - E.2.2.1 Processor Model Specific Error Code Field Type B: Bus and Interconnect Error
    - E.2.2.2 Processor Model Specific Error Code Field Type C: Cache Bus Controller Error
- E.3 Incremental Decoding Information: Processor Family with CPUID DisplayFamily_DisplayModel Signature 06_1AH, Machine Error Codes For Machine Check
- E.4 Incremental Decoding Information: Processor Family 0FH Machine Error Codes For Machine Check
Appendix F APIC Bus Message Formats
- F.1 Bus Message Formats
- F.2 EOI Message
Appendix G VMX Capability Reporting Facility
- G.1 Basic VMX Information
- G.2 Reserved Controls and Default Settings
- G.3 VM-Execution Controls
- G.4 VM-Exit Controls
- G.5 VM-Entry Controls
- G.6 Miscellaneous Data
- G.7 VMX-Fixed Bits in CR0
- G.8 VMX-Fixed Bits in CR4
- G.9 VMCS Enumeration
- G.10 VPID and EPT Capabilities
Appendix H Field Encoding in VMCS
- H.1 16-Bit Fields
- H.2 64-Bit Fields
- H.3 32-Bit Fields
- H.4 Natural-Width Fields
Appendix I VMX Basic Exit Reasons

Vol. 3 26-15

VIRTUAL-MACHINE MONITOR PROGRAMMING CONSIDERATIONS

26.8 MULTI-PROCESSOR CONSIDERATIONS

The most common VMM design will be the symmetric VMM. This type of VMM runs the

same VMM binary on all logical processors. Like a symmetric operating system, the

symmetric VMM is written to ensure all critical data is updated by only one processor

at a time, IO devices are accessed sequentially, and so forth. Asymmetric VMM

designs are possible. For example, an asymmetric VMM may run its scheduler on one

processor and run just enough of the VMM on other processors to allow the correct

execution of guest VMs. The remainder of this section focuses on the multi-processor

considerations for a symmetric VMM.

A symmetric VMM design does not preclude asymmetry in its operations. For

example, a symmetric VMM can support asymmetric allocation of logical processor

resources to guests. Multiple logical processors can be brought into a single guest

environment to support an MP-aware guest OS. Because an active VMCS can not

control more than one logical processor simultaneously, a symmetric VMM must

make copies of its VMCS to control the VM allocated to support an MP-aware guest

OS. Care must be taken when accessing data structures shared between these

VMCSs. See Section 26.8.4.

Although it may be easier to develop a VMM that assumes a fully-symmetric view of

hardware capabilities (with all processors supporting the same processor feature

sets, including the same revision of VMX), there are advantages in developing a VMM

that comprehends different levels of VMX capability (reported by VMX capability

MSRs). One possible advantage of such an approach could be that an existing soft-

ware installation (VMM and guest software stack) could continue to run without

requiring software upgrades to the VMM, when the software installation is upgraded

to run on hardware with enhancements in the processor’s VMX capabilities. Another

advantage could be that a single software installation image, consisting of a VMM and

guests, could be deployed to multiple hardware platforms with varying VMX capabil-

ities. In such cases, the VMM could fall back to a common subset of VMX features

supported by all VMX revisions, or choose to understand the asymmetry of the VMX

capabilities and assign VMs accordingly.

This section outlines some of the considerations to keep in mind when developing an

MP-aware VMM.

26.8.1 Initialization

Before enabling VMX, an MP-aware VMM must check to make sure that all processors

in the system are compatible and support features required. This can be done by:

• Checking the CPUID on each logical processor to ensure VMX is supported and

that the overall feature set of each logical processor is compatible.

• Checking VMCS revision identifiers on each logical processor.

• Checking each of the “allowed-1” or “allowed-0” fields of the VMX capability

MSR’s on each processor.