vSphere Networking Update 1 Modified on 12 FEB 2018 VMware vSphere 6.5 VMware ESXi 6.5 vCenter Server 6.
vSphere Networking You can find the most up-to-date technical documentation on the VMware website at: https://docs.vmware.com/ If you have comments about this documentation, submit your feedback to docfeedback@vmware.com VMware, Inc. 3401 Hillview Ave. Palo Alto, CA 94304 www.vmware.com Copyright © 2009–2018 VMware, Inc. All rights reserved. Copyright and trademark information. VMware, Inc.
Contents About vSphere Networking Updated Information 10 11 1 Introduction to Networking 12 Networking Concepts Overview Network Services in ESXi 12 14 VMware ESXi Dump Collector Support 14 2 Setting Up Networking with vSphere Standard Switches 16 vSphere Standard Switches 16 Create a vSphere Standard Switch 18 Port Group Configuration for Virtual Machines Add a Virtual Machine Port Group 19 20 Edit a Standard Switch Port Group 21 Remove a Port Group from a vSphere Standard Switch vSphere S
vSphere Networking Managing Networking on Host Proxy Switches 46 Migrate Network Adapters on a Host to a vSphere Distributed Switch 46 Migrate a VMkernel Adapter on a Host to a vSphere Standard Switch 47 Assign a Physical NIC of a Host to a vSphere Distributed Switch Remove a Physical NIC from a vSphere Distributed Switch Removing NICs from Active Virtual Machines Distributed Port Groups 48 48 48 49 Add a Distributed Port Group 49 Edit General Distributed Port Group Settings 52 Configure Ove
vSphere Networking Set the Link Aggregation Group as Active in the Teaming and Failover Order of the Distributed Port Group 81 Edit a Link Aggregation Group 82 Enable LACP 5.
vSphere Networking Edit the Resource Allocation Policy on a Distributed Port Monitoring Policy 116 117 Enable or Disable NetFlow Monitoring on a Distributed Port Group or Distributed Port Traffic Filtering and Marking Policy 118 Traffic Filtering and Marking on a Distributed Port Group or Uplink Port Group Traffic Filtering and Marking on a Distributed Port or Uplink Port Qualifying Traffic for Filtering and Marking 118 126 134 Manage Policies for Multiple Port Groups on a vSphere Distributed Swit
vSphere Networking Enable Jumbo Frames on a vSphere Standard Switch Enable Jumbo Frames for a VMkernel Adapter 169 Enable Jumbo Frame Support on a Virtual Machine TCP Segmentation Offload 168 169 170 Enable or Disable Software TSO in the VMkernel 170 Determine Whether TSO Is Supported on the Physical Network Adapters on an ESXi Host 171 Enable or Disable TSO on an ESXi Host 171 Determine Whether TSO Is Enabled on an ESXi Host Enable or Disable TSO on a Linux Virtual Machine 172 172 Enable or D
vSphere Networking Delete a Network Resource Pool 197 Move a Physical Adapter Out the Scope of Network I/O Control Working with Network I/O Control Version 2 197 198 Create a Network Resource Pool in Network I/O Control Version 2 199 Edit the Settings of a Network Resource Pool in Network I/O Control Version 2 200 12 MAC Address Management 202 MAC Address Assignment from vCenter Server VMware OUI Allocation 202 203 Prefix-Based MAC Address Allocation Range-Based MAC Address Allocation Assigning
vSphere Networking View Port Mirroring Session Details 241 Edit Port Mirroring Session Details, Sources, and Destinations vSphere Distributed Switch Health Check 243 Enable or Disable vSphere Distributed Switch Health Check View vSphere Distributed Switch Health Status Switch Discovery Protocol 241 244 244 245 Enable Cisco Discovery Protocol on a vSphere Distributed Switch 245 Enable Link Layer Discovery Protocol on a vSphere Distributed Switch View Switch Information 246 247 15 Configuring Pr
About vSphere Networking ® vSphere Networking provides information about configuring networking for VMware vSphere , including how to create vSphere distributed switches and vSphere standard switches. vSphere Networking also provides information on monitoring networks, managing network resources, and networking best practices.
Updated Information This vSphere Networking is updated with each release of the product or when necessary. This table provides the update history of the vSphere Networking. Revision Description 12 FEB 2018 Updated information in Enable IPv6 on an Upgraded vSphere Environment 04 OCT 2017 Minor revisions. EN-002628-00 Initial release. VMware, Inc.
Introduction to Networking 1 The basic concepts of ESXi networking and how to set up and configure a network in a vSphere environment are discussed. This chapter includes the following topics: n Networking Concepts Overview n Network Services in ESXi n VMware ESXi Dump Collector Support Networking Concepts Overview A few concepts are essential for a thorough understanding of virtual networking. If you are new to ESXi, it is helpful to review these concepts.
vSphere Networking to it. The switch learns which hosts are connected to which of its ports and uses that information to forward traffic to the correct physical machines. Switches are the core of a physical network. Multiple switches can be connected together to form larger networks. vSphere Standard Switch It works much like a physical Ethernet switch.
vSphere Networking VLAN VLAN enable a single physical LAN segment to be further segmented so that groups of ports are isolated from one another as if they were on physically different segments. The standard is 802.1Q. VMkernel TCP/IP Networking Layer The VMkernel networking layer provides connectivity to hosts and handles the standard infrastructure traffic of vSphere vMotion, IP storage, Fault Tolerance, and vSAN.
vSphere Networking There is no authentication or encryption in the file transfer session from a crashed host to the ESXi Dump Collector. You should configure the ESXi Dump Collector on a separate VLAN when possible to isolate the ESXi core dump from regular network traffic. For information about installing and configuring the ESXi Dump Collector, see the vSphere Installation and Setup documentation. VMware, Inc.
Setting Up Networking with vSphere Standard Switches 2 vSphere standard switches handle network traffic at the host level in a vSphere deployment. This chapter includes the following topics: n vSphere Standard Switches n Create a vSphere Standard Switch n Port Group Configuration for Virtual Machines n vSphere Standard Switch Properties vSphere Standard Switches You can create abstracted network devices called vSphere Standard Switches.
vSphere Networking Figure 2‑1.
vSphere Networking Number of Standard Ports To ensure efficient use of host resources on hosts running ESXi 5.5 and later, the number of ports of standard switches are dynamically scaled up and down. A standard switch on such a host can expand up to the maximum number of ports supported on the host. Create a vSphere Standard Switch Create a vSphere Standard Switch to provide network connectivity for hosts, virtual machines, and to handle VMkernel traffic.
vSphere Networking 7 If you create the new standard switch with a VMkernel adapter or virtual machine port group, enter connection settings for the adapter or the port group. Option Description VMkernel adapter a Enter a label that indicates the traffic type for the VMkernel adapter, for example vMotion. b Set a VLAN ID to identify the VLAN that the network traffic of the VMkernel adapter will use. c Select IPv4, Ipv6 or both. d Select a TCP/IP stack.
vSphere Networking Virtual machines reach physical networks through uplink adapters. A vSphere Standard Switch can transfer data to external networks only when one or more network adapters are attached to it. When two or more adapters are attached to a single standard switch, they are transparently teamed. Add a Virtual Machine Port Group Create port groups on a vSphere Standard Switch to provide connectivity and common network configuration for virtual machines.
vSphere Networking 7 On the Connection settings page, identify traffic through the ports of the group. a Type a Network label for the port group, or accept the generated label. b Set the VLAN ID to configure VLAN handling in the port group. The VLAN ID also reflects the VLAN tagging mode in the port group. c 8 VLAN Tagging Mode VLAN ID Description External Switch Tagging (EST) 0 The virtual switch does not pass traffic associated with a VLAN.
vSphere Networking 10 On the Teaming and failover page, override the teaming and failover settings inherited from the standard switch. You can configure traffic distribution and rerouting between the physical adapters associated with the port group. You can also change the order in which host physical adapters are used upon failure. 11 Click OK.
vSphere Networking Change the Size of the MTU on a vSphere Standard Switch Change the size of the maximum transmission unit (MTU) on a vSphere Standard Switch to improve the networking efficiency by increasing the amount of payload data transmitted with a single packet, that is, enabling jumbo frames. Procedure 1 In the vSphere Web Client, navigate to the host. 2 On the Configure tab, expand Networking and select Virtual switches. 3 Select a standard switch from the table and click Edit settings.
vSphere Networking Procedure 1 In the vSphere Web Client, navigate to the host. 2 On the Configure tab, expand Networking and select Virtual switches. 3 Select the standard switch you want to add a physical adapter to. 4 Click the Manage the physical network adapters connected to the selected switch icon. 5 Add one or more available physical network adapters to the switch. a Click Add adapters. b Select the failover order group to assign the adapters to.
vSphere Networking Figure 2‑2. Topology Diagram of a Standard Switch That Connects the VMkernel and Virtual Machines to the Network VMware, Inc.
Setting Up Networking with vSphere Distributed Switches 3 With vSphere distributed switches you can set up and configure networking in a vSphere environment.
vSphere Networking Figure 3‑1.
vSphere Networking The vSphere Distributed Switch introduces two abstractions that you use to create consistent networking configuration for physical NICs, virtual machines, and VMkernel services. Uplink port group An uplink port group or dvuplink port group is defined during the creation of the distributed switch and can have one or more uplinks. An uplink is a template that you use to configure physical connections of hosts as well as failover and load balancing policies.
vSphere Networking vSphere Distributed Switch Data Flow The data flow from the virtual machines and VMkernel adapters down to the physical network depends on the NIC teaming and load balancing policies that are set to the distributed port groups. The data flow also depends on the port allocation on the distributed switch. Figure 3‑2.
vSphere Networking Figure 3‑3. Packet Flow on the Host Proxy Switch Host 1 VM1 VM2 vmknic1 VMkernel network VM network 0 1 3 Host Proxy Switch 5 6 7 Uplink port group vmnic0 vmnic2 vmnic1 Physical Switch On the host side, the packet flow from virtual machines and VMkernel services passes through particular ports to reach the physical network. For example, a packet sent from VM1 on Host 1 first reaches port 0 on the VM network distributed port group.
vSphere Networking Option Description Distributed Switch: 5.5.0 Compatible with ESXi 5.5 and later. Features released with later vSphere distributed switch versions are not supported. Distributed Switch: 5.1.0 Compatible with VMware ESXi 5.1 and later. Features released with later vSphere distributed switch versions are not supported. Distributed Switch: 5.0.0 Compatible with VMware ESXi 5.0 and later. Features released with later vSphere distributed switch versions are not supported.
vSphere Networking The upgrade of a distributed switch is a nondisruptive operation, that is, the hosts and virtual machines attached to the switch do not experience any downtime. Note To be able to restore the connectivity of the virtual machines and VMkernel adapters if the upgrade fails, back up the configuration of the distributed switch. If the upgrade is not successful, to recreate the switch with its port groups and connected hosts, you can import the switch configuration file.
vSphere Networking For information about converting to enhanced LACP support, see Convert to the Enhanced LACP Support on a vSphere Distributed Switch. For information about converting to Network I/O Control version 3, see Upgrade Network I/O Control to Version 3 on a vSphere Distributed Switch. Edit General and Advanced vSphere Distributed Switch Settings General settings for a vSphere Distributed Switch include the switch name and number of uplinks.
vSphere Networking Option Description Discovery Protocol a Select Cisco Discovery Protocol, Link Layer Discovery Protocol, or (disabled) from the Type drop-down menu. b Set Operation to Listen, Advertise, or Both. For information about Discovery Protocol, see Switch Discovery Protocol. Administrator Contact 6 Type the name and other details of the administrator for the distributed switch. Click OK.
vSphere Networking n Use a Host as a Template to Create a Uniform Networking Configuration on a vSphere Distributed Switch If you plan to have hosts with a uniform networking configuration, you can select a host as a template and apply its configuration for physical NICs and VMkernel adapters to other hosts on the distributed switch. n Remove Hosts from a vSphere Distributed Switch Remove hosts from a vSphere distributed switch if you have configured a different switch for the hosts.
vSphere Networking If you migrate physical NICs, leave at least one active NIC that handles the traffic of port groups. For example, if vmnic0 and vmnic1 handle the traffic of the VM Network port group, migrate vmnic0 and leave vmnic1 connected to the group. Removing Hosts from a vSphere Distributed Switch Before you remove hosts from a distributed switch, you must migrate the network adapters that are in use to a different switch.
vSphere Networking 5 On the Select network adapter tasks page, select the tasks for configuring network adapters to the distributed switch and click Next. 6 On the Manage physical network adapters page, configure physical NICs on the distributed switch. a From the On other switches/unclaimed list, select a physical NIC. If you select physical NICs that are already connected to other switches, they are migrated to the current distributed switch. b Click Assign uplink.
vSphere Networking What to do next Having hosts associated with the distributed switch, you can manage physical NICs, VMkernel adapters, and virtual machine network adapters. Configure Physical Network Adapters on a vSphere Distributed Switch For hosts that are associated with a distributed switch, you can assign physical NICs to uplinks on the switch. You can configure physical NICs on the distributed switch for multiple hosts at a time.
vSphere Networking 11 Review the impacted services as well as the level of impact. Option Description No impact iSCSI will continue its normal function after the new networking configuration is applied. Important impact The normal function of iSCSI might be disrupted if the new networking configuration is applied. Critical impact The normal function of iSCSI will be interrupted if the new networking configuration is applied.
vSphere Networking 10 Review the impacted services as well as the level of impact. Option Description No impact iSCSI will continue its normal function after the new networking configuration is applied. Important impact The normal function of iSCSI might be disrupted if the new networking configuration is applied. Critical impact The normal function of iSCSI will be interrupted if the new networking configuration is applied.
vSphere Networking 9 On the Port properties page, configure the settings for the VMkernel adapter. Option Description Network label The network label is inherited from the label of the distributed port group. IP settings Select IPv4, IPv6, or both. Note The IPv6 option does not appear on hosts that do not have IPv6 enabled. TCP/IP stack Select a TCP/IP stack from the list. Once you set a TCP/IP stack for the VMkernel adapter, you cannot change it later.
vSphere Networking 11 (Optional) On the IPv4 settings page, select an option for obtaining IP addresses. Option Description Obtain IPv4 settings automatically Use DHCP to obtain IP settings. A DHCP server must be present on the network. Use static IPv4 settings Enter the IPv4 IP address and subnet mask for the VMkernel adapter. The VMkernel Default Gateway and DNS server addresses for IPv4 are obtained from the selected TCP/IP stack.
vSphere Networking 5 Click Next. 6 In Select network adapter tasks, select Migrate virtual machine networking and click Next. 7 Configure virtual machine network adapters to the distributed switch. 8 a To connect all network adapters of a virtual machine to a distributed port group, select the virtual machine, or select an individual network adapter to connect only that adapter. b Click Assign port group. c Select a distributed port group from the list and click OK.
vSphere Networking Figure 3‑4. Applying Physical NICs Configuration on a vSphere Distributed Switch by Using a Template Host On the Manage VMkernel network adapters page, assign a VMkernel adapter to a port group and click Apply to all to apply the same configuration to the other host. After you click the Apply to all button, the destination VMkernel adapter has both the Modified and the Reassigned qualifiers.
vSphere Networking Figure 3‑5. Applying VMkernel Adapter Configuration on a vSphere Distributed Switch by Using a Template Host Remove Hosts from a vSphere Distributed Switch Remove hosts from a vSphere distributed switch if you have configured a different switch for the hosts. Prerequisites n Verify that physical NICs on the target hosts are migrated to a different switch. n Verify that VMkernel adapters on the hosts are migrated to a different switch.
vSphere Networking Managing Networking on Host Proxy Switches You can change the configuration of the proxy switch on every host that is associated with a vSphere distributed switch. You can manage physical NICs, VMkernel adapters, and virtual machine network adapters. For details about setting up VMkernel networking on host proxy switches, see Create a VMkernel Adapter on a vSphere Distributed Switch.
vSphere Networking 7 Review the services that are affected from the new networking configuration. a If there is an important or serious impact reported on a service, click the service and review the analysis details. For example, an important impact on iSCSI might be reported as a result from an incorrect teaming and failover configuration on the distributed port group where you migrate the iSCSI VMkernel adapter.
vSphere Networking Assign a Physical NIC of a Host to a vSphere Distributed Switch You can assign physical NICs of a host that is associated with a distributed switch to uplink port on the host proxy switch. Procedure 1 In the vSphere Web Client, navigate to the host. 2 On the Configure tab, expand Networking and select Virtual switches. 3 Select a distributed switch from the list. 4 Click the Manage the physical network adapters connected to the selected switch icon.
vSphere Networking Removing NICs from an Active Virtual Machine with a Guest Operating System Installed You can remove a NIC from an active virtual machine, but it might not be reported to the vSphere Web Client for some time. If you click Edit Settings for the virtual machine, you might see the removed NIC listed even after the task is complete. The Edit Settings dialog box for the virtual machine does not immediately display the removed NIC.
vSphere Networking Setting Description Network resource pool Use the drop-down menu to assign the new distributed port group to a userdefined network resource pool. If you have not created a network resource pool, this menu is empty. VLAN Advanced 5 Use the VLAN type drop-down menu to select VLAN options: n None: Do not use VLAN. n VLAN: In the VLAN ID text box, enter a number between 1 and 4094. n VLAN trunking: Enter a VLAN trunk range. n Private VLAN: Select a private VLAN entry.
vSphere Networking 7 Setting Description Peak bandwidth The maximum number of bits per second to allow across a port when it is sending and receiving a burst of traffic. This tops the bandwidth used by a port whenever it is using its burst bonus. Burst size The maximum number of bytes to allow in a burst. If this parameter is set, a port might gain a burst bonus when it does not use all its allocated bandwidth.
vSphere Networking Setting Description Failback Select Yes or No to disable or enable failback. This option determines how a physical adapter is returned to active duty after recovering from a failure. If failback is set to Yes (default), the adapter is returned to active duty immediately upon recovery, displacing the standby adapter that took over its slot, if any.
vSphere Networking 3 Select General to edit the following distributed port group settings. Option Description Name The name of distributed port group. You can edit the name in the text field. Port binding Choose when ports are assigned to virtual machines connected to this distributed port group. Port allocation 4 n Static binding: Assign a port to a virtual machine when the virtual machine connects to the distributed port group.
vSphere Networking 4 (Optional) Use the policy pages to set overrides for each port policy. 5 Click OK. Remove a Distributed Port Group Remove a distributed port group when you no longer need the corresponding labeled network to provide connectivity and configure connection settings for virtual machines or VMkernel networking. Prerequisites n Verify that all virtual machines connected to the corresponding labeled network are migrated to a different labeled network.
vSphere Networking 4 Click the Start Monitoring Port State icon. The ports table for the distributed port group displays runtime statistics for each distributed port. The State column displays the current state for each distributed port. Option Description Link Up The link for this distributed port is up. Link Down The link for this distributed port is down. Blocked This distributed port is blocked. -- The state of this distributed port is currently unavailable.
vSphere Networking Migrate Virtual Machines to or from a vSphere Distributed Switch In addition to connecting virtual machines to a distributed switch at the individual virtual machine level, you can migrate a group of virtual machines between a vSphere Distributed Switch network and a vSphere Standard Switch network. Procedure 1 In the vSphere Web Client, navigate to a data center. 2 Right-click the data center in the navigator and select Migrate VMs to Another Network. 3 Select a source network.
vSphere Networking Topology Diagrams of a vSphere Distributed Switch in the vSphere Web Client The topology diagrams of a vSphere Distributed Switch in the vSphere Web Client show the structure of virtual machine adapters, VMkernel adapters, and physical adapters in the switch. You can examine the components, arranged in port groups, whose traffic is handled by the switch, and the connections between them.
vSphere Networking 2 On the Configure tab, expand Settings and selectTopology. By default the diagram shows up to 32 distributed port groups, 32 hosts, and 1024 virtual machines. Example: Diagram of a Distributed Switch That Connects the VMkernel and Virtual Machines to the Network In your virtual environment, a vSphere Distributed Switch handles VMkernel adapters for vSphere vMotion and for the management network, and virtual machines grouped.
vSphere Networking n Handle networking components on multiple hosts by using the Add and Manage Hosts wizard. n View the physical NIC or NIC team that carries the traffic related to a selected virtual machine adapter or VMkernel adapter. In this way you can also view the host on which a selected VMkernel adapter resides. Select the adapter, trace the route to the associated physical NIC, and view the IP address or domain name next to the NIC. n Determine the VLAN mode and ID for a port group.
Setting Up VMkernel Networking 4 You set up VMkernel adapters to provide network connectivity to hosts and to accommodate system traffic of vMotion, IP storage, Fault Tolerance logging, vSAN, and so on. n VMkernel Networking Layer The VMkernel networking layer provides connectivity to hosts and handles the standard system traffic of vSphere vMotion, IP storage, Fault Tolerance, vSAN, and others.
vSphere Networking n View TCP/IP Stack Configuration on a Host You can view the DNS and routing configuration of a TCP/IP stack on a host. You can also view the IPv4 and IPv6 routing tables, the congestion control algorithm, and the maximum number of allowed connections. n Change the Configuration of a TCP/IP Stack on a Host You can change the DNS and default gateway configuration of a TCP/IP stack on a host.
vSphere Networking configured with the provisioning TCP/IP stack handle the traffic from cloning the virtual disks of the migrated virtual machines in long-distance vMotion. By using the provisioning TCP/IP stack, you can isolate the traffic from the cloning operations on a separate gateway. After you configure a VMkernel adapter with the provisioning TCP/IP stack, all adapters on the default TCP/IP stack are disabled for the Provisioning traffic.
vSphere Networking IP storage traffic and discovery Handles the connection for storage types that use standard TCP/IP networks and depend on the VMkernel networking. Such storage types are software iSCSI, dependent hardware iSCSI, and NFS. If you have two or more physical NICs for iSCSI, you can configure iSCSI multipathing. ESXi hosts support NFS 3 and 4.1. To configure a software Fibre Channel over Ethernet (FCoE) adapter, you must have a dedicated VMkernel adapter.
vSphere Networking Tab Description IP Settings Displays all IPv4 and IPv6 settings for the VMkernel adapter. IPv6 information is not displayed if IPv6 has not been enabled on the host. Policies Displays the configured traffic shaping, teaming and failover, and security policies that apply for the port group to which the VMkernel adapter is connected.
vSphere Networking 8 Option Description TCP/IP stack Select a TCP/IP stack from the list. After you set a TCP/IP stack for the VMkernel adapter, you cannot change it later. If you select the vMotion or the Provisioning TCP/IP stack, you will be able to use only this stack to handle vMotion or Provisioning traffic on the host. All VMkernel adapters for vMotion on the default TCP/IP stack are disabled for future vMotion sessions.
vSphere Networking 10 (Optional) On the IPv6 settings page, select an option for obtaining IPv6 addresses. Option Description Obtain IPv6 addresses automatically through DHCP Use DHCP to obtain IPv6 addresses. A DHCPv6 server must be present on the network. Obtain IPv6 addresses automatically through Router Advertisement Use router advertisement to obtain IPv6 addresses. Static IPv6 addresses In ESXi 6.
vSphere Networking 7 Option Description TCP/IP stack Select a TCP/IP stack from the list. Once you set a TCP/IP stack for the VMkernel adapter, you cannot change it later. If you select the vMotion or the Provisioning TCP/IP stack, you will be able to use only these stacks to handle vMotion or Provisioning traffic on the host. All VMkernel adapters for vMotion on the default TCP/IP stack are disabled for future vMotion sessions.
vSphere Networking 9 (Optional) On the IPv6 settings page, select an option for obtaining IPv6 addresses. Option Description Obtain IPv6 addresses automatically through DHCP Use DHCP to obtain IPv6 addresses. A DHCPv6 server must be present on the network. Obtain IPv6 addresses automatically through Router Advertisement Use router advertisement to obtain IPv6 addresses. Static IPv6 addresses In ESXi 6.
vSphere Networking 5 On the NIC settings page, set the MTU for the network adapter. 6 With IPv4 enabled, in the IPv4 settings section, select the method by which IP addresses are obtained. Option Description Obtain IPv4 settings automatically Use DHCP to obtain IP settings. A DHCP server must be present on the network. Use static IPv4 settings Enter the IPv4 IP address and subnet mask for the VMkernel adapter.
vSphere Networking For example, the VMkernel adapters vmk0 and vmk1 can be configured on a host. n vmk0 is used for management traffic on the 10.162.10.0/24 subnet, with default gateway 10.162.10.1 n vmk1 is used for vMotion traffic on the 172.16.1.0/24 subnet If you set 172.16.1.1 as the default gateway for vmk1, vMotion uses vmk1 as its egress interface with the gateway 172.16.1.1. The 172.16.1.1 gateway is a part of the vmk1 configuration and is not in the routing table.
vSphere Networking DNS and routing details about the selected TCP/IP stack appear below the TCP/IP Stacks table. You can view the IPv4 and IPv6 routing tables, and the DNS and routing configuration for the stack. Note The IPv6 routing table is only visible if IPv6 is enabled on the host. The Advanced tab contains information about the configured congestion control algorithm and the maximum number of allowed connections to the stack.
vSphere Networking Create a Custom TCP/IP Stack You can create a custom TCP/IP stack on a host to forward networking traffic through a custom application. Procedure 1 Open an SSH connection to the host. 2 Log in as the root user. 3 Run the vSphere CLI command. esxcli network ip netstack add -N="stack_name" The custom TCP/IP stack is created on the host. You can assign VMkernel adapters to the stack.
LACP Support on a vSphere Distributed Switch 5 With LACP support on a vSphere Distributed Switch, you can connect ESXi hosts to physical switches by using dynamic link aggregation. You can create multiple link aggregation groups (LAGs) on a distributed switch to aggregate the bandwidth of physical NICs on ESXi hosts that are connected to LACP port channels. VMware, Inc.
vSphere Networking Figure 5‑1.
vSphere Networking On a host proxy switch, you can connect one physical NIC to only one LAG port. On the distributed switch, one LAG port can have multiple physical NICs from different hosts connected to it. The physical NICs on a host that you connect to the LAG ports must be connected to links that participate in an LACP port channel on the physical switch. You can create up to 64 LAGs on a distributed switch. A host can support up to 32 LAGs.
vSphere Networking If the conversion to the enhanced LACP support fails, see vSphere Troubleshooting for details about how to complete it manually. Prerequisites n Verify that the vSphere Distributed Switch is version 5.5, 6.0 or 6.5. n Verify that none of the distributed port groups permit overriding their uplink teaming policy on individual ports. n If you convert from an existing LACP configuration, verify that only one uplink port group exists on the distributed switch.
vSphere Networking 7 If you convert from an existing LACP configuration, type the name of the new LAG in the Name text field. 8 Click Next to review the details about the conversion and click Finish. You converted to the Enhanced LACP support on the vSphere Distributed Switch. What to do next Create LAGs on the distributed switch to aggregate the bandwidth of multiple physical NICs on the associated hosts.
vSphere Networking Procedure 1 Create a Link Aggregation Group To migrate the network traffic of distributed port groups to a link aggregation group (LAG), you create a new LAG on the distributed switch. 2 Set a Link Aggregating Group as Standby in the Teaming and Failover Order of Distributed Port Groups The new link aggregation group (LAG) by default is unused in the teaming and failover order of distributed port groups.
vSphere Networking 6 Select the LACP negotiating mode of the LAG. Option Description Active All LAG ports are in an Active negotiating mode. The LAG ports initiate negotiations with the LACP port channel on the physical switch by sending LACP packets. Passive The LAG ports are in Passive negotiating mode. They respond to LACP packets they receive but do not initiate LACP negotiation.
vSphere Networking 3 Select Teaming and failover and click Next. 4 Select the port groups where you want to use the LAG. 5 In Failover order, select the LAG and use the up arrow to move it to the Standby uplinks list. 6 Click Next, review the message that informs you about the usage of the intermediate teaming and failover configuration, and click OK. 7 On the Ready to complete page, click Finish. What to do next Migrate physical NICs from standalone uplinks to the LAG ports.
vSphere Networking What to do next Set the LAG as active and all standalone uplinks to unused in the teaming and failover order of distributed port groups. Set the Link Aggregation Group as Active in the Teaming and Failover Order of the Distributed Port Group You migrated physical NICs to the ports of the link aggregation group (LAG). Set the LAG as active and move all standalone uplinks as unused in the teaming and failover order of the distributed port groups.
vSphere Networking Edit a Link Aggregation Group Edit the settings of a link aggregation group ( LAG) if you need to add more ports to the group or change the LACP negotiating mode, the load balancing algorithm, or the VLAN and NetFlow policies. Procedure 1 In the vSphere Web Client, navigate to the vSphere Distributed Switch. 2 On the Configure tab, expand Settings and select LACP. 3 Click the New Link Aggregation Group icon. 4 In the Name text box, type a new name for the LAG.
vSphere Networking n Verify that all physical NICs that are connected to the uplinks have the same speed and are configured at full duplex. Procedure 1 In the vSphere Web Client, navigate to an uplink port group. a Select a distributed switch and click the Networks tab. b Click Uplink Port Groups and select the uplink port group. 2 Click the Configure tab and select Properties. 3 Click Edit. 4 In the LACP section, use the drop-down list to enable LACP.
vSphere Networking n The LACP 5.1 support only works with IP Hash load balancing and Link Status Network failover detection. n The LACP 5.1 support only provides one LAG per distributed switch and per host. VMware, Inc.
Backing Up and Restoring Networking Configurations 6 vSphere 5.1 and later enables you to backup and restore the configuration of a vSphere Distributed Switch , distributed and uplink port groups in cases of invalid changes or a transfer to another deployment.
vSphere Networking Prerequisites Verify that vCenter Server is version 5.1 and later. Procedure 1 In the vSphere Web Client, navigate to the distributed switch. 2 Right-click the distributed switch and select Settings > Export Configuration. 3 Choose to export the distributed switch configuration, or export the distributed switch configuration and all port groups. 4 (Optional) Enter notes about this configuration in the Descriptions field. 5 Click OK.
vSphere Networking 3 Browse to the location of the configuration file. 4 To assign the keys from the configuration file to the switch and its port groups, select the Preserve original distributed switch and port group identifiers check box and click Next. You can use the Preserve original distributed switch and port group identifiers option in the following cases: n Recreate a deleted switch. n Restore a switch whose upgrade has failed.
vSphere Networking Export, Import, and Restore vSphere Distributed Port Group Configurations You can export vSphere distributed port group configurations to a file. The configuration file allows you to preserve valid port group configurations, enabling distribution of these configurations to other deployments. You can export port group information at the same time you export distributed switch configurations. See Backing Up and Restoring a vSphere Distributed Switch Configuration.
vSphere Networking If an existing port group has the same name as the imported port group, the new port group name has a number appended in parentheses. The settings from the imported configuration are applied to the new port group and the settings of the original port group remain unchanged. This functionality is available only with the vSphere Web Client 5.1 or later. However, you can export settings from any version of distributed port if you use the vSphere Web Client 5.1 and later.
Rollback and Recovery of the Management Network 7 In vSphere 5.1 and later, you can prevent and recover from misconfiguration of the management network by using the rollback and recovery support of the vSphere Distributed Switch and vSphere Standard Switch. Rollback is available for use on both standard and distributed switches. To fix invalid configuration of the management network, you can connect directly to a host to fix the issues through the DCUI.
vSphere Networking n Removing the management VMkernel network adapter from a standard or distributed switch. n Removing a physical NIC of a standard or distributed switch containing the management VMkernel network adapter. n Migrating the management VMkernel adapter from vSphere standard to distributed switch. If a network disconnects for any of these reasons, the task fails and the host reverts to the last valid configuration.
vSphere Networking 3 Click Edit. 4 Select the config.vpxd.network.rollback key, and change the value to false. If the key is not present, you can add it and set the value to false. 5 Click OK. 6 Restart vCenter Server to apply the changes. Disable Network Rollback by Using the vCenter Server Configuration File Rollback is enabled by default in vSphere 5.1 and later. You can disable rollback by editing the vpxd.cfg configuration file of vCenter Server directly.
vSphere Networking If the uplinks that you use to restore the management network are also used by VMkernel adapters that handle other types of traffic (vMotion, Fault Tolerance, and so on), the adapters loose network connectivity after the restore. For more information about accessing and using the DCUI, see the vSphere Security documentation. Note Recovery of the management connection on a distributed switch is not supported on stateless ESXi instances.
Networking Policies 8 Policies set at the standard switch or distributed port group level apply to all of the port groups on the standard switch or to ports in the distributed port group. The exceptions are the configuration options that are overridden at the standard port group or distributed port level. Watch the video about applying networking policies on vSphere standard and distributed switches. Working with Networking Policies (http://link.brightcove.
vSphere Networking n Resource Allocation Policy The Resource Allocation policy allows you to associate a distributed port or port group with a usercreated network resource pool. This policy provides you with greater control over the bandwidth given to the port or port group. n Monitoring Policy The monitoring policy enables or disables NetFlow monitoring on a distributed port or port group. n Traffic Filtering and Marking Policy In a vSphere distributed switch 5.
vSphere Networking Table 8‑2. Policies Available for a vSphere Standard Switch and vSphere Distributed Switch Policy Standard Switch Distributed Switch Teaming and failover Yes Yes Lets you configure the physical NICs that handle the network traffic for a standard switch, standard port group, distributed port group, or distributed port. You arrange the physical NICs in a failover order and apply different load balancing policies over them.
vSphere Networking 3 Select the Advanced page. Option Description Configure reset at disconnect From the drop-down menu, enable or disable reset at disconnect. When a distributed port is disconnected from a virtual machine, the configuration of the distributed port is reset to the distributed port group setting. Any per-port overrides are discarded. Override port policies Select the distributed port group policies to be overridden on a per-port level.
vSphere Networking Network Failure Detection Policy You can specify one of the following methods that a virtual switch uses for failover detection. Link status only Beacon probing Relies only on the link status that the network adapter provides. Detects failures, such as removed cables and physical switch power failures. However, link status does not detect the following configuration errors: n Physical switch port that is blocked by spanning tree or is misconfigured to the wrong VLAN .
vSphere Networking Notify Switches Policy By using the notify switches policy, you can determine how the ESXi host communicates failover events. When a physical NIC connects to the virtual switch or when traffic is rerouted to a different physical NIC in the team, the virtual switch sends notifications over the network to update the lookup tables on physical switches. Notifying the physical switch offers lowest latency when a failover or a migration with vSphere vMotion occurs.
vSphere Networking Each virtual machine running on an ESXi host has an associated virtual port ID on the virtual switch. To calculate an uplink for a virtual machine, the virtual switch uses the virtual machine port ID and the number of uplinks in the NIC team. After the virtual switch selects an uplink for a virtual machine, it always forwards traffic through the same uplink for this virtual machine as long as the machine runs on the same port.
vSphere Networking Table 8‑4. Considerations on Using Route Based on Source MAC Hash Considerations Description Advantages n A more even distribution of the traffic than Route Based on Originating Virtual Port, because the virtual switch calculates an uplink for every packet. n Virtual machines use the same uplink because the MAC address is static. Powering a virtual machine on or off does not change the uplink that the virtual machine uses. n No changes on the physical switch are required.
vSphere Networking Physical Switch Configuration To ensure that IP hash load balancing works correctly, you must have an Etherchannel configured on the physical switch. An Etherchannel bonds multiple network adapters into a single logical link. When ports are bound into an Etherchannel, every time the physical switch receives a packet from the same virtual machine MAC address on different ports, the switch updates its content addressable memory (CAM) table correctly.
vSphere Networking Route Based on Physical NIC Load Route Based on Physical NIC Load is based on Route Based on Originating Virtual Port, where the virtual switch checks the actual load of the uplinks and takes steps to reduce it on overloaded uplinks. Available only for vSphere Distributed Switch. The distributed switch calculates uplinks for virtual machines by taking their port ID and the number of uplinks in the NIC team.
vSphere Networking 3 Navigate to the Teaming and Failover policy for the standard switch, or standard port group. Option Action Standard Switch a Select the switch from the list. b Click Edit settings and select Teaming and failover. a Select the switch where the port group resides. b From the switch topology diagram, select the standard port group and click Edit settings. c Select Teaming and failover. d Select Override next to the policies that you want to override.
vSphere Networking 7 From the Failback drop-down menu, select whether a physical adapter is returned to active status after recovering from a failure. If failback is set to Yes, the default selection, the adapter is returned to active duty immediately upon recovery, displacing the standby adapter that took over its slot, if any. If failback is set to No for a standard port, a failed adapter is left inactive after recovery until another currently active adapter fails and must be replaced.
vSphere Networking 2 Navigate the Teaming and Failover policy on the distributed port group or port. Option Action Distributed port group a From the Actions menu, select Distributed Port Group > Manage Distributed Port Groups. b Select Teaming and failover. c Select the port group and click Next. a On the Networks tab, click Distributed Port Groups and double-click a distributed port group. Distributed port 3 b On the Ports tab, select a port and click Edit distributed port settings.
vSphere Networking 5 From the Notify switches drop-down menu, select whether the standard or distributed switch notifies the physical switch in case of a failover. Note Set this option to No if a connected virtual machine is using Microsoft Network Load Balancing in unicast mode. No issues exist with Network Load Balancing running in multicast mode. 6 From the Failback drop-down menu, select whether a physical adapter is returned to active status after recovering from a failure.
vSphere Networking Configure VLAN Tagging on a Distributed Port Group or Distributed Port To apply VLAN tagging globally on all distributed ports, you must set the VLAN policy on a distributed port group. To integrate the virtual traffic on the port with physical VLANs in a different way from the parent distributed port group, you must use the VLAN policy on a distributed port. Prerequisites To override a policy on distributed port level, enable the port-level override option for this policy.
vSphere Networking Configure VLAN Tagging on an Uplink Port Group or Uplink Port To configure VLAN traffic processing generally for all member uplinks, you must set the VLAN policy on an uplink port. To handle VLAN traffic through the port in a different way than for the parent uplink port group, you must set the VLAN policy on an uplink . Use the VLAN policy at the uplink port level to propagate a trunk range of VLAN IDs to the physical network adapters for traffic filtering.
vSphere Networking The security policy of a standard or distributed switch is implemented in Layer 2 (Data Link Layer) of the network protocol stack. The three elements of the security policy are promiscuous mode, MAC address changes, and forged transmits. See the vSphere Security documentation for information about potential networking threats.
vSphere Networking 4 Reject or accept promiscuous mode activation or MAC address changes in the guest operating system of the virtual machines attached to the standard switch or port group. Option Description Promiscuous mode n Reject. The VM network adapter receives only frames that are addressed to the virtual machine. n Accept.The virtual switch forwards all frames to the virtual machine in compliance with the active VLAN policy for the port to which the VM network adapter is connected.
vSphere Networking 2 Navigate to the Security policy for the distributed port group or port. Option Action Distributed port group a From the Actions menu, select Distributed Port Group > Manage Distributed Port Groups. b Select Security. c Select the port group and click Next. a On the Networks tab, click Distributed Port Groups and double-click a distributed port group . b On the Ports tab, select a port and click the Edit distributed port settings icon. c Select Security.
vSphere Networking ESXi shapes outbound network traffic on standard switches and inbound and outbound traffic on distributed switches. Traffic shaping restricts the network bandwidth available on a port, but can also be configured to allow bursts of traffic to flow through at higher speeds. Average Bandwidth Establishes the number of bits per second to allow across a port, averaged over time. This number is the allowed average load.
vSphere Networking 4 Configure traffic shaping policies. Option Description Status Enables setting limits on the amount of networking bandwidth allocated for each port that is associated with the standard switch or port group. Average Bandwidth Establishes the number of bits per second to allow across a port, averaged over time (the allowed average load). Peak Bandwidth The maximum number of bits per second to allow across a port when it is sending a burst of traffic.
vSphere Networking 2 Navigate to the Traffic Shaping policy for the distributed port group or port. Option Action Distributed port group a From the Actions menu, select Distributed Port Group > Manage Distributed Port Groups. b Select Traffic shaping. c Select the port group and click Next. a On the Networks tab, click Distributed Port Groups and double-click a distributed port group . b On the Ports tab, select a port and click the Edit distributed port settings icon.
vSphere Networking Edit the Resource Allocation Policy on a Distributed Port Group Associate a distributed port group with a network resource pool to give you greater control over the bandwidth that is given to the distributed port group. Prerequisites n Enable Network I/O Control on the distributed switch. See Enable Network I/O Control on a vSphere Distributed Switch. n Create and configure network resource pools. See Create a Network Resource Pool.
vSphere Networking Procedure 1 In the vSphere Web Client, navigate to the distributed switch. 2 On the Configure tab, expand More and click Ports. 3 Select a port from the list and click Edit distributed port settings icon. 4 Select Properties. 5 Under Network Resource Pool, click the Override check box, and from the drop-down menu assign a network resource pool to the port. If you did not enable port-level overrides for the resource allocation policy, the option is disabled.
vSphere Networking 2 Navigate to the monitoring policy for the distributed port group or distributed port. Option Action Distributed port group a Distributed port From the Actions menu, select Distributed Port Group > Manage Distributed Port Groups. b Select Monitoring. c Select the port group and click Next. a On the Networks tab, click Distributed Port Groups and double-click a distributed port group . b On the Ports tab, select a port and click the Edit distributed port settings icon.
vSphere Networking n Filter Traffic on a Distributed Port Group or Uplink Port Group Allow or stop traffic for securing the data that flows through the ports of a distributed port group or uplink port group. n Working with Network Traffic Rules on a Distributed Port Group or Uplink Port Group Define traffic rules in a distributed port group or uplink port group to introduce a policy for processing traffic related to virtual machines or to physical adapters.
vSphere Networking Priority tagging is a mechanism to mark traffic that has higher QoS demands. In this way, the network can recognize different classes of traffic. The network devices can handle the traffic from each class according to its priority and requirements. You can also re-tag traffic to either raise or lower the importance of the flow. By using a low QoS tag, you can restrict data tagged in a guest operating system.
vSphere Networking 8 Specify the kind of traffic that the rule is applicable to. To determine if a data flow is in the scope of a rule for marking or filtering, the vSphere distributed switch examines the direction of the traffic, and properties like source and destination, VLAN, next level protocol, infrastructure traffic type, and so on. a From the Traffic direction drop-down menu, select whether the traffic must be ingress, egress, or both so that the rule recognizes it as matching.
vSphere Networking Rule Parameter Parameter Value Destination port 5060 Source address IP address matches 192.168.2.0 with prefix length 24 Filter Traffic on a Distributed Port Group or Uplink Port Group Allow or stop traffic for securing the data that flows through the ports of a distributed port group or uplink port group. Procedure 1 Locate a distributed port group or an uplink port group in the vSphere Web Client. a Select a distributed switch and click the Networks tab.
vSphere Networking 7 Specify the kind of traffic that the rule is applicable to. To determine if a data flow is in the scope of a rule for marking or filtering, the vSphere distributed switch examines the direction of the traffic, and properties like source and destination, VLAN, next level protocol, infrastructure traffic type, and so on. a From the Traffic direction drop-down menu, select whether the traffic must be ingress, egress, or both so that the rule recognizes it as matching.
vSphere Networking n Edit a Traffic Rule on a Distributed Port Group or Uplink Port Group Create or edit traffic rules, and use their parameters to configure a policy for filtering or marking the traffic on a distributed port group or uplink port group. n Change Rule Priorities on a Distributed Port Group or Uplink Port Group Reorder the rules that form the traffic filtering and marking policy of a distributed port group or uplink port group to change the sequence of actions for processing traffic.
vSphere Networking 4 If traffic filtering and marking is disabled, enable it from the Status drop-down menu. 5 Click New to create a new rule, or select a rule and click Edit to edit it. What to do next Name the network traffic rule, and deny, allow, or tag the target traffic.
vSphere Networking Disable Traffic Filtering and Marking on a Distributed Port Group or Uplink Port Group Let traffic flow to virtual machines or physical adapters without additional control related to security or QoS by disabling the traffic filtering and marking policy. Note You can enable and set up the traffic filtering and marking policy on a particular port. See Enable Traffic Filtering and Marking on a Distributed Port or Uplink Port.
vSphere Networking n Disable Traffic Filtering and Marking on a Distributed Port or Uplink Port Disable the traffic filtering and marking policy on a port to let traffic flow to a virtual machine or a physical adapter without filtering for security or marking for QoS.
vSphere Networking Prerequisites To override a policy on distributed port level, enable the port-level override option for this policy. See Configure Overriding Networking Policies on Port Level. Procedure 1 Navigate to a distributed switch and then navigate to a distributed port or an uplink port. n To navigate to the distributed ports of the switch, click Networks > Distributed Port Groups, double-click a distributed port group from the list, and click the Ports tab.
vSphere Networking 8 Specify the kind of traffic that the rule is applicable to. To determine if a data flow is in the scope of a rule for marking or filtering, the vSphere distributed switch examines the direction of the traffic, and properties like source and destination, VLAN, next level protocol, infrastructure traffic type, and so on. a From the Traffic direction drop-down menu, select whether the traffic must be ingress, egress, or both so that the rule recognizes it as matching.
vSphere Networking 2 Select a port from the list. 3 Click Edit distributed port settings. 4 If traffic filtering and marking is not enabled at the port level, click Override, and from the Status drop-down menu, select Enabled. 5 Click New to create a new rule, or select a rule and click Edit to edit it. You can change a rule inherited from the distributed port group or uplink port group. In this way, the rule becomes unique within the scope of the port.
vSphere Networking Working with Network Traffic Rules on a Distributed Port or Uplink Port Define traffic rules in a distributed port or uplink port group to introduce a policy for processing traffic related to a virtual machine or to a physical adapter. You can filter specific traffic or describe its QoS demands. n View Traffic Rules on a Distributed Port or Uplink Port Review the traffic rules that form the traffic filtering and marking policy of a distributed port or uplink port.
vSphere Networking 7 From the upper list, select the rule for which you want to view the criteria for locating traffic. The traffic qualifying parameters of the rule appear in the Traffic Qualifiers list. Edit a Traffic Rule on a Distributed Port or Uplink Port Create or edit traffic rules, and use their parameters to configure a policy for filtering or marking the traffic on a distributed port or uplink port.
vSphere Networking Procedure 1 Navigate to a distributed switch and then navigate to a distributed port or an uplink port. n To navigate to the distributed ports of the switch, click Networks > Distributed Port Groups, double-click a distributed port group from the list, and click the Ports tab. n To navigate to the uplink ports of an uplink port group, click Networks > Uplink Port Groups, double-click an uplink port group from the list, and click the Ports tab. 2 Select a port from the list.
vSphere Networking Disable Traffic Filtering and Marking on a Distributed Port or Uplink Port Disable the traffic filtering and marking policy on a port to let traffic flow to a virtual machine or a physical adapter without filtering for security or marking for QoS. Prerequisites To override a policy on distributed port level, enable the port-level override option for this policy. See Configure Overriding Networking Policies on Port Level.
vSphere Networking MAC Traffic Qualifier By using the MAC traffic qualifier in a rule, you can define matching criteria for the Layer 2 (Data Link Layer) properties of packets such as MAC address, VLAN ID, and next level protocol that consumes the frame payload. Protocol Type The Protocol type attribute of the MAC traffic qualifier corresponds to the EtherType field in Ethernet frames. EtherType represents the type of next level protocol that is going to consume the payload of the frame.
vSphere Networking Destination Address By using the Destination Address group of attributes, you can match packets to their destination address. The MAC destination address options have the same format as those for the source address. Comparison Operators To match traffic in a MAC qualifier more closely to your needs, you can use affirmative comparison or negation. You can use operators such that all packets except the ones with certain attributes fall in the scope of a rule.
vSphere Networking Destination Address Use the Destination Address to match packets by IP address, subnet, or IP version. The destination address has the same format as the one for the source. Comparison Operators To match traffic in an IP qualifier more closely to your needs, you can use affirmative comparison or negation. You can define that all packets fall in the scope of a rule except packets with certain attributes.
vSphere Networking 5 (Optional) On the Security page, use the drop-down menus to edit the security exceptions and click Next. Option Description Promiscuous mode n Reject. Placing a guest adapter in promiscuous mode has no effect on which frames are received by the adapter. n Accept. Placing a guest adapter in promiscuous mode causes it to detect all frames passed on the vSphere Distributed Switch that are allowed under the VLAN policy for the port group that the adapter is connected to. n Reject.
vSphere Networking 8 (Optional) On the Teaming and failover page, use the drop-down menus to edit the settings and click Next. Option Description Load balancing IP-based teaming requires that the physical switch be configured with ether channel. For all other options, ether channel should be disabled. Select how to choose an uplink. Network failure detection Notify switches n Route based on the originating virtual port.
vSphere Networking 9 (Optional) On the Resource allocation page, use the Network resource pool drop-down menu to add or remove resource allocations and click Next. 10 (Optional) On the Monitoring page, use the drop-menu to enable or disable NetFlow and click Next. Option Description Disabled NetFlow is disabled on the distributed port group. Enabled NetFlow is enabled on the distributed port group. You can configure NetFlow settings at the vSphere Distributed Switch level.
vSphere Networking Option MAC qualifier Description Qualify the traffic for the rule by Layer 2 header. n Protocol type. Set the next level protocol (IPv4, IPv6, etc.) consuming the payload. This attribute corresponds to the EtherType field in Ethernet frames. You can select a protocol from the drop-down menu or type its hexadecimal number For example, to locate traffic for the Link Layer Discovery Protocol (LLDP) protocol, type 88CC. n VLAN ID. Locate traffic by VLAN.
vSphere Networking 12 (Optional) On the Miscellaneous page, select Yes or No from the drop-down menu and click Next. SelectYes to shut down all ports in the port group. This shutdown might disrupt the normal network operations of the hosts or virtual machines using the ports. 13 Review your settings on the Ready to complete page and click Finish. Use the Back button to change any settings. Port Blocking Policies Port blocking policies allow you to selectively block ports from sending or receiving data.
vSphere Networking 2 Select a port from the list. 3 Click Edit distributed port settings. 4 In the Miscellaneous section, select the Override check box, and from the drop-down menu enable or disable port blocking. 5 Click OK. VMware, Inc.
Isolating Network Traffic by Using VLANs 9 VLANs let you segment a network into multiple logical broadcast domains at Layer 2 of the network protocol stack. This chapter includes the following topics: n VLAN Configuration n Private VLANs VLAN Configuration Virtual LANs (VLANs) enable a single physical LAN segment to be further isolated so that groups of ports are isolated from one another as if they were on physically different segments.
vSphere Networking Tagging Mode VLAN ID on switch port groups EST 0 The physical switch performs the VLAN tagging. The host network adapters are connected to access ports on the physical switch. VST Between 1 and 4094 The virtual switch performs the VLAN tagging before the packets leave the host. The host network adapters must be connected to trunk ports on the physical switch.
vSphere Networking 4 To add a primary VLAN, under Primary VLAN ID click Add and enter the ID of a primary VLAN. 5 Click the plus sign (+) in front of the primary VLAN ID to add it to the list. The primary private VLAN also appears under Secondary Private VLAN ID. 6 To add a secondary VLAN, in the right pane click Add and enter the ID of the VLAN. 7 Click the plus sign (+) in front of the secondary VLAN ID to add it to the list.
vSphere Networking 2 On the Configure tab, expand Settings and select Private VLAN. 3 Click Edit. 4 Select a primary private VLAN. The secondary private VLANs associated with it appear on the right. 5 Select the secondary private VLAN to remove. 6 Under the secondary VLAN ID list, click Remove and click OK. VMware, Inc.
Managing Network Resources 10 vSphere provides several different methods to help you manage your network resources.
vSphere Networking n Hot adding and removing of virtual devices n Suspend and resume n High availability n DRS n Snapshots See Cisco VM-FEX documentation for details on supported switches and switch configuration information. n Enable Passthrough for a Network Device on a Host Passthrough devices provide the means to use resources efficiently and improve performance of your environment. You can enable DirectPath I/O passthrough for a network device on a host.
vSphere Networking 4 Select the network device to be used for passthrough and click OK. The selected PCI device appears in the table. Device information is displayed at the bottom of the screen. 5 Reboot the host to make the PCI network device available for use. Configure a PCI Device on a Virtual Machine Passthrough devices provide the means to more efficiently use resources and improve performance in your environment.
vSphere Networking Prerequisites Enable high-performance network I/O on at least one Cisco UCS port profile on a supported Cisco VMFEX distributed switch. For supported switches and switch configuration, see documentation at the CIsco websitehttp://www.cisco.com/go/unifiedcomputing/b-series-doc . Procedure 1 Locate the virtual machine in the vSphere Web Client. a Select a data center, folder, cluster, resource pool, or host and click the VMs tab.
vSphere Networking Using SR-IOV in vSphere In vSphere, a virtual machine can use an SR-IOV virtual function for networking. The virtual machine and the physical adapter exchange data directly without using the VMkernel as an intermediary. Bypassing the VMkernel for networking reduces latency and improves CPU efficiency. In vSphere 5.
vSphere Networking Table 10‑1. Supported Configurations for Using SR-IOV (Continued) Component Requirements Guest OS Must be supported by the NIC on the installed ESXi release according to the technical documentation from the NIC vendor. VF driver in the guest OS n Must be compatible with the NIC. n Must be supported on the guest OS release according to the technical documentation from the NIC vendor. n Must be Microsoft WLK or WHCK certified for Windows virtual machines.
vSphere Networking Supported NICs All NICs must have drivers and firmware that support SR-IOV. Some NICs might require SR-IOV to be enabled on the firmware.
vSphere Networking Figure 10‑1.
vSphere Networking 2 The VF forwards the request to the PF through a mailbox mechanism. 3 The PF driver checks the configuration request with the virtual switch (standard switch or host proxy switch of a distributed switch). 4 The virtual switch verifies the configuration request against the policy on the port with which the VF enabled virtual machine adapter is associated. 5 The PF driver configures the VF if the new settings are in compliance with the port policy of the virtual machine adapter.
vSphere Networking n If you have Intel and Emulex NICs present with SR-IOV enabled, the number of VFs available for the Intel NICs depends on how many VFs are configured for the Emulex NIC, and the reverse. You can use the following formula to estimate the maximum number of VFs for use if all 3072 interrupt vectors are available for passthrough: 3X + 2Y < 3072 where X is the number of Intel VFs, and Y is the number of Emulex VFs.
vSphere Networking 2 Assign a Virtual Function as SR-IOV Passthrough Adapter to a Virtual Machine To ensure that a virtual machine and a physical NIC can exchange data, you must associate the virtual machine with one or more virtual functions as SR-IOV passthrough network adapters. The traffic passes from an SR-IOV passthrough adapter to the physical adapter in compliance with the active policy on the associated port on the standard or distributed switch.
vSphere Networking Assign a Virtual Function as SR-IOV Passthrough Adapter to a Virtual Machine To ensure that a virtual machine and a physical NIC can exchange data, you must associate the virtual machine with one or more virtual functions as SR-IOV passthrough network adapters. Prerequisites n Verify that the virtual functions exist on the host. n Verify that the passthrough networking devices for the virtual functions are active in the PCI Devices list on the Settings tab for the host.
vSphere Networking When you power on the virtual machine, the ESXi host selects a free virtual function from the physical adapter and maps it to the SR-IOV passthrough adapter. The host validates all properties of the virtual machine adapter and the underlying virtual function against the settings of the port group to which the virtual machine belongs. Networking Options for the Traffic Related to an SR-IOV Enabled Virtual Machine In vSphere 5.
vSphere Networking SR-IOV Only Mode The physical adapter provides virtual functions to virtual machines connected to a virtual switch, but does not back traffic from non SR-IOV virtual machines on the switch. To verify whether the physical adapter is in SR-IOV only mode, examine the topology diagram of the switch. In this mode, the physical adapter is in a separate list called External SR-IOV Adapters and appears with the icon.
vSphere Networking 5 In the Value text box, type a comma-separated list of valid virtual function numbers. Each list entry indicates the number of virtual functions that you want to configure for each physical function. A value of 0 ensures that SR-IOV is not enabled for that physical function. For example, if you have a dual port, set the value to x,y where x or y is the number of virtual functions you want to enable for a single port.
vSphere Networking You can use a comma-separated list to set values for the vf_param parameter, where each entry indicates the number of virtual functions for a port. A value of 0 ensures that SR-IOV is not enabled for that physical function. If you have two dual port NICs, you can set the value to w,x,y,z, where w,x,y, and z is the number of virtual functions you want to enable for a single port.
vSphere Networking When a virtual machine powers on and the guest operating system VF driver starts, interrupt vectors are consumed. If the required number of interrupt vectors is not available, the guest operating system shuts down unexpectedly without any error messages. No rule presently exists to determine the number of interrupt vectors consumed or available on a host. This number depends on the hardware configuration of the host.
vSphere Networking Supported Configurations To use PVRDMA in vSphere 6.5, your environment must meet several configuration requirements. Table 10‑3. Supported Configurations for Using PVRDMA Component Requirements vSphere n ESXi host 6.5 or later. n vCenter Server or vCenter Server Appliance 6.5 or later. n vSphere Distributed Switch. Physical host n Must be compatible with the ESXi release. Host Channel Adapter (HCA) n Must be compatible with the ESXi release.
vSphere Networking Procedure 1 In the vSphere Web Client, navigate to the host. 2 On the Configure tab, expand System. 3 Click Advanced System Settings. 4 Locate Net.PVRDMAVmknic and click Edit. 5 Enter the value of the VMkernel adapter that you want to use, for example vmk0, and click OK . Enable the Firewall Rule for PVRDMA Enable the firewall rule for PVRDMA in the security profile of the ESXi host. Procedure 1 In the vSphere Web Client, navigate to the host.
vSphere Networking 5 From the New device drop-down menu, select Network and click Add. 6 Expand the New Network section and connect the virtual machine to a distributed port group. 7 From the Adapter type drop-down menu, select PVRDMA. 8 Expand the Memory section, select Reserve all guest memory (All locked), and click OK . 9 Power on the virtual machine.
vSphere Networking Like RoCE v1, RoCE v2 must run on a PFC priority-enabled VLAN. Note Do not team RoCE NICs, if you intend to use RDMA on those NICs. For vendor-specific configuration information, refer to the official documentation of the respective device or switch vendor. Jumbo Frames Jumbo frames let ESXi hosts send larger frames out onto the physical network. The network must support jumbo frames end-to-end that includes physical network adapters, physical switches, and storage devices.
vSphere Networking Enable Jumbo Frames for a VMkernel Adapter Jumbo frames reduce the CPU load caused by transferring data. Enable jumbo frames on a VMkernel adapter by changing the maximum transmission units (MTU) of the adapter. Procedure 1 In the vSphere Web Client, navigate to the host. 2 On the Configure tab, expand Networking and select VMkernel adapters. 3 Select a VMkernel adapter from the adapter table. The properties of the adapter appear. 4 Click the name of the VMkernel adapter.
vSphere Networking What to do next n Check that the enhanced VMXNET adapter is connected to a standard switch or to a distributed switch with jumbo frames enabled. n Inside the guest operating system, configure the network adapter to allow jumbo frames. See the documentation of your guest operating system. n Configure all physical switches and any physical or virtual machines to which this virtual machine connects to support jumbo frames.
vSphere Networking Procedure u Run these esxcli network nic software set console commands to enable or disable the software simulation of TSO in the VMkernel. n Enable the software simulation of TSO in the VMkernel. esxcli network nic software set --ipv4tso=1 -n vmnicX esxcli network nic software set --ipv6tso=1 -n vmnicX n Disable the software simulation of TSO in the VMkernel.
vSphere Networking 5 Click OK to apply the changes. 6 To reload the driver module of the physical adapter, run the esxcli system module set console command in the ESXi Shell on the host. a To disable the driver, run the esxcli system module set command with the --enabled false option. esxcli b system module set --enabled false --module nic_driver_module To enable the driver, run the esxcli system module set command with the --enabled true option.
vSphere Networking Procedure u In a terminal window on the Linux guest operating system, to enable or disable TSO, run the ethtool command with the -K and tso options. n To enable TSO, run the following command: ethtool -K ethY tso on n To disable TSO, run the following command: ethtool -K ethY tso off where Y in ethY is the sequence number of the NIC in the virtual machine.
vSphere Networking LRO reassembles incoming network packets into larger buffers and transfers the resulting larger but fewer packets to the network stack of the host or virtual machine. The CPU has to process fewer packets than when LRO is disabled, which reduces its utilization for networking especially in the case of connections that have high bandwidth. To benefit from the performance improvement of LRO, enable LRO along the data path on an ESXi host including VMkernel and guest operating system.
vSphere Networking 4 5 Edit the value of the Net.Vmxnet3SwLRO parameter for VMXNET3 adapters. n To enable software LRO, set Net.Vmxnet3SwLRO to 1. n To disable software LRO, set Net.Vmxnet3SwLRO to 0. Click OK to apply the changes. Determine Whether LRO Is Enabled for VMXNET3 Adapters on an ESXi Host Examine the status of LRO on an ESXi when you estimate the networking performance on a host that runs latency-sensitive workloads.
vSphere Networking Procedure 1 In the vSphere Web Client, navigate to the host. 2 On the Configure tab, expand System. 3 Click Advanced System Settings. 4 Edit the value of the Net.TcpipDefLROEnabled parameter. n To enable LRO for the VMkernel network adapters on the host, set Net.TcpipDefLROEnabled to 1. n 5 To disable software LRO for the VMkernel network adapters on the host, set Net.TcpipDefLROEnabled to 0. Click OK to apply the changes.
vSphere Networking Procedure u In a terminal window on the Linux guest operating system, run the ethtool command with the -K and lro options. n To enable LRO, run the following command: ethtool -K ethY lro on where Y in ethY is the sequence number of the NIC in the virtual machine. n To disable LRO, run the following command: ethtool -K ethY lro off where Y in ethY is the sequence number of the NIC in the virtual machine.
vSphere Networking Enable LRO Globally on a Windows Virtual Machine To use LRO on a VMXNET3 adapter on a virtual machine that runs Windows 8 and later or Windows Server 2012 and later, you must enable LRO globally on the guest operating system. On Windows, the LRO technology is also referred to as Receive Side Coalescing (RSC).
vSphere Networking Enable NetQueue on a Host NetQueue is enabled by default. To use NetQueue after it has been disabled, you must reenable it. Prerequisites Procedure 1 In the ESXi Shell on the host, use the following command: esxcli system settings kernel set --setting="netNetqueueEnabled" --value="TRUE" 2 Use the esxcli module parameters set command to configure the NIC driver to use NetQueue.
vSphere Network I/O Control 11 Use vSphere Network I/O Control to allocate network bandwidth to business-critical applications and to resolve situations where several types of traffic compete for common resources. n About vSphere Network I/O Control Version 3 vSphere Network I/O Control version 3 introduces a mechanism to reserve bandwidth for system traffic based on the capacity of the physical adapters on a host.
vSphere Networking About vSphere Network I/O Control Version 3 vSphere Network I/O Control version 3 introduces a mechanism to reserve bandwidth for system traffic based on the capacity of the physical adapters on a host. It enables fine-grained resource control at the VM network adapter level similar to the model that you use for allocating CPU and memory resources.. Version 3 of the Network I/O Control feature offers improved network resource reservation and allocation across the entire switch.
vSphere Networking Table 11‑1. Network I/O Control Version According to the Version of vSphere Distributed Switch and ESXi vSphere Network I/O Control vSphere Distributed Switch Version ESXi Version 2.0 5.1.0 n 5.1 n 5.5 n 6.0 n 5.5 n 6.0 5.5.0 3.0 6.0.0 6.0 Availability of Features SR-IOV is not available for virtual machines configured to use Network I/O Control version 3.
vSphere Networking Table 11‑2. Functionality Removed During the Upgrade to Network I/O Control Version 3 Functionality Removed During the Upgrade Description User-defined network resource pools including all associations between them and existing distributed port groups You can preserve certain resource allocation settings by transferring the shares from the user-defined network resource pools to shares for individual network adapters.
vSphere Networking 6 Prerequisite Description CoS priority tag for system traffic The distributed switch does not have network resource pools that have a CoS tag assigned. User-defined network resource pools The distributed switch does not contain user-defined resource pools for virtual machine bandwidth control. Resource allocation policy override No distributed port groups on the switch allow overriding the Network I/O Control policy on individual ports.
vSphere Networking When enabled, the model that Network I/O Control uses to handle bandwidth allocation for system traffic and virtual machine traffic is based on the Network I/O Control version that is active on the distributed switch. See About vSphere Network I/O Control Version 3. Bandwidth Allocation for System Traffic You can configure Network I/O Control to allocate certain amount of bandwidth for traffic generated by vSphere Fault Tolerance, iSCSI storage, vSphere vMotion, and so on.
vSphere Networking Table 11‑3. Allocation Parameters for System Traffic Parameter for Bandwidth Allocation Description Shares Shares, from 1 to 100, reflect the relative priority of a system traffic type against the other system traffic types that are active on the same physical adapter. The amount of bandwidth available to a system traffic type is determined by its relative shares and by the amount of data that the other system features are transmitting.
vSphere Networking You might leave more capacity unreserved to let the host allocate bandwidth dynamically according to shares, limits, and use, and to reserve only bandwidth that is enough for the operation of a system feature. Figure 11‑1.
vSphere Networking 3 Click System Traffic. You see the bandwidth allocation for the types of system traffic. 4 Select the traffic type according to the vSphere feature that you want to provision and click Edit. The network resource settings for the traffic type appear. 5 From the Shares drop-down menu, edit the share of the traffic in the overall flow through a physical adapter. Network I/O Control applies the configured shares when a physical adapter is saturated.
vSphere Networking The bandwidth quota that is dedicated to a network resource pool is shared among the distributed port groups associated with the pool. A virtual machine receives bandwidth from the pool through the distributed port group the VM is connected to. By default, distributed port groups on the switch are assigned to a network resource pool, called default, whose quota is not configured. Figure 11‑2.
vSphere Networking The total bandwidth reservation of the virtual machines on a host cannot exceed the reserved bandwidth that is configured for the virtual machine system traffic. The actual limit and reservation also depends on the traffic shaping policy for the distributed port group the adapter is connected to. For example, if a VM network adapter requires a limit of 200 Mbps and the average bandwidth configured in the traffic shaping policy is 100 Mbps, then the effective limit becomes 100Mbps.
vSphere Networking Admission Control for Virtual Machine Bandwidth To guarantee that sufficient bandwidth is available to a virtual machine, vSphere implements admission control at host and cluster levels based on bandwidth reservation and teaming policy. Bandwidth Admission Control in vSphere Distributed Switch When you power on a virtual machine, the Network I/O Control feature on a distributed switch verifies that these conditions are satisfied on the host.
vSphere Networking Bandwidth Admission Control in vSphere HA When a host fails or is isolated, vSphere HA powers on a virtual machine on another host in the cluster according to the bandwidth reservation and teaming policy. To use admission control in vSphere HA, perform the following tasks: n Allocate bandwidth for the virtual machine system traffic. n Configure the bandwidth requirements of a virtual machine that is connected to the distributed switch.
vSphere Networking 6 Enter a value for Reservation quota, in Mbps, from the free bandwidth that is reserved for the virtual machine system traffic.
vSphere Networking 3 In the Edit Settings dialog box, click General. 4 From the Network resource pool drop-down menu, select the network resource pool and click OK. If the distributed switch does not contain network resource pools, you see only the (default) option in the drop-down menu. Configure Bandwidth Allocation for a Virtual Machine You can configure bandwidth allocation to individual virtual machines that are connected to a distributed port group.
vSphere Networking 8 In the Reservation text box, reserve a minimum bandwidth that must be available to the VM network adapter when the virtual machine is powered on. If you provision bandwidth by using a network resource pool, the reservation from the network adapters of powered on VMs that are associated with the pool must not exceed the quota of the pool.
vSphere Networking 7 From the Shares drop-down menu, set the relative priority of traffic from these virtual machines in the scope of the physical adapters that carry the traffic. Network I/O Control applies the configured shares when a physical adapter is saturated. 8 In the Reservation text box, reserve a minimum bandwidth that must be available to each VM network adapter when the virtual machines are powered on.
vSphere Networking Procedure 1 Locate a distributed port group in the vSphere Web Client. a Select a distributed switch and click the Networks tab. b Click Distributed Port Groups. 2 Select the distributed port group and click Edit distributed port group settings. 3 In the Edit Settings dialog box for the port group, click General. 4 From the Network resource pool drop-down menu, select (default) and click OK.
vSphere Networking Procedure 1 In the vSphere Web Client, navigate to the host. 2 On the Configure tab, expand System and select Advanced System Settings . 3 Set the physical adapters that you need to function outside the scope of Network I/O Control as a comma-separated list to the Net.IOControlPnicOptOut parameter. For example: vmnic0,vmnic3 4 Click OK to apply the changes. Working with Network I/O Control Version 2 On a vSphere Distributed Switch 5.
vSphere Networking Bandwidth Allocation Parameters for Network Resource Pools in Network I/O Control Version 2 Parameter for Bandwidth Allocation Shares Description When a physical adapter is saturated, the virtual machines or VMkernel adapters that use the adapter receive bandwidth to the external network according to the shares configured on the network resource pool.
vSphere Networking 5 In the Limit text box. enter bandwidth limit, in Mbps, for the network resource pool with regard to the connected physical adapters on the host. By default, no limit on the traffic is applied. 6 From the Physical adapter shares drop-down menu, enter the shares from the physical adapter capacity that the virtual machines or the VMkernel adapters associated with the network resource pool have.
vSphere Networking 5 From the Physical adapter shares drop-down menu, enter the shares from the physical adapter capacity that the virtual machines or the VMkernel adapters associated with the network resource pool have. Network I/O Control applies the configured shares when the connected physical adapter becomes saturated. You can select an option to set a pre-defined value, or select Custom and enter a number from 1 to 100 to set another share.
MAC Address Management 12 MAC addresses are used in the Layer 2 (Data Link Layer) of the network protocol stack to transmit frames to a recipient. In vSphere, vCenter Server generates MAC addresses for virtual machine adapters and VMkernel adapters, or you can assign addresses manually. Each network adapter manufacturer is assigned a unique three-byte prefix called an Organizationally Unique Identifier (OUI), which it can use to generate unique MAC addresses.
vSphere Networking n Range-based allocation After the MAC address is generated, it does not change unless the virtual machine's MAC address conflicts with that of another registered virtual machine. The MAC address is saved in the configuration file of the virtual machine. Note If you use invalid prefix- or range-based allocation values, an error is logged in the vpxd.log file. vCenter Server does not allocate MAC addresses when provisioning a virtual machine.
vSphere Networking Prefix-based MAC address allocation overcomes the limits of the default VMware allocation to provide unique addresses in larger scale deployments. Introducing an LAA prefix leads to a very large MAC address space (2 to the power of 46) instead of an universally unique address OUI which can give only 16 million MAC addresses. Verify that the prefixes that you provide for different vCenter Server instances in the same network are unique.
vSphere Networking To switch from range- or prefixed-based allocation back to VMware OUI allocation, or between range- and prefixed-based allocation, edit the vpxd.cfg file manually. See Set or Change Allocation Type. Note You should use prefix-based MAC address allocation in vCenter Server 5.1 and ESXi 5.1 hosts, and later. If a vCenter Server 5.1 instance manages hosts running ESXi versions earlier than ESXi 5.1, use VMware OUI prefix-based MAC address allocation.
vSphere Networking Prerequisites Decide on an allocation type before changing the vpxd.cfg file. For information on allocation types, see MAC Address Assignment from vCenter Server Procedure 1 On the host machine of vCenter Server, navigate to the directory that contains the configuration file: n On a Windows Server operating system, the location of the directory is C:\ProgramData\VMware\CIS\cfg\vmware-vpx. n On the vCenter Server Appliance, the location of the directory is /etc/vmware-vpx.
vSphere Networking 4 Save the vpxd.cfg. 5 Restart the vCenter Server host. MAC Address Generation on ESXi Hosts An ESXi host generates the MAC address for a virtual machine adapter when the host is not connected to vCenter Server. Such addresses have a separate VMware OUI to avoid conflicts. The ESXi host generates the MAC address for a virtual machine adapter in one of the following cases: n The host is not connected to vCenter Server.
vSphere Networking By default, VMware uses the Organizationally Unique Identifier (OUI) 00:50:56 for manually generated addresses, but all unique manually generated addresses are supported. Note Make sure that no other non-VMware devices use addresses assigned to VMware components. For example, you might have physical servers in the same subnet, which use 11:11:11:11:11:11, 22:22:22:22:22:22 as static MAC addresses.
vSphere Networking 5 In the Virtual Hardware tab, expand the network adapter section. 6 Under MAC Address, select Manual from the drop-down menu. 7 Type the static MAC address, and click OK. 8 Power on the virtual machine. Assign a Static MAC Address in the Virtual Machine Configuration File To set a static MAC address for a virtual machine, you can edit the configuration file of the virtual machine by using the vSphere Web Client. Procedure 1 Locate the virtual machine in the vSphere Web Client.
Configuring vSphere for IPv6 13 Configure ESXi hosts and vCenter Server for operation in a pure IPv6 environment for larger address space and improved address assignment. IPv6 is designated by the Internet Engineering Task Force (IETF) as the successor to IPv4 providing the following benefits: n Increased address length. The increased address space resolves the problem of address exhaustion and eliminates the need for network address translation.
vSphere Networking Table 13‑1.
vSphere Networking IPv6 Connectivity of Virtual Machines Virtual machines can exchange data in the network over IPv6. vSphere supports both static and automatic assignment of IPv6 addresses for virtual machines. Configuring one or more IPv6 addresses is also possible when you customize the guest operating system of a virtual machine. FQDNs and IPv6 Addresses In vSphere, you should use fully qualified domain names (FQDNs) that are mapped to IPv6 addresses on the DNS server.
vSphere Networking n Verify that the hosts have ESXi 6.5 installed. See the vSphere Installation and Setup documentation. Procedure 1 In the Direct Console User Interface (DCUI), configure each ESXi host as a pure IPv6 node. a In the DCUI, press F2 and log in to the host. b From the Configure Management Network menu, select IPv6 Configuration and press Enter. c Assign an IPv6 address to the host.
vSphere Networking 2 Configure each ESXi host as a pure IPv6 node. a Open an SSH connection and log in to the ESXi host. b Run the following command: esxcli network ip interface ipv6 set -i vmk0 -e true c Assign an IPv6 address to the management network. Address Assignment Option Description Static address assignment 1 Open an SSH connection and log in to the ESXi host.
vSphere Networking 3 Disable IPv4 configuration for management network a Open an SSH connection and log in to the ESXi host. b Run the following command: esxcli network ip interface ipv4 set -i vmk0 --type=none 4 If vCenter Server uses an external database, configure the database as an IPv6 node. 5 Configure vCenter Server as a pure IPv6 node and restart it. 6 Disable IPv4 on the database server. 7 In the vSphere Web Client, add the hosts to the inventory.
vSphere Networking 2 On the Configure tab, expand Networking and select VMkernel adapters. 3 Select the VMkernel adapter on the target distributed or standard switch and click Edit. 4 In the Edit Settings dialog box, click IPv6 settings. 5 Configure the address assignment of the VMkernel adapter. IPv6 Address Option Description Obtain IPv6 address automatically through DHCP Receive an IPv6 address for the VMkernel adapter from a DHCPv6 server.
vSphere Networking 6 Edit the IPv6 settings. Option Description Obtain IPv6 settings automatically through DHCP Assigns IPv6 addresses to the appliance automatically from the network by using DHCP. Obtain IPv6 settings automatically through Router Advertisement Assigns IPv6 addresses to the appliance automatically from the network by using router advertisement. Static IPv6 addresses Uses static IPv6 addresses that you set up manually. 1 Click the Add icon.
Monitoring Network Connection and Traffic 14 Monitor network connection and packets that pass through the ports of a vSphere Standard Switch or a vSphere Distributed Switch to analyze the traffic between virtual machines and hosts.
vSphere Networking pktcap-uw Syntax for Capturing Packets The pktcap-uw command has the following syntax for capturing packets at a certain place in the network stack: pktcap-uw switch_port_arguments capture_point_options filter_options output_control_options Note Certain options of the pktcap-uw utility are designed for VMware internal use only and you should use them only under the supervision of VMware Technical Support. These options are not described in the vSphere Networking guide. Table 14‑1.
vSphere Networking Table 14‑1. pktcap-uw Arguments for Capturing Packets (Continued) Argument Group capture_point_options Argument Description --lifID lif_ID Capture packets that are related to the logical interface of a distributed router. See the VMware NSX documentation. --capture capture_point Capture packets at a particular place in the network stack. For example, you can monitor packets right after they arrive from a physical adapter.
vSphere Networking Table 14‑1. pktcap-uw Arguments for Capturing Packets (Continued) Argument Group Argument Description filter_options Filter captured packets according to source or destination address, VLAN ID, VXLAN ID, Layer 3 protocol, and TCP port. See pktcap-uw Options for Filtering Packets. output_control_options Save the contents of a packet to a file, capture only a number of packets, and capture a number of bytes at the beginning of packets, and so on.
vSphere Networking Table 14‑3. Options for Output Control That Are Supported by the pktcap-uw Utility Option Description {-o | --outfile} pcap_file Save captured or traced packets to a file in packet capture (PCAP) format. Use this option to examine packets in a visual analyzer tool such as Wireshark. -P | --ng Save packet content in the PCAPNG file format. Use this option together with the -o or --outfile option. --console Print packet details and content to the console output.
vSphere Networking Table 14‑4. Filter Options of the pktcap-uw Utility (Continued) Option Description --ethtype 0xEthertype Capture or trace packets at Layer 2 according to the next level protocol that consumes packet payload. EtherType corresponds to the EtherType field in Ethernet frames . It represents the type of next level protocol that consumes the payload of the frame. For example, to monitor traffic for the Link Layer Discovery Protocol (LLDP) protocol, type --ethtype 0x88CC.
vSphere Networking Procedure 1 (Optional) Find the name of the physical adapter that you want to monitor in the host adapter list. n In the vSphere Web Client, on the Configure tab for the host, expand Networking and select Physical adapters. n In the ESXi Shell to the host, to view a list of the physical adapters and examine their state, run the following ESXCLI command: esxcli network nic list Each physical adapter is represented as vmnicX.
vSphere Networking c Use options to save the contents of each packet or the contents of a limited number of packets to a .pcap or .pcapng file. n To save packets to a .pcap file, use the --outfile option. n To save packets to a .pcapng file, use the --ng and --outfile options. You can open the file in a network analyzer tool such as Wireshark. By default, the pktcap-uw utility saves the packet files to the root folder of the ESXi file system.
vSphere Networking c In the USED-BY column, locate the virtual machine adapter, and write down the PORT-ID value for it. The USED-BY field contains the name of the virtual machine and the port to which the virtual machine adapter is connected. d 2 Press Q to exit esxtop. In the ESXi Shell, run pktcap-uw --switchport port_ID. port_ID is the ID that the esxtop utility displays for the virtual machine adapter in the PORT-ID column.
vSphere Networking c Use options to save the contents of each packet or the contents of a limited number of packets to a .pcap or .pcapng file. n To save packets to a .pcap file, use the --outfile option. n To save packets to a .pcapng file, use the --ng and --outfile options. You can open the file in a network analyzer tool such as Wireshark. By default, the pktcap-uw utility saves the packet files to the root folder of the ESXi file system.
vSphere Networking Each VMkernel adapter is represented as vmkX, where X is the sequence number that ESXi assigned to the adapter. 2 In the ESXi Shell to the host, run the pktcap-uw command with the --vmk vmkX argument and with options to monitor packets at a particular point, filter captured packets and save the result to a file.
vSphere Networking Capture Dropped Packets Troubleshoot lost connectivity by capturing dropped packets through the pktcap-uw utility. A packet might be dropped at a point in the network stream for many reasons, for example, a firewall rule, filtering in an IOChain and DVfilter, VLAN mismatch, physical adapter malfunction, checksum failure, and so on. You can use the pktcap-uw utility to examine where packets are dropped and the reason for the drop.
vSphere Networking What to do next If the contents of the packet are saved to a file, copy the file from the ESXi host to the system that runs a graphical analyzer tool, such as Wireshark, and open it in the tool to examine the packet details. Capture Packets at DVFilter Level Examine how packets change when they pass through a vSphere Network Appliance (DVFilter). DVFilters are agents that reside in the stream between a virtual machine adapter and a virtual switch.
vSphere Networking 3 If you have not limited the number of packets by using the --count option, press Ctrl+C to stop capturing or tracing packets. What to do next If the contents of the packet are saved to a file, copy the file from the ESXi host to the system that runs a graphical analyzer tool, such as Wireshark, and open it in the tool to examine the packet details.
vSphere Networking Capture Point Description PortInput The function that passes a list of packets from UplinkRcv to a port on the virtual switch. PortOutput The function that passes a list of packets from a port on the virtual switch to the UplinkSnd point.
vSphere Networking Capture Point Description PktFree Capture packets right before they are released. VdrRxLeaf Capture packets at the receive leaf I/O chain of a dynamic router in VMware NSX. Use this capture point together with the --lifID option. VdrRxTerminal Capture packets at the receive terminal I/O chain of a dynamic router in VMware NSX. Use this capture point together with the --lifID option. VdrTxLeaf Capture packets at the transmit leaf I/O chain of a dynamic router in VMware NSX.
vSphere Networking Procedure 1 In the ESXi Shell to the host, run the pktcap-uw --trace command with options to filter traced packets, save the result to a file and limit the number of traced packets. pktcap-uw --trace [filter_options] [--outfile pcap_file_path [--ng]] [--count number_of_packets] where the square brackets [] enclose optional items of the pktcap-uw --trace command and the vertical bars | represent alternative values.
vSphere Networking 3 Type the Collector IP address and Collector port of the NetFlow collector. You can contact the NetFlow collector by IPv4 or IPv6 address. 4 Set an Observation Domain ID that identifies the information related to the switch. 5 To see the information from the distributed switch in the NetFlow collector under a single network device instead of under a separate device for each host on the switch, type an IPv4 address in the Switch IP address text box.
vSphere Networking Table 14‑5. Port mirroring compatibility vCenter Server version vSphere distributed switch version Host version vSphere 5.1 port mirroring functionality vSphere 5.1 and later vSphere 5.1 and later vSphere 5.1 and later vSphere 5.1 port mirroring is available for use. Features for vSphere 5.0 and earlier port mirroring are not available. vSphere 5.1 and later vSphere 5.1 and later vSphere 5.0 and earlier vSphere 5.0 and earlier hosts can be added to vSphere 5.
vSphere Networking Table 14‑6. vMotion Interoperability with port mirroring Port mirroring session type Source and destination Interoperable with vMotion Functionality Distributed Port Mirroring Non-uplink distributed port source and destination Yes Port mirroring between distributed ports can only be local. If the source and destination are on different hosts due to vMotion, mirroring between them will not work. However, if the source and destination move to the same host, port mirroring works.
vSphere Networking TSO and LRO TCP Segmentation Offload (TSO) and large receive offload (LRO) might cause the number of mirroring packets to not equal to the number of mirrored packets. When TSO is enabled on a vNIC, the vNIC might send a large packet to a distributed switch. When LRO is enabled on a vNIC, small packets sent to it might be merged into a large packet.
vSphere Networking 2 Click the Configure tab and expand Settings. 3 Select the Port mirroring option and click New. 4 Select the session type for the port mirroring session. 5 Option Description Distributed Port Mirroring Mirror packets from a number of distributed ports to other distributed ports on the same host . If the source and the destination are on different hosts, this session type does not function.
vSphere Networking Select Port Mirroring Sources To continue creating a port mirroring session, select sources and traffic direction for the new port mirroring session. You can create a port mirroring session without setting the source and destinations. When a source and destination are not set, a port mirroring session is created without the mirroring path. This allows you to create a port mirroring session with the correct properties set.
vSphere Networking Procedure 1 Select the destination for the port mirroring session. Depending on which type of session you chose, different options are available. Option Description Select a destination distributed port Click Select distributed ports to select ports from a list, or click Add distributed ports to add ports by port number. You can add more than one distributed port.
vSphere Networking Procedure 1 In the vSphere Web Client, navigate to the distributed switch. 2 On the Configure tab, expand Settings and click Port mirroring. 3 Select a port mirroring session from the list and click Edit. 4 On the Properties page, edit the session properties. Depending on the type of port mirroring session being edited, different options are available for configuration.
vSphere Networking 6 In the Destinations section, edit the destinations for the port mirroring session. Depending on the type of port mirroring session being edited, different options are available for configuration. Option Description Select a destination distributed port Click the Select distributed ports… button to select ports from a list, or click the Add distributed ports… button to add ports by port number. You can add more than one distributed port.
vSphere Networking Required Configuration on the Distributed Switch Configuration Error Health Check The VLAN trunk ranges configured on the distributed switch do not match the trunk ranges on the physical switch. Checks whether the VLAN settings on the distributed switch match the trunk port configuration on the connected physical switch ports. At least two active physical NICs The MTU settings on the physical network adapters, distributed switch, and physical switch ports do not match.
vSphere Networking Prerequisites Verify that health check for VLAN and MTU, and for teaming policy is enabled on the vSphere Distributed Switch. See Enable or Disable vSphere Distributed Switch Health Check. Procedure 1 In the vSphere Web Client, navigate to the distributed switch. 2 On the Monitor tab, click Health. 3 In the Health Status Details section, examine the overall, VLAN, MTU and teaming health of the hosts connected to the switch.
vSphere Networking 5 6 From the Operation drop-down menu, select the operational mode of the ESXi hosts connected to the switch. Option Description Listen ESXi detects and displays information about the associated Cisco switch port, but information about the vSphere Distributed Switch is not available to the Cisco switch administrator.
vSphere Networking View Switch Information When Cisco Discovery Protocol (CDP) or Link Layer Discovery Protocol (LLDP) is enabled on the distributed switch and the hosts connected to the switch are in Listen or Both operational mode, you can view physical switch information from the vSphere Web Client. Procedure 1 In the vSphere Web Client, navigate to the host. 2 On the Configure tab, expand Networking and click Physical adapters.
Configuring Protocol Profiles for Virtual Machine Networking 15 A network protocol profile contains a pool of IPv4 and IPv6 addresses that vCenter Server assigns to vApps or to virtual machines with vApp functionality that are connected to port groups associated with the profile. Network protocol profiles also contain settings for the IP subnet, DNS, and HTTP proxy server.
vSphere Networking Add a Network Protocol Profile A network protocol profile contains a pool of IPv4 and IPv6 addresses. vCenter Server assigns those resources to vApps or to virtual machines with vApp functionality that are connected to port groups associated with the profile. Network protocol profiles also contain settings for the IP subnet, DNS, and HTTP proxy server.
vSphere Networking 4 Select the Enable IP Pool check box to specify an IP pool range. 5 If you enable IP Pools, enter a comma-separated list of host address ranges in the IP pool range field. A range consists of an IP address, a pound sign (#), and a number indicating the length of the range. The gateway and the ranges must be within the subnet. The ranges that you enter in the IP pool range field cannot include the gateway address. For example, 10.20.60.4#10, 10.20.61.
vSphere Networking Specify Network Protocol Profile DNS and Other Configuration When you create a network protocol profile, you can specify the DNS domain, DNS search path, a host prefix, and HTTP proxy. Procedure 1 Enter the DNS domain. 2 Enter the host prefix. 3 Enter the DNS search path. The search paths are specified as a list of DNS domains separated by commas, semi-colons, or spaces. 4 Enter the server name and port number for the proxy server.
vSphere Networking 4 On the Set association type page of the Associate Network Protocol Profile wizard, select Use an existing network protocol profile and click Next. If the existing network protocol profiles do not contain settings suitable for the vApp virtual machines in the port group, you must create a new profile. 5 Select the network protocol profile and click Next. 6 Examine the association and settings of the network protocol profile, and click Finish.
Multicast Filtering 16 In vSphere 6.0 and later, vSphere Distributed Switch supports basic and snooping models for filtering of multicast packets that are related to individual multicast groups. Choose a model according to the number of multicast groups to which the virtual machines on the switch subscribe. n Multicast Filtering Modes In addition to the default basic mode for filtering multicast traffic, vSphere Distributed Switch 6.0.
vSphere Networking Basic Multicast Filtering In basic multicast filtering mode, a vSphere Standard Switch or vSphere Distributed Switch forwards multicast traffic for virtual machines according to the destination MAC address of the multicast group. When joining a multicast group, the guest operating system pushes the multicast MAC address of the group down to the network through the switch. The switch saves the mapping between the port and the destination multicast MAC address in a local forwarding table.
vSphere Networking Use multicast snooping if virtualized workloads on the switch subscribe to more than 32 multicast groups or must receive traffic from specific source nodes. For information about the multicast filtering modes of vSphere Distributed Switch, see Multicast Filtering Modes. Prerequisites Verify that vSphere Distributed Switch is version 6.0.0 and later. Procedure 1 In the vSphere Web Client, navigate to the distributed switch. 2 From the Actions menu, select Settings > Edit Settings.
vSphere Networking 4 Click Edit and enter a new value between 1 and 32 for the setting. 5 Click OK. VMware, Inc.
Stateless Network Deployment 17 Stateless is a mode of execution for ESXi hosts with no local storage that formerly would save configuration or state. Configurations are abstracted into a host profile, which is a template that applies to a class of machines. Stateless allows easy replacement, removal, and addition of failed hardware, and improves the ease of scaling a hardware deployment. Every stateless ESXi boot is like a first boot.
vSphere Networking n Creates all standard switch instances, along with port groups. It selects uplinks based on policy. If the policy is based on the VLAN ID, there is a probing process to gather relevant information. n For VMkernel network adapters connected to the standard switch, it creates VMkernel network adapters and connects them to port groups.
Networking Best Practices 18 Consider these best practices when you configure your network. n To ensure a stable connection between vCenter Server, ESXi, and other products and services, do not set connection limits and timeouts between the products. Setting limits and timeouts can affect the packet flow and cause services interruption. n Isolate from one another the networks for host management, vSphere vMotion, vSphere FT, and so on, to improve security and performance.
vSphere Networking n Physical network adapters connected to the same vSphere Standard Switch or vSphere Distributed Switch should also be connected to the same physical network. n Configure the same MTU on all VMkernel network adapters in a vSphere Distributed Switch. If several VMkernel network adapters, configured with different MTUs, are connected to vSphere distributed switches, you might experience network connectivity problems. VMware, Inc.
