vSphere Networking 17 APR 2018 VMware vSphere 6.7 VMware ESXi 6.7 vCenter Server 6.
vSphere Networking You can find the most up-to-date technical documentation on the VMware website at: https://docs.vmware.com/ If you have comments about this documentation, submit your feedback to docfeedback@vmware.com VMware, Inc. 3401 Hillview Ave. Palo Alto, CA 94304 www.vmware.com Copyright © 2018 VMware, Inc. All rights reserved. Copyright and trademark information. VMware, Inc.
Contents About vSphere Networking 11 1 Introduction to vSphere Networking 12 Networking Concepts Overview Network Services in ESXi 12 14 VMware ESXi Dump Collector Support 14 2 Setting Up Networking with vSphere Standard Switches 16 vSphere Standard Switches 16 Create a vSphere Standard Switch 18 Port Group Configuration for Virtual Machines Add a Virtual Machine Port Group 19 20 Edit a Standard Switch Port Group 21 Remove a Port Group from a vSphere Standard Switch vSphere Standard Switch P
vSphere Networking Assign a Physical NIC of a Host to a vSphere Distributed Switch Remove a Physical NIC from a vSphere Distributed Switch Removing NICs from Active Virtual Machines Distributed Port Groups 47 47 47 48 Add a Distributed Port Group 48 Edit General Distributed Port Group Settings 51 Configure Overriding Networking Policies on Port Level Remove a Distributed Port Group Working with Distributed Ports 52 53 53 Monitor the State of Distributed Ports Configure Distributed Port Setting
vSphere Networking 6 Backing Up and Restoring Networking Configurations 82 Backing Up and Restoring a vSphere Distributed Switch Configuration Export vSphere Distributed Switch Configurations 82 Import a vSphere Distributed Switch Configuration 83 Restore a vSphere Distributed Switch Configuration 82 84 Export, Import, and Restore vSphere Distributed Port Group Configurations Export vSphere Distributed Port Group Configurations 84 Import a vSphere Distributed Port Group Configuration 85 Restore
vSphere Networking Qualifying Traffic for Filtering and Marking 130 Manage Policies for Multiple Port Groups on a vSphere Distributed Switch Port Blocking Policies 133 138 Edit the Port Blocking Policy for a Distributed Port Group 138 Edit the Blocking Policy for a Distributed Port or Uplink Port 138 9 Isolating Network Traffic by Using VLANs 140 VLAN Configuration Private VLANs 140 141 Create a Private VLAN 141 Remove a Primary Private VLAN 142 Remove a Secondary Private VLAN 142 10 Mana
vSphere Networking Determine Whether TSO Is Enabled on an ESXi Host Enable or Disable TSO on a Linux Virtual Machine 166 166 Enable or Disable TSO on a Windows Virtual Machine Large Receive Offload 167 167 Enable Hardware LRO for All VMXNET3 Adapters on an ESXi Host 168 Enable or Disable Software LRO for All VMXNET3 Adapters on an ESXi Host 168 Determine Whether LRO Is Enabled for VMXNET3 Adapters on an ESXi Host 169 Change the Size of the LRO Buffer for VMXNET 3 Adapters 169 Enable or Disabl
vSphere Networking Assigning a MAC Address 191 MAC Address Generation on ESXi Hosts 193 Setting a Static MAC Address to a Virtual Machine VMware OUI in Static MAC Addresses 194 194 Assign a Static MAC Address by Using the vSphere Web Client 195 Assign a Static MAC Address in the Virtual Machine Configuration File 195 13 Configuring vSphere for IPv6 197 vSphere IPv6 Connectivity 197 Deploying vSphere on IPv6 199 Enable IPv6 on a vSphere Installation 199 Enable IPv6 on an Upgraded vSphere En
vSphere Networking 15 Configuring Protocol Profiles for Virtual Machine Networking 236 Add a Network Protocol Profile 237 Select the Network Protocol Profile Name and Network 237 Specify Network Protocol Profile IPv4 Configuration 237 Specify Network Protocol Profile IPv6 Configuration 238 Specify Network Protocol Profile DNS and Other Configuration Complete the Network Protocol Profile Creation 239 239 Associate a Port Group with a Network Protocol Profile 239 Configure a Virtual Machine or v
vSphere Networking Virtual Machines on the Same Distributed Port Group and on Different Hosts Cannot Communicate with Each Other 268 Attempt to Power On a Migrated vApp Fails Because the Associated Protocol Profile Is Missing 269 Networking Configuration Operation Is Rolled Back and a Host Is Disconnected from vCenter Server VMware, Inc.
About vSphere Networking ® vSphere Networking provides information about configuring networking for VMware vSphere , including how to create vSphere distributed switches and vSphere standard switches. vSphere Networking also provides information on monitoring networks, managing network resources, and networking best practices.
Introduction to vSphere Networking 1 Get to know the basic concepts of vSphere networking and how to set up and configure a network in a vSphere environment. This chapter includes the following topics: n Networking Concepts Overview n Network Services in ESXi n VMware ESXi Dump Collector Support Networking Concepts Overview A few concepts are essential for a thorough understanding of virtual networking. If you are new to vSphere, it is helpful to review these concepts.
vSphere Networking machine connected to it. The switch learns which hosts are connected to which of its ports and uses that information to forward traffic to the correct physical machines. Switches are the core of a physical network. Multiple switches can be connected together to form larger networks. vSphere Standard Switch It works much like a physical Ethernet switch.
vSphere Networking VLAN VLAN enable a single physical LAN segment to be further segmented so that groups of ports are isolated from one another as if they were on physically different segments. The standard is 802.1Q. VMkernel TCP/IP Networking Layer The VMkernel networking layer provides connectivity to hosts and handles the standard infrastructure traffic of vSphere vMotion, IP storage, Fault Tolerance, and vSAN.
vSphere Networking There is no authentication or encryption in the file transfer session from a crashed host to the ESXi Dump Collector. You should configure the ESXi Dump Collector on a separate VLAN when possible to isolate the ESXi core dump from regular network traffic. For information about installing and configuring the ESXi Dump Collector, see the vCenter Server Installation and Setup documentation. VMware, Inc.
Setting Up Networking with vSphere Standard Switches 2 vSphere standard switches handle network traffic at the host level in a vSphere deployment. This chapter includes the following topics: n vSphere Standard Switches n Create a vSphere Standard Switch n Port Group Configuration for Virtual Machines n vSphere Standard Switch Properties vSphere Standard Switches You can create abstracted network devices called vSphere Standard Switches.
vSphere Networking Figure 2‑1.
vSphere Networking Number of Standard Ports To ensure efficient use of host resources on ESXi hosts, the number of ports of standard switches are dynamically scaled up and down. A standard switch on such a host can expand up to the maximum number of ports supported on the host. Create a vSphere Standard Switch Create a vSphere Standard Switch to provide network connectivity for hosts, virtual machines, and to handle VMkernel traffic.
vSphere Networking 7 If you create the new standard switch with a VMkernel adapter or virtual machine port group, enter connection settings for the adapter or the port group. Option Description VMkernel adapter a Enter a label that indicates the traffic type for the VMkernel adapter, for example vMotion. b Set a VLAN ID to identify the VLAN that the network traffic of the VMkernel adapter will use. c Select IPv4, Ipv6 or both. d Select a TCP/IP stack.
vSphere Networking Virtual machines reach physical networks through uplink adapters. A vSphere Standard Switch can transfer data to external networks only when one or more network adapters are attached to it. When two or more adapters are attached to a single standard switch, they are transparently teamed. Add a Virtual Machine Port Group Create port groups on a vSphere Standard Switch to provide connectivity and common network configuration for virtual machines.
vSphere Networking 7 On the Connection settings page, identify traffic through the ports of the group. a Type a Network label for the port group, or accept the generated label. b Set the VLAN ID to configure VLAN handling in the port group. The VLAN ID also reflects the VLAN tagging mode in the port group. c 8 VLAN Tagging Mode VLAN ID Description External Switch Tagging (EST) 0 The virtual switch does not pass traffic associated with a VLAN.
vSphere Networking 10 On the Teaming and failover page, override the teaming and failover settings inherited from the standard switch. You can configure traffic distribution and rerouting between the physical adapters associated with the port group. You can also change the order in which host physical adapters are used upon failure. 11 Click OK.
vSphere Networking Procedure 1 In the vSphere Web Client, navigate to the host. 2 On the Configure tab, expand Networking and select Virtual switches. 3 Select a standard switch from the table and click Edit settings. 4 Change the MTU (Bytes) value for the standard switch. You can enable jumbo frames by setting an MTU value greater than 1500. You cannot set an MTU size greater than 9000 bytes. 5 Click OK.
vSphere Networking 4 Click the Manage the physical network adapters connected to the selected switch icon. 5 Add one or more available physical network adapters to the switch. a Click Add adapters. b Select the failover order group to assign the adapters to. The failover group determines the role of the adapter for exchanging data with the external network, that is, active, standby or unused. By default, the adapters are added as active to the standard switch.
vSphere Networking Figure 2‑2. Topology Diagram of a Standard Switch That Connects the VMkernel and Virtual Machines to the Network VMware, Inc.
Setting Up Networking with vSphere Distributed Switches 3 With vSphere distributed switches you can set up and configure networking in a vSphere environment.
vSphere Networking Figure 3‑1.
vSphere Networking The vSphere Distributed Switch introduces two abstractions that you use to create consistent networking configuration for physical NICs, virtual machines, and VMkernel services. Uplink port group An uplink port group or dvuplink port group is defined during the creation of the distributed switch and can have one or more uplinks. An uplink is a template that you use to configure physical connections of hosts as well as failover and load balancing policies.
vSphere Networking vSphere Distributed Switch Data Flow The data flow from the virtual machines and VMkernel adapters down to the physical network depends on the NIC teaming and load balancing policies that are set to the distributed port groups. The data flow also depends on the port allocation on the distributed switch. Figure 3‑2.
vSphere Networking Figure 3‑3. Packet Flow on the Host Proxy Switch Host 1 VM1 VM2 vmknic1 VMkernel network VM network 0 1 3 Host Proxy Switch 5 6 7 Uplink port group vmnic0 vmnic1 vmnic2 Physical Switch On the host side, the packet flow from virtual machines and VMkernel services passes through particular ports to reach the physical network. For example, a packet sent from VM1 on Host 1 first reaches port 0 on the VM network distributed port group.
vSphere Networking 4 5 On the Select version page, select a distributed switch version and click Next. Option Description Distributed Switch: 6.6.0 Compatible with ESXi 6.7 and later. Distributed Switch: 6.5.0 Compatible with ESXi 6.5 and later. Features released with later vSphere distributed switch versions are not supported. Distributed Switch: 6.0.0 Compatible with ESXi 6.0 and later. Features released with later vSphere distributed switch versions are not supported.
vSphere Networking The upgrade of a distributed switch causes the hosts and virtual machines attached to the switch to experience a brief downtime. For more information, see KB 52621. Note To be able to restore the connectivity of the virtual machines and VMkernel adapters if the upgrade fails, back up the configuration of the distributed switch. If the upgrade is not successful, to recreate the switch with its port groups and connected hosts, you can import the switch configuration file.
vSphere Networking Procedure 1 In the vSphere Web Client, navigate to the distributed switch. 2 On the Configure tab, expand Settings and select Properties. 3 Click Edit. 4 Click General to edit the vSphere Distributed Switch settings. Option Description Name Type the name for the distributed switch. Number of uplinks Select the number of uplink ports for the distributed switch. Click Edit uplink names to change the names of the uplinks.
vSphere Networking n Tasks for Managing Host Networking on a vSphere Distributed Switch You can add new hosts to a vSphere Distributed Switch, connect network adapters to the switch, and remove hosts from the switch. In a production environment, you might need to keep the network connectivity up for virtual machines and VMkernel services while you manage host networking on the distributed switch.
vSphere Networking Adding Hosts to a vSphere Distributed Switch Consider preparing your environment before you add new hosts to a distributed switch. n Create distributed port groups for virtual machine networking. n Create distributed port groups for VMkernel services. For example, create distributed port groups for management network, vMotion, and Fault Tolerance. n Configure enough uplinks on the distributed switch for all physical NICs that you want to connect to the switch.
vSphere Networking n To migrate host networking to standard switches, you must migrate the network adapters in stages. For example, remove physical NICs on the hosts from the distributed switch by leaving one physical NIC on every host connected to the switch to keep the network connectivity up. Next, attach the physical NICs to the standard switches and migrate VMkernel adapters and virtual machine network adapters to the switches.
vSphere Networking For consistent network configuration, you can connect one and the same physical NIC on every host to the same uplink on the distributed switch. For example, if you are adding two hosts connect vmnic1 on of each host to Uplink1 on the distributed switch. 7 Click Next. 8 On the Manage VMkernel network adapters page, configure VMkernel adapters. 9 a Select a VMkernel adapter and click Assign port group. b Select a distributed port group and click OK.
vSphere Networking For consistent networking configuration throughout all hosts, you can assign the same physical NIC on every host to the same uplink on the distributed switch. For example, you can assign vmnic1 from hosts ESXi A and ESXi B to Uplink 1. Procedure 1 In the vSphere Web Client, navigate to the distributed switch. 2 From the Actions menu, select Add and Manage Hosts. 3 In Select task, select Manage host networking and click Next.
vSphere Networking Procedure 1 In the vSphere Web Client, navigate to the distributed switch. 2 From the Actions menu, select Add and Manage Hosts. 3 In Select task, select Manage host networking and click Next. 4 In Select hosts, click Attached hosts and select from the hosts that are associated with the distributed switch. 5 Click Next. 6 In Select network adapter tasks, select Manage VMkernel adapters and click Next.
vSphere Networking 4 In Select hosts, click Attached hosts and select from the hosts that are associated with the distributed switch. 5 Click Next. 6 In Select network adapter tasks, select Manage VMkernel adapters and click Next. 7 Click New adapter. The Add Networking wizard opens. 8 In Select target device, select a distributed port group, and click Next. 9 On the Port properties page, configure the settings for the VMkernel adapter.
vSphere Networking 10 If you selected the vMotion TCP/IP or the Provisioning stack, click OK in the warning dialog that appears. If a live migration is already initiated, it completes successfully even after the involved VMkernel adapters on the default TCP/IP stack are disabled for vMotion. Same refers to operations that include VMkernel adapters on the default TCP/IP stack that are set for the Provisioning traffic. 11 (Optional) On the IPv4 settings page, select an option for obtaining IP addresses.
vSphere Networking Procedure 1 In the vSphere Web Client, navigate to the distributed switch. 2 From the Actions menu, select Add and Manage Hosts. 3 In Select task, select Manage host networking and click Next. 4 In Select hosts, click Attached hosts and select from the hosts that are associated with the distributed switch. 5 Click Next. 6 In Select network adapter tasks, select Migrate virtual machine networking and click Next.
vSphere Networking Example: Configure Physical and VMkernel Adapters by Using a Template Host Use the template host mode in the Add and Manage Hosts wizard to create a uniform networking configuration among all the hosts on a distributed switch. On the Manage physical network adapters page of the wizard, assign a physical NIC to an uplink on the template host and then click Apply to all to create the same configuration on the other host. Figure 3‑4.
vSphere Networking Figure 3‑5. Applying VMkernel Adapter Configuration on a vSphere Distributed Switch by Using a Template Host Remove Hosts from a vSphere Distributed Switch Remove hosts from a vSphere distributed switch if you have configured a different switch for the hosts. Prerequisites n Verify that physical NICs on the target hosts are migrated to a different switch. n Verify that VMkernel adapters on the hosts are migrated to a different switch.
vSphere Networking Managing Networking on Host Proxy Switches You can change the configuration of the proxy switch on every host that is associated with a vSphere distributed switch. You can manage physical NICs, VMkernel adapters, and virtual machine network adapters. For details about setting up VMkernel networking on host proxy switches, see Create a VMkernel Adapter on a vSphere Distributed Switch.
vSphere Networking 7 Review the services that are affected from the new networking configuration. a If there is an important or serious impact reported on a service, click the service and review the analysis details. For example, an important impact on iSCSI might be reported as a result from an incorrect teaming and failover configuration on the distributed port group where you migrate the iSCSI VMkernel adapter.
vSphere Networking Assign a Physical NIC of a Host to a vSphere Distributed Switch You can assign physical NICs of a host that is associated with a distributed switch to uplink port on the host proxy switch. Procedure 1 In the vSphere Web Client, navigate to the host. 2 On the Configure tab, expand Networking and select Virtual switches. 3 Select a distributed switch from the list. 4 Click the Manage the physical network adapters connected to the selected switch icon.
vSphere Networking Removing NICs from an Active Virtual Machine with a Guest Operating System Installed You can remove a NIC from an active virtual machine, but it might not be reported to the vSphere Web Client for some time. If you click Edit Settings for the virtual machine, you might see the removed NIC listed even after the task is complete. The Edit Settings dialog box for the virtual machine does not immediately display the removed NIC.
vSphere Networking Setting Description Network resource pool Use the drop-down menu to assign the new distributed port group to a userdefined network resource pool. If you have not created a network resource pool, this menu is empty. VLAN Advanced 5 Use the VLAN type drop-down menu to select VLAN options: n None: Do not use VLAN. n VLAN: In the VLAN ID text box, enter a number between 1 and 4094. n VLAN trunking: Enter a VLAN trunk range. n Private VLAN: Select a private VLAN entry.
vSphere Networking 7 Setting Description Peak bandwidth The maximum number of bits per second to allow across a port when it is sending and receiving a burst of traffic. This tops the bandwidth used by a port whenever it is using its burst bonus. Burst size The maximum number of bytes to allow in a burst. If this parameter is set, a port might gain a burst bonus when it does not use all its allocated bandwidth.
vSphere Networking Setting Description Failback Select Yes or No to disable or enable failback. This option determines how a physical adapter is returned to active duty after recovering from a failure. If failback is set to Yes (default), the adapter is returned to active duty immediately upon recovery, displacing the standby adapter that took over its slot, if any.
vSphere Networking 3 Select General to edit the following distributed port group settings. Option Description Name The name of distributed port group. You can edit the name in the text field. Port binding Choose when ports are assigned to virtual machines connected to this distributed port group. Port allocation 4 n Static binding: Assign a port to a virtual machine when the virtual machine connects to the distributed port group.
vSphere Networking 4 (Optional) Use the policy pages to set overrides for each port policy. 5 Click OK. Remove a Distributed Port Group Remove a distributed port group when you no longer need the corresponding labeled network to provide connectivity and configure connection settings for virtual machines or VMkernel networking. Prerequisites n Verify that all virtual machines connected to the corresponding labeled network are migrated to a different labeled network.
vSphere Networking 4 Click the Start Monitoring Port State icon. The ports table for the distributed port group displays runtime statistics for each distributed port. The State column displays the current state for each distributed port. Option Description Link Up The link for this distributed port is up. Link Down The link for this distributed port is down. Blocked This distributed port is blocked. -- The state of this distributed port is currently unavailable.
vSphere Networking Migrate Virtual Machines to or from a vSphere Distributed Switch In addition to connecting virtual machines to a distributed switch at the individual virtual machine level, you can migrate a group of virtual machines between a vSphere Distributed Switch network and a vSphere Standard Switch network. Procedure 1 In the vSphere Web Client, navigate to a data center. 2 Right-click the data center in the navigator and select Migrate VMs to Another Network. 3 Select a source network.
vSphere Networking Topology Diagrams of a vSphere Distributed Switch in the vSphere Web Client The topology diagrams of a vSphere Distributed Switch in the vSphere Web Client show the structure of virtual machine adapters, VMkernel adapters, and physical adapters in the switch. You can examine the components, arranged in port groups, whose traffic is handled by the switch, and the connections between them.
vSphere Networking 2 On the Configure tab, expand Settings and selectTopology. By default the diagram shows up to 32 distributed port groups, 32 hosts, and 1024 virtual machines. Example: Diagram of a Distributed Switch That Connects the VMkernel and Virtual Machines to the Network In your virtual environment, a vSphere Distributed Switch handles VMkernel adapters for vSphere vMotion and for the management network, and virtual machines grouped.
vSphere Networking n Handle networking components on multiple hosts by using the Add and Manage Hosts wizard. n View the physical NIC or NIC team that carries the traffic related to a selected virtual machine adapter or VMkernel adapter. In this way you can also view the host on which a selected VMkernel adapter resides. Select the adapter, trace the route to the associated physical NIC, and view the IP address or domain name next to the NIC. n Determine the VLAN mode and ID for a port group.
Setting Up VMkernel Networking 4 You set up VMkernel adapters to provide network connectivity to hosts and to accommodate system traffic of vMotion, IP storage, Fault Tolerance logging, vSAN, and so on. n VMkernel Networking Layer The VMkernel networking layer provides connectivity to hosts and handles the standard system traffic of vSphere vMotion, IP storage, Fault Tolerance, vSAN, and others.
vSphere Networking n View TCP/IP Stack Configuration on a Host You can view the DNS and routing configuration of a TCP/IP stack on a host. You can also view the IPv4 and IPv6 routing tables, the congestion control algorithm, and the maximum number of allowed connections. n Change the Configuration of a TCP/IP Stack on a Host You can change the DNS and default gateway configuration of a TCP/IP stack on a host.
vSphere Networking configured with the provisioning TCP/IP stack handle the traffic from cloning the virtual disks of the migrated virtual machines in long-distance vMotion. By using the provisioning TCP/IP stack, you can isolate the traffic from the cloning operations on a separate gateway. After you configure a VMkernel adapter with the provisioning TCP/IP stack, all adapters on the default TCP/IP stack are disabled for the Provisioning traffic.
vSphere Networking IP storage traffic and discovery Handles the connection for storage types that use standard TCP/IP networks and depend on the VMkernel networking. Such storage types are software iSCSI, dependent hardware iSCSI, and NFS. If you have two or more physical NICs for iSCSI, you can configure iSCSI multipathing. ESXi hosts support NFS 3 and 4.1. To configure a software Fibre Channel over Ethernet (FCoE) adapter, you must have a dedicated VMkernel adapter.
vSphere Networking Tab Description IP Settings Displays all IPv4 and IPv6 settings for the VMkernel adapter. IPv6 information is not displayed if IPv6 has not been enabled on the host. Policies Displays the configured traffic shaping, teaming and failover, and security policies that apply for the port group to which the VMkernel adapter is connected.
vSphere Networking 8 Option Description TCP/IP stack Select a TCP/IP stack from the list. After you set a TCP/IP stack for the VMkernel adapter, you cannot change it later. If you select the vMotion or the Provisioning TCP/IP stack, you will be able to use only this stack to handle vMotion or Provisioning traffic on the host. All VMkernel adapters for vMotion on the default TCP/IP stack are disabled for future vMotion sessions.
vSphere Networking 10 (Optional) On the IPv6 settings page, select an option for obtaining IPv6 addresses. Option Description Obtain IPv6 addresses automatically through DHCP Use DHCP to obtain IPv6 addresses. A DHCPv6 server must be present on the network. Obtain IPv6 addresses automatically through Router Advertisement Use router advertisement to obtain IPv6 addresses. Static IPv6 addresses In ESXi 6.
vSphere Networking 7 Option Description TCP/IP stack Select a TCP/IP stack from the list. Once you set a TCP/IP stack for the VMkernel adapter, you cannot change it later. If you select the vMotion or the Provisioning TCP/IP stack, you will be able to use only these stacks to handle vMotion or Provisioning traffic on the host. All VMkernel adapters for vMotion on the default TCP/IP stack are disabled for future vMotion sessions.
vSphere Networking 9 (Optional) On the IPv6 settings page, select an option for obtaining IPv6 addresses. Option Description Obtain IPv6 addresses automatically through DHCP Use DHCP to obtain IPv6 addresses. A DHCPv6 server must be present on the network. Obtain IPv6 addresses automatically through Router Advertisement Use router advertisement to obtain IPv6 addresses. Static IPv6 addresses In ESXi 6.
vSphere Networking 5 On the NIC settings page, set the MTU for the network adapter. 6 With IPv4 enabled, in the IPv4 settings section, select the method by which IP addresses are obtained. Option Description Obtain IPv4 settings automatically Use DHCP to obtain IP settings. A DHCP server must be present on the network. Use static IPv4 settings Enter the IPv4 IP address and subnet mask for the VMkernel adapter.
vSphere Networking For example, the VMkernel adapters vmk0 and vmk1 can be configured on a host. n vmk0 is used for management traffic on the 10.162.10.0/24 subnet, with default gateway 10.162.10.1 n vmk1 is used for vMotion traffic on the 172.16.1.0/24 subnet If you set 172.16.1.1 as the default gateway for vmk1, vMotion uses vmk1 as its egress interface with the gateway 172.16.1.1. The 172.16.1.1 gateway is a part of the vmk1 configuration and is not in the routing table.
vSphere Networking Where vmknic is the name of the VMkernel adapter, gateway is the IP address of the gateway, IP address is the address of the VMkernel adapter, and mask is the network mask. View TCP/IP Stack Configuration on a Host You can view the DNS and routing configuration of a TCP/IP stack on a host. You can also view the IPv4 and IPv6 routing tables, the congestion control algorithm, and the maximum number of allowed connections. Procedure 1 In the vSphere Web Client, navigate to the host.
vSphere Networking 3 Select a stack from the table, click Edit and make the appropriate changes. Page Option Name Change the name of a custom TCP/IP stack DNS Configuration Select a method of obtaining the DNS server. Routing n Select Obtain settings automatically from a VMkernel network adapter and select a network adapter from the VMKernel network adapter drop-down menu n Select Enter settings manually and edit the DNS configuration settings. a Edit the Host name. b Edit the Domain name.
vSphere Networking Procedure 1 In the vSphere Web Client, navigate to the host. 2 On the Configure tab, expand Networking and select VMkernel adapters. 3 Select a VMkernel adapter from the list, and click the Remove selected network adapter icon. 4 In the confirmation dialog box, click Analyze impact. 5 If you use software iSCSI adapters with port binding, review the impact on their networking configuration.
LACP Support on a vSphere Distributed Switch 5 With LACP support on a vSphere Distributed Switch, you can connect ESXi hosts to physical switches by using dynamic link aggregation. You can create multiple link aggregation groups (LAGs) on a distributed switch to aggregate the bandwidth of physical NICs on ESXi hosts that are connected to LACP port channels. VMware, Inc.
vSphere Networking Figure 5‑1.
vSphere Networking On a host proxy switch, you can connect one physical NIC to only one LAG port. On the distributed switch, one LAG port can have multiple physical NICs from different hosts connected to it. The physical NICs on a host that you connect to the LAG ports must be connected to links that participate in an LACP port channel on the physical switch. You can create up to 64 LAGs on a distributed switch. A host can support up to 32 LAGs.
vSphere Networking Table 5‑1. LACP Teaming and failover configuration of distributed port groups Failover Order Uplinks Description Active A single LAG You can only use one active LAG or multiple standalone uplinks to handle the traffic of distributed port groups . You cannot configure multiple active LAGs or mix active LAGs and standalone uplinks. Standby Empty Having an active LAG and standby uplinks and the reverse is not supported. Having a LAG and another standby LAG is not supported.
vSphere Networking 4 Set the Link Aggregation Group as Active in the Teaming and Failover Order of the Distributed Port Group You migrated physical NICs to the ports of the link aggregation group (LAG). Set the LAG as active and move all standalone uplinks as unused in the teaming and failover order of the distributed port groups. Create a Link Aggregation Group To migrate the network traffic of distributed port groups to a link aggregation group (LAG), you create a new LAG on the distributed switch.
vSphere Networking The new LAG is unused in the teaming and failover order of distributed port groups. No physical NICs are assigned to the LAG ports. As with standalone uplinks, the LAG has a representation on every host that is associated with the distributed switch. For example, if you create LAG1 with two ports on the distributed switch, a LAG1 with two ports is created on every host that is associated with the distributed switch.
vSphere Networking n Verify that the physical NICs that you want to assign to the LAG ports have the same speed and are configured at full duplex. Procedure 1 In the vSphere Web Client, navigate to the distributed switch where the LAG resides. 2 From the Actions menu, select Add and Manage Hosts. 3 Select Manage host networking. 4 Select the host whose physical NICs you want to assign to the LAG ports and click Next.
vSphere Networking You safely migrated network traffic from standalone uplinks to a LAG for distributed port groups and created a valid LACP teaming and failover configuration for the groups. Example: Topology of a Distributed Switch that Uses a LAG If you configure a LAG with two ports to handle the traffic of a distributed port group, you can check the topology of the distributed switch to view how it changed as a result of the new configuration. Figure 5‑2.
vSphere Networking 8 Change the VLAN and the NetFlow policies. This option is active when the option for overriding the VLAN and NetFlow policies for individual ports is enabled on the uplink port group. If you change the VLAN and NetFlow policies for the LAG, they override the policies set at the uplink port group level. 9 Click OK.
Backing Up and Restoring Networking Configurations 6 vSphere enables you to backup and restore the configuration of a vSphere Distributed Switch , distributed and uplink port groups in cases of invalid changes or a transfer to another deployment.
vSphere Networking 4 (Optional) Enter notes about this configuration in the Descriptions field. 5 Click OK. 6 Click Yes to save the configuration file to your local system. What to do next Use the exported configuration file to do the following tasks: n Create a copy of the exported distributed switch in a vSphere environment. See Import a vSphere Distributed Switch Configuration. n Overwrite the settings on an existing distributed switch. See Restore a vSphere Distributed Switch Configuration.
vSphere Networking Restore a vSphere Distributed Switch Configuration Use the restore option to reset the configuration of an existing distributed switch to the settings in the configuration file. Restoring a distributed switch changes the settings on the selected switch back to the settings saved in the configuration file. Note You can use a saved configuration file to restore policies and hosts associations on the distributed switch.
vSphere Networking 2 Right-click the distributed port group and select Export Configuration. 3 (Optional) In the Descriptions field, type notes about this configuration. 4 Click OK. Click Yes to save the configuration file to your local system. You now have a configuration file that contains all the settings for the selected distributed port group.
vSphere Networking 2 Right-click the distributed port group and select Restore Configuration. 3 Select one of the following and click Next: 4 u Restore to previous configuration to roll your port group configuration back one step. You cannot restore the port group configuration completely if you have performed more than one step. u Restore configuration from a file lets you restore the port group configuration from an exported backup file.
Rollback and Recovery of the Management Network 7 You can prevent and recover from misconfiguration of the management network by using the rollback and recovery support of the vSphere Distributed Switch and vSphere Standard Switch. Rollback is available for use on both standard and distributed switches. To fix invalid configuration of the management network, you can connect directly to a host to fix the issues through the DCUI.
vSphere Networking n Removing the management VMkernel network adapter from a standard or distributed switch. n Removing a physical NIC of a standard or distributed switch containing the management VMkernel network adapter. n Migrating the management VMkernel adapter from vSphere standard to distributed switch. If a network disconnects for any of these reasons, the task fails and the host reverts to the last valid configuration.
vSphere Networking 3 Click Edit. 4 Select the config.vpxd.network.rollback key, and change the value to false. If the key is not present, you can add it and set the value to false. 5 Click OK. 6 Restart vCenter Server to apply the changes. Disable Network Rollback by Using the vCenter Server Configuration File Rollback is enabled by default in vSphere. You can disable rollback by editing the vpxd.cfg configuration file of vCenter Server directly.
vSphere Networking If the uplinks that you use to restore the management network are also used by VMkernel adapters that handle other types of traffic (vMotion, Fault Tolerance, and so on), the adapters loose network connectivity after the restore. For more information about accessing and using the DCUI, see the vSphere Security documentation. Note Recovery of the management connection on a distributed switch is not supported on stateless ESXi instances.
Networking Policies 8 Policies set at the standard switch or distributed port group level apply to all of the port groups on the standard switch or to ports in the distributed port group. The exceptions are the configuration options that are overridden at the standard port group or distributed port level. Watch the video about applying networking policies on vSphere standard and distributed switches. Working with Networking Policies (http://link.brightcove.
vSphere Networking n Resource Allocation Policy The Resource Allocation policy allows you to associate a distributed port or port group with a usercreated network resource pool. This policy provides you with greater control over the bandwidth given to the port or port group. n Monitoring Policy The monitoring policy enables or disables NetFlow monitoring on a distributed port or port group.
vSphere Networking Table 8‑2. Policies Available for a vSphere Standard Switch and vSphere Distributed Switch Policy Standard Switch Distributed Switch Teaming and failover Yes Yes Lets you configure the physical NICs that handle the network traffic for a standard switch, standard port group, distributed port group, or distributed port. You arrange the physical NICs in a failover order and apply different load balancing policies over them.
vSphere Networking 3 Select the Advanced page. Option Description Configure reset at disconnect From the drop-down menu, enable or disable reset at disconnect. When a distributed port is disconnected from a virtual machine, the configuration of the distributed port is reset to the distributed port group setting. Any per-port overrides are discarded. Override port policies Select the distributed port group policies to be overridden on a per-port level.
vSphere Networking Network Failure Detection Policy You can specify one of the following methods that a virtual switch uses for failover detection. Link status only Beacon probing Relies only on the link status that the network adapter provides. Detects failures, such as removed cables and physical switch power failures. However, link status does not detect the following configuration errors: n Physical switch port that is blocked by spanning tree or is misconfigured to the wrong VLAN .
vSphere Networking Notify Switches Policy By using the notify switches policy, you can determine how the ESXi host communicates failover events. When a physical NIC connects to the virtual switch or when traffic is rerouted to a different physical NIC in the team, the virtual switch sends notifications over the network to update the lookup tables on physical switches. Notifying the physical switch offers lowest latency when a failover or a migration with vSphere vMotion occurs.
vSphere Networking Each virtual machine running on an ESXi host has an associated virtual port ID on the virtual switch. To calculate an uplink for a virtual machine, the virtual switch uses the virtual machine port ID and the number of uplinks in the NIC team. After the virtual switch selects an uplink for a virtual machine, it always forwards traffic through the same uplink for this virtual machine as long as the machine runs on the same port.
vSphere Networking Table 8‑4. Considerations on Using Route Based on Source MAC Hash Considerations Description Advantages n A more even distribution of the traffic than Route Based on Originating Virtual Port, because the virtual switch calculates an uplink for every packet. n Virtual machines use the same uplink because the MAC address is static. Powering a virtual machine on or off does not change the uplink that the virtual machine uses. n No changes on the physical switch are required.
vSphere Networking Physical Switch Configuration To ensure that IP hash load balancing works correctly, you must have an Etherchannel configured on the physical switch. An Etherchannel bonds multiple network adapters into a single logical link. When ports are bound into an Etherchannel, every time the physical switch receives a packet from the same virtual machine MAC address on different ports, the switch updates its content addressable memory (CAM) table correctly.
vSphere Networking Route Based on Physical NIC Load Route Based on Physical NIC Load is based on Route Based on Originating Virtual Port, where the virtual switch checks the actual load of the uplinks and takes steps to reduce it on overloaded uplinks. Available only for vSphere Distributed Switch. The distributed switch calculates uplinks for virtual machines by taking their port ID and the number of uplinks in the NIC team.
vSphere Networking 3 Navigate to the Teaming and Failover policy for the standard switch, or standard port group. Option Action Standard Switch a Select the switch from the list. b Click Edit settings and select Teaming and failover. a Select the switch where the port group resides. b From the switch topology diagram, select the standard port group and click Edit settings. c Select Teaming and failover. d Select Override next to the policies that you want to override.
vSphere Networking 7 From the Failback drop-down menu, select whether a physical adapter is returned to active status after recovering from a failure. If failback is set to Yes, the default selection, the adapter is returned to active duty immediately upon recovery, displacing the standby adapter that took over its slot, if any. If failback is set to No for a standard port, a failed adapter is left inactive after recovery until another currently active adapter fails and must be replaced.
vSphere Networking 2 Navigate the Teaming and Failover policy on the distributed port group or port. Option Action Distributed port group a From the Actions menu, select Distributed Port Group > Manage Distributed Port Groups. b Select Teaming and failover. c Select the port group and click Next. a On the Networks tab, click Distributed Port Groups and double-click a distributed port group. Distributed port 3 b On the Ports tab, select a port and click Edit distributed port settings.
vSphere Networking 5 From the Notify switches drop-down menu, select whether the standard or distributed switch notifies the physical switch in case of a failover. Note Set this option to No if a connected virtual machine is using Microsoft Network Load Balancing in unicast mode. No issues exist with Network Load Balancing running in multicast mode. 6 From the Failback drop-down menu, select whether a physical adapter is returned to active status after recovering from a failure.
vSphere Networking Configure VLAN Tagging on a Distributed Port Group or Distributed Port To apply VLAN tagging globally on all distributed ports, you must set the VLAN policy on a distributed port group. To integrate the virtual traffic on the port with physical VLANs in a different way from the parent distributed port group, you must use the VLAN policy on a distributed port. Prerequisites To override a policy on distributed port level, enable the port-level override option for this policy.
vSphere Networking Configure VLAN Tagging on an Uplink Port Group or Uplink Port To configure VLAN traffic processing generally for all member uplinks, you must set the VLAN policy on an uplink port. To handle VLAN traffic through the port in a different way than for the parent uplink port group, you must set the VLAN policy on an uplink . Use the VLAN policy at the uplink port level to propagate a trunk range of VLAN IDs to the physical network adapters for traffic filtering.
vSphere Networking The security policy of a standard or distributed switch is implemented in Layer 2 (Data Link Layer) of the network protocol stack. The three elements of the security policy are promiscuous mode, MAC address changes, and forged transmits. See the vSphere Security documentation for information about potential networking threats.
vSphere Networking 4 Reject or accept promiscuous mode activation or MAC address changes in the guest operating system of the virtual machines attached to the standard switch or port group. Option Description Promiscuous mode n Reject. The VM network adapter receives only frames that are addressed to the virtual machine. n Accept.The virtual switch forwards all frames to the virtual machine in compliance with the active VLAN policy for the port to which the VM network adapter is connected.
vSphere Networking 2 Navigate to the Security policy for the distributed port group or port. Option Action Distributed port group a From the Actions menu, select Distributed Port Group > Manage Distributed Port Groups. b Select Security. c Select the port group and click Next. a On the Networks tab, click Distributed Port Groups and double-click a distributed port group . b On the Ports tab, select a port and click the Edit distributed port settings icon. c Select Security.
vSphere Networking ESXi shapes outbound network traffic on standard switches and inbound and outbound traffic on distributed switches. Traffic shaping restricts the network bandwidth available on a port, but can also be configured to allow bursts of traffic to flow through at higher speeds. Average Bandwidth Establishes the number of bits per second to allow across a port, averaged over time. This number is the allowed average load.
vSphere Networking 4 Configure traffic shaping policies. Option Description Status Enables setting limits on the amount of networking bandwidth allocated for each port that is associated with the standard switch or port group. Average Bandwidth Establishes the number of bits per second to allow across a port, averaged over time (the allowed average load). Peak Bandwidth The maximum number of bits per second to allow across a port when it is sending a burst of traffic.
vSphere Networking 2 Navigate to the Traffic Shaping policy for the distributed port group or port. Option Action Distributed port group a From the Actions menu, select Distributed Port Group > Manage Distributed Port Groups. b Select Traffic shaping. c Select the port group and click Next. a On the Networks tab, click Distributed Port Groups and double-click a distributed port group . b On the Ports tab, select a port and click the Edit distributed port settings icon.
vSphere Networking Edit the Resource Allocation Policy on a Distributed Port Group Associate a distributed port group with a network resource pool to give you greater control over the bandwidth that is given to the distributed port group. Prerequisites n Enable Network I/O Control on the distributed switch. See Enable Network I/O Control on a vSphere Distributed Switch. n Create and configure network resource pools. See Create a Network Resource Pool.
vSphere Networking Prerequisites To override a policy on distributed port level, enable the port-level override option for this policy. See Configure Overriding Networking Policies on Port Level. Procedure 1 In the vSphere Web Client, navigate to the distributed switch. 2 Navigate to the monitoring policy for the distributed port group or distributed port. Option Action Distributed port group a From the Actions menu, select Distributed Port Group > Manage Distributed Port Groups.
vSphere Networking n Mark Traffic on a Distributed Port Group or Uplink Port Group Assign priority tags to traffic, such as VoIP and streaming video, that has higher networking requirements for bandwidth, low latency, and so on. You can mark the traffic with a CoS tag in Layer 2 of the network protocol stack or with a DSCP tag in Layer 3.
vSphere Networking Mark Traffic on a Distributed Port Group or Uplink Port Group Assign priority tags to traffic, such as VoIP and streaming video, that has higher networking requirements for bandwidth, low latency, and so on. You can mark the traffic with a CoS tag in Layer 2 of the network protocol stack or with a DSCP tag in Layer 3. Priority tagging is a mechanism to mark traffic that has higher QoS demands. In this way, the network can recognize different classes of traffic.
vSphere Networking 8 Specify the kind of traffic that the rule is applicable to. To determine if a data flow is in the scope of a rule for marking or filtering, the vSphere distributed switch examines the direction of the traffic, and properties like source and destination, VLAN, next level protocol, infrastructure traffic type, and so on. a From the Traffic direction drop-down menu, select whether the traffic must be ingress, egress, or both so that the rule recognizes it as matching.
vSphere Networking Rule Parameter Parameter Value Destination port 5060 Source address IP address matches 192.168.2.0 with prefix length 24 Filter Traffic on a Distributed Port Group or Uplink Port Group Allow or stop traffic for securing the data that flows through the ports of a distributed port group or uplink port group. Procedure 1 Locate a distributed port group or an uplink port group in the vSphere Web Client. a Select a distributed switch and click the Networks tab.
vSphere Networking 7 Specify the kind of traffic that the rule is applicable to. To determine if a data flow is in the scope of a rule for marking or filtering, the vSphere distributed switch examines the direction of the traffic, and properties like source and destination, VLAN, next level protocol, infrastructure traffic type, and so on. a From the Traffic direction drop-down menu, select whether the traffic must be ingress, egress, or both so that the rule recognizes it as matching.
vSphere Networking n Edit a Traffic Rule on a Distributed Port Group or Uplink Port Group Create or edit traffic rules, and use their parameters to configure a policy for filtering or marking the traffic on a distributed port group or uplink port group. n Change Rule Priorities on a Distributed Port Group or Uplink Port Group Reorder the rules that form the traffic filtering and marking policy of a distributed port group or uplink port group to change the sequence of actions for processing traffic.
vSphere Networking 4 If traffic filtering and marking is disabled, enable it from the Status drop-down menu. 5 Click New to create a new rule, or select a rule and click Edit to edit it. What to do next Name the network traffic rule, and deny, allow, or tag the target traffic.
vSphere Networking Disable Traffic Filtering and Marking on a Distributed Port Group or Uplink Port Group Let traffic flow to virtual machines or physical adapters without additional control related to security or QoS by disabling the traffic filtering and marking policy. Note You can enable and set up the traffic filtering and marking policy on a particular port. See Enable Traffic Filtering and Marking on a Distributed Port or Uplink Port.
vSphere Networking n Disable Traffic Filtering and Marking on a Distributed Port or Uplink Port Disable the traffic filtering and marking policy on a port to let traffic flow to a virtual machine or a physical adapter without filtering for security or marking for QoS.
vSphere Networking Prerequisites To override a policy on distributed port level, enable the port-level override option for this policy. See Configure Overriding Networking Policies on Port Level. Procedure 1 Navigate to a distributed switch and then navigate to a distributed port or an uplink port. n To navigate to the distributed ports of the switch, click Networks > Distributed Port Groups, double-click a distributed port group from the list, and click the Ports tab.
vSphere Networking 8 Specify the kind of traffic that the rule is applicable to. To determine if a data flow is in the scope of a rule for marking or filtering, the vSphere distributed switch examines the direction of the traffic, and properties like source and destination, VLAN, next level protocol, infrastructure traffic type, and so on. a From the Traffic direction drop-down menu, select whether the traffic must be ingress, egress, or both so that the rule recognizes it as matching.
vSphere Networking 2 Select a port from the list. 3 Click Edit distributed port settings. 4 If traffic filtering and marking is not enabled at the port level, click Override, and from the Status drop-down menu, select Enabled. 5 Click New to create a new rule, or select a rule and click Edit to edit it. You can change a rule inherited from the distributed port group or uplink port group. In this way, the rule becomes unique within the scope of the port.
vSphere Networking Working with Network Traffic Rules on a Distributed Port or Uplink Port Define traffic rules in a distributed port or uplink port group to introduce a policy for processing traffic related to a virtual machine or to a physical adapter. You can filter specific traffic or describe its QoS demands. n View Traffic Rules on a Distributed Port or Uplink Port Review the traffic rules that form the traffic filtering and marking policy of a distributed port or uplink port.
vSphere Networking 7 From the upper list, select the rule for which you want to view the criteria for locating traffic. The traffic qualifying parameters of the rule appear in the Traffic Qualifiers list. Edit a Traffic Rule on a Distributed Port or Uplink Port Create or edit traffic rules, and use their parameters to configure a policy for filtering or marking the traffic on a distributed port or uplink port.
vSphere Networking Procedure 1 Navigate to a distributed switch and then navigate to a distributed port or an uplink port. n To navigate to the distributed ports of the switch, click Networks > Distributed Port Groups, double-click a distributed port group from the list, and click the Ports tab. n To navigate to the uplink ports of an uplink port group, click Networks > Uplink Port Groups, double-click an uplink port group from the list, and click the Ports tab. 2 Select a port from the list.
vSphere Networking Disable Traffic Filtering and Marking on a Distributed Port or Uplink Port Disable the traffic filtering and marking policy on a port to let traffic flow to a virtual machine or a physical adapter without filtering for security or marking for QoS. Prerequisites To override a policy on distributed port level, enable the port-level override option for this policy. See Configure Overriding Networking Policies on Port Level.
vSphere Networking MAC Traffic Qualifier By using the MAC traffic qualifier in a rule, you can define matching criteria for the Layer 2 (Data Link Layer) properties of packets such as MAC address, VLAN ID, and next level protocol that consumes the frame payload. Protocol Type The Protocol type attribute of the MAC traffic qualifier corresponds to the EtherType field in Ethernet frames. EtherType represents the type of next level protocol that is going to consume the payload of the frame.
vSphere Networking Destination Address By using the Destination Address group of attributes, you can match packets to their destination address. The MAC destination address options have the same format as those for the source address. Comparison Operators To match traffic in a MAC qualifier more closely to your needs, you can use affirmative comparison or negation. You can use operators such that all packets except the ones with certain attributes fall in the scope of a rule.
vSphere Networking Destination Address Use the Destination Address to match packets by IP address, subnet, or IP version. The destination address has the same format as the one for the source. Comparison Operators To match traffic in an IP qualifier more closely to your needs, you can use affirmative comparison or negation. You can define that all packets fall in the scope of a rule except packets with certain attributes.
vSphere Networking 5 (Optional) On the Security page, use the drop-down menus to edit the security exceptions and click Next. Option Description Promiscuous mode n Reject. Placing a guest adapter in promiscuous mode has no effect on which frames are received by the adapter. n Accept. Placing a guest adapter in promiscuous mode causes it to detect all frames passed on the vSphere Distributed Switch that are allowed under the VLAN policy for the port group that the adapter is connected to. n Reject.
vSphere Networking 8 (Optional) On the Teaming and failover page, use the drop-down menus to edit the settings and click Next. Option Description Load balancing IP-based teaming requires that the physical switch be configured with ether channel. For all other options, ether channel should be disabled. Select how to choose an uplink. Network failure detection Notify switches n Route based on the originating virtual port.
vSphere Networking 9 (Optional) On the Resource allocation page, use the Network resource pool drop-down menu to add or remove resource allocations and click Next. 10 (Optional) On the Monitoring page, use the drop-menu to enable or disable NetFlow and click Next. Option Description Disabled NetFlow is disabled on the distributed port group. Enabled NetFlow is enabled on the distributed port group. You can configure NetFlow settings at the vSphere Distributed Switch level.
vSphere Networking Option MAC qualifier Description Qualify the traffic for the rule by Layer 2 header. n Protocol type. Set the next level protocol (IPv4, IPv6, etc.) consuming the payload. This attribute corresponds to the EtherType field in Ethernet frames. You can select a protocol from the drop-down menu or type its hexadecimal number For example, to locate traffic for the Link Layer Discovery Protocol (LLDP) protocol, type 88CC. n VLAN ID. Locate traffic by VLAN.
vSphere Networking 12 (Optional) On the Miscellaneous page, select Yes or No from the drop-down menu and click Next. SelectYes to shut down all ports in the port group. This shutdown might disrupt the normal network operations of the hosts or virtual machines using the ports. 13 Review your settings on the Ready to complete page and click Finish. Use the Back button to change any settings. Port Blocking Policies Port blocking policies allow you to selectively block ports from sending or receiving data.
vSphere Networking 2 Select a port from the list. 3 Click Edit distributed port settings. 4 In the Miscellaneous section, select the Override check box, and from the drop-down menu enable or disable port blocking. 5 Click OK. VMware, Inc.
Isolating Network Traffic by Using VLANs 9 VLANs let you segment a network into multiple logical broadcast domains at Layer 2 of the network protocol stack. This chapter includes the following topics: n VLAN Configuration n Private VLANs VLAN Configuration Virtual LANs (VLANs) enable a single physical LAN segment to be further isolated so that groups of ports are isolated from one another as if they were on physically different segments.
vSphere Networking Tagging Mode VLAN ID on switch port groups EST 0 The physical switch performs the VLAN tagging. The host network adapters are connected to access ports on the physical switch. VST Between 1 and 4094 The virtual switch performs the VLAN tagging before the packets leave the host. The host network adapters must be connected to trunk ports on the physical switch.
vSphere Networking 4 To add a primary VLAN, under Primary VLAN ID click Add and enter the ID of a primary VLAN. 5 Click the plus sign (+) in front of the primary VLAN ID to add it to the list. The primary private VLAN also appears under Secondary Private VLAN ID. 6 To add a secondary VLAN, in the right pane click Add and enter the ID of the VLAN. 7 Click the plus sign (+) in front of the secondary VLAN ID to add it to the list.
vSphere Networking 2 On the Configure tab, expand Settings and select Private VLAN. 3 Click Edit. 4 Select a primary private VLAN. The secondary private VLANs associated with it appear on the right. 5 Select the secondary private VLAN to remove. 6 Under the secondary VLAN ID list, click Remove and click OK. VMware, Inc.
Managing Network Resources 10 vSphere provides several different methods to help you manage your network resources.
vSphere Networking n Configure a PCI Device on a Virtual Machine Passthrough devices provide the means to more efficiently use resources and improve performance in your environment. You can configure a passthrough PCI device on a virtual machine in the vSphere Web Client. Enable Passthrough for a Network Device on a Host Passthrough devices provide the means to use resources efficiently and improve performance of your environment. You can enable DirectPath I/O passthrough for a network device on a host.
vSphere Networking Procedure 1 Locate the virtual machine in the vSphere Web Client. a Select a data center, folder, cluster, resource pool, or host and click the VMs tab. b Click Virtual Machines and double-click the virtual machine from the list. 2 Power off the virtual machine. 3 On the Configure tab of the virtual machine, expand Settings and select VM Hardware. 4 Click Edit and select the Virtual Hardware tab in the dialog box displaying the settings.
vSphere Networking In vSphere, though a virtual switch (standard switch or distributed switch) does not handle the network traffic of an SR-IOV enabled virtual machine connected to the switch, you can control the assigned virtual functions by using switch configuration policies at port group or port level. SR-IOV Support vSphere supports SR-IOV in an environment with specific configuration only. Some features of vSphere are not functional when SR-IOV is enabled.
vSphere Networking Availability of Features The following features are not available for virtual machines configured with SR-IOV: n vSphere vMotion n Storage vMotion n vShield n NetFlow n VXLAN Virtual Wire n vSphere High Availability n vSphere Fault Tolerance n vSphere DRS n vSphere DPM n Virtual machine suspend and resume n Virtual machine snapshots n MAC-based VLAN for passthrough virtual functions n Hot addition and removal of virtual devices, memory, and vCPU n Participation
vSphere Networking In a host that runs virtual machine traffic on top of SR-IOV physical adapters, virtual machine adapters directly contact the virtual functions to communicate data. However, the ability to configure networks is based on the active policies for the port holding the virtual machines. On an ESXi host without SR-IOV, the virtual switch sends external network traffic through its ports on the host from or to the physical adapter for the relevant port group.
vSphere Networking Configuration Path in SR-IOV If the guest operating system attempts to change the configuration of a virtual machine adapter mapped to a VF, the change takes place if it is allowed by the policy on the port associated with the virtual machine adapter. The configuration workflow consists of the following operations: 1 The guest operating system requests a configuration change on the VF. 2 The VF forwards the request to the PF through a mailbox mechanism.
vSphere Networking The total number of interrupt vectors on each ESXi host can scale up to 4096 in the case of 32 CPUs. When the host boots, devices on the host such as storage controllers, physical network adapters, and USB controllers consume a subset of the 4096 vectors. If these devices require more than 1024 vectors, the maximum number of potentially supported VFs is reduced. n The number of VFs that is supported on an Intel NIC might be different from the number that is supported on an Emulex NIC.
vSphere Networking Prerequisites Verify that the configuration of your environment supports SR-IOV. See SR-IOV Support. Procedure 1 Enable SR-IOV on a Host Physical Adapter Before you can connect virtual machines to virtual functions, use the vSphere Web Client to enable SR-IOV and set the number of virtual functions on your host.
vSphere Networking The virtual functions become active on the NIC port represented by the physical adapter entry. They appear in the PCI Devices list in the Settings tab for the host. You can use the esxcli network sriovnic vCLI commands to examine the configuration of virtual functions on the host. What to do next Associate a virtual machine with a virtual function through an SR-IOV passthrough network adapter.
vSphere Networking 10 Expand the Memory section, select Reserve all guest memory (All locked) and click OK. I/O memory management unit (IOMMU) must reach all virtual machine memory so that the passthrough device can access the memory by using direct memory access (DMA). 11 Power on the virtual machine. When you power on the virtual machine, the ESXi host selects a free virtual function from the physical adapter and maps it to the SR-IOV passthrough adapter.
vSphere Networking Mixed Mode The physical adapter provides virtual functions to virtual machines attached to the switch and directly handles traffic from non SR-IOV virtual machines on the switch. You can check whether an SR-IOV physical adapter is in mixed mode in the topology diagram of the switch. An SR-IOV physical adapter in mixed mode appears with the icon in the list of physical adapters for a standard switch or in the list of uplink group adapters for a distributed switch.
vSphere Networking 2 Select the host profile from the list and click the Configure tab. 3 Click Edit Host Profile and expand the General System Settings node. 4 Expand Kernel Module Parameter and select the parameter of the physical function driver for creating virtual functions. For example, the parameter for the physical function driver of an Intel physical NIC is max_vfs. 5 In the Value text box, type a comma-separated list of valid virtual function numbers.
vSphere Networking Where driver is the name of the NIC driver, and vf_param is the driver-specific parameter for creating the virtual function. You can use a comma-separated list to set values for the vf_param parameter, where each entry indicates the number of virtual functions for a port. A value of 0 ensures that SR-IOV is not enabled for that physical function.
vSphere Networking Cause The number of allocatable interrupt vectors scales up with the number of physical CPUs on an ESXi host. An ESXi host that has 32 CPUs can provide a total of 4096 interrupt vectors. When the host boots, devices on the host such as storage controllers, physical network adapters, and USB controllers consume a subset of the 4096 vectors. If these devices require more than 1024 vectors, the maximum number of potentially supported VFs is reduced.
vSphere Networking For two virtual machines that run on different ESXi hosts, when at least one of the hosts does not have a physical RDMA device, the communication falls back to a TCP-based channel and the performance is reduced. PVRDMA Support vSphere 6.5 and later supports PVRDMA only in environments with specific configuration. Supported Configurations To use PVRDMA in vSphere 6.5, your environment must meet several configuration requirements. Table 10‑3.
vSphere Networking n Enable the Firewall Rule for PVRDMA Enable the firewall rule for PVRDMA in the security profile of the ESXi host. Tag a VMkernel Adapter for PVRDMA Select a VMkernel adapter and enable it for PVRDMA communication. Procedure 1 In the vSphere Web Client, navigate to the host. 2 On the Configure tab, expand System. 3 Click Advanced System Settings. 4 Locate Net.PVRDMAVmknic and click Edit.
vSphere Networking Procedure 1 Locate the virtual machine in the vSphere Web Client. a Select a data center, folder, cluster, resource pool, or host and click the VMs tab. b Click Virtual Machines and double-click the virtual machine from the list. 2 Power off the virtual machine. 3 On the Configure tab of the virtual machine, expand Settings and select VM Hardware. 4 Click Edit and select the Virtual Hardware tab in the dialog box displaying the settings.
vSphere Networking Lossless Layer 3 Network RoCE v2 requires that lossless data transfer is preserved at layer 3 routing devices. To enable the transfer of layer 2 PFC lossless priorities across layer 3 routers, configure the router to map the received priority setting of a packet to the corresponding Differentiated Serviced Code Point (DSCP) QoS setting that operates at layer 3. The transferred RDMA packets are marked with layer 3 DSCP, layer 2 Priority Code Points (PCP) or with both.
vSphere Networking Enable Jumbo Frames on a vSphere Standard Switch Enable jumbo frames for all traffic through a vSphere Standard Switch on a host. Procedure 1 In the vSphere Web Client, navigate to the host. 2 On the Configure tab, expand Networking and select Virtual switches. 3 Select a standard switch from the virtual switch table and click Edit settings. 4 In the Properties section, set the MTU property to a value greater than 1500 bytes. You can increase the MTU size up to 9000 bytes.
vSphere Networking 3 Click Edit and select the Virtual Hardware tab in the dialog box displaying the settings. 4 Expand the Network adapter section. Record the network settings and MAC address that the network adapter is using. 5 Click Remove to remove the network adapter from the virtual machine. 6 From the New device drop-down menu, select Network and click Add. 7 From the Adapter Type drop-down menu, select VMXNET 2 (Enhanced) or VMXNET 3.
vSphere Networking Procedure u Run these esxcli network nic software set console commands to enable or disable the software simulation of TSO in the VMkernel. n Enable the software simulation of TSO in the VMkernel. esxcli network nic software set --ipv4tso=1 -n vmnicX esxcli network nic software set --ipv6tso=1 -n vmnicX n Disable the software simulation of TSO in the VMkernel.
vSphere Networking 5 Click OK to apply the changes. 6 To reload the driver module of the physical adapter, run the esxcli system module set console command in the ESXi Shell on the host. a To disable the driver, run the esxcli system module set command with the --enabled false option. esxcli b system module set --enabled false --module nic_driver_module To enable the driver, run the esxcli system module set command with the --enabled true option.
vSphere Networking Procedure u In a terminal window on the Linux guest operating system, to enable or disable TSO, run the ethtool command with the -K and tso options. n To enable TSO, run the following command: ethtool -K ethY tso on n To disable TSO, run the following command: ethtool -K ethY tso off where Y in ethY is the sequence number of the NIC in the virtual machine.
vSphere Networking LRO reassembles incoming network packets into larger buffers and transfers the resulting larger but fewer packets to the network stack of the host or virtual machine. The CPU has to process fewer packets than when LRO is disabled, which reduces its utilization for networking especially in the case of connections that have high bandwidth. To benefit from the performance improvement of LRO, enable LRO along the data path on an ESXi host including VMkernel and guest operating system.
vSphere Networking 4 5 Edit the value of the Net.Vmxnet3SwLRO parameter for VMXNET3 adapters. n To enable software LRO, set Net.Vmxnet3SwLRO to 1. n To disable software LRO, set Net.Vmxnet3SwLRO to 0. Click OK to apply the changes. Determine Whether LRO Is Enabled for VMXNET3 Adapters on an ESXi Host Examine the status of LRO on an ESXi when you estimate the networking performance on a host that runs latency-sensitive workloads.
vSphere Networking Procedure 1 In the vSphere Web Client, navigate to the host. 2 On the Configure tab, expand System. 3 Click Advanced System Settings. 4 Edit the value of the Net.TcpipDefLROEnabled parameter. n To enable LRO for the VMkernel network adapters on the host, set Net.TcpipDefLROEnabled to 1. n 5 To disable software LRO for the VMkernel network adapters on the host, set Net.TcpipDefLROEnabled to 0. Click OK to apply the changes.
vSphere Networking Procedure u In a terminal window on the Linux guest operating system, run the ethtool command with the -K and lro options. n To enable LRO, run the following command: ethtool -K ethY lro on where Y in ethY is the sequence number of the NIC in the virtual machine. n To disable LRO, run the following command: ethtool -K ethY lro off where Y in ethY is the sequence number of the NIC in the virtual machine.
vSphere Networking Enable LRO Globally on a Windows Virtual Machine To use LRO on a VMXNET3 adapter on a virtual machine that runs Windows 8 and later or Windows Server 2012 and later, you must enable LRO globally on the guest operating system. On Windows, the LRO technology is also referred to as Receive Side Coalescing (RSC).
vSphere Networking You can enable or disable different types of Rx queues. For more information, see the esxcli network nic queue loadbalancer set command in the vSphere Command-Line Interface Reference documentation. Enable NetQueue on a Host NetQueue is enabled by default. To use NetQueue after it has been disabled, you must reenable it.
vSphere Network I/O Control 11 Use vSphere Network I/O Control to allocate network bandwidth to business-critical applications and to resolve situations where several types of traffic compete for common resources. n About vSphere Network I/O Control Version 3 vSphere Network I/O Control version 3 introduces a mechanism to reserve bandwidth for system traffic based on the capacity of the physical adapters on a host.
vSphere Networking Models for Bandwidth Resource Reservation Network I/O Control version 3 supports separate models for resource management of system traffic related to infrastructure services, such as vSphere Fault Tolerance, and of virtual machines. The two traffic categories have different nature. System traffic is strictly associated with an ESXi host. The network traffic routes change when you migrate a virtual machine across the environment.
vSphere Networking You can use Network I/O Control on a distributed switch to configure bandwidth allocation for the traffic that is related to the main vSphere features: n Management n Fault Tolerance n NFS n vSAN n vMotion n vSphere Replication n vSphere Data Protection Backup n Virtual machine vCenter Server propagates the allocation from the distributed switch to each physical adapter on the hosts that are connected to the switch.
vSphere Networking Table 11‑1. Allocation Parameters for System Traffic Parameter for Bandwidth Allocation Description Shares Shares, from 1 to 100, reflect the relative priority of a system traffic type against the other system traffic types that are active on the same physical adapter. The amount of bandwidth available to a system traffic type is determined by its relative shares and by the amount of data that the other system features are transmitting.
vSphere Networking Prerequisites n Verify that vSphere Distributed Switch is version 6.0.0 and later. n Verify that Network I/O Control on the switch is version 3. n Verify that Network I/O Control is enabled. See Enable Network I/O Control on a vSphere Distributed Switch. Procedure 1 In the vSphere Web Client, navigate to the distributed switch. 2 On the Configure tab, expand Resource Allocation. 3 Click System Traffic. You see the bandwidth allocation for the types of system traffic.
vSphere Networking About Allocating Bandwidth for Virtual Machines Network I/O Control allocates bandwidth for virtual machines by using two models: allocation across the entire vSphere Distributed Switch based on network resource pools and allocation on the physical adapter that carries the traffic of a virtual machine.
vSphere Networking Defining Bandwidth Requirements for a Virtual Machine You allocate bandwidth to an individual virtual machine similarly to allocating CPU and memory resources. Network I/O Control version 3 provisions bandwidth to a virtual machine according to shares, reservation, and limits that are defined for a network adapter in the VM hardware settings. The reservation represents a guarantee that the traffic from the virtual machine can consume at least the specified bandwidth.
vSphere Networking Bandwidth Allocation Parameters for Virtual Machine Traffic Network I/O Control version 3 allocates bandwidth to individual virtual machines based on configured shares, reservation, and limit for the network adapters in the VM hardware settings. Table 11‑2.
vSphere Networking Bandwidth Admission Control in vSphere DRS If you power on a virtual machine that is in a cluster, vSphere DRS places the virtual machine on a host that has the capacity to guarantee the bandwidth reserved for the virtual machine according to the active teaming policy. vSphere DRS migrates a virtual machine to another host to satisfy the bandwidth reservation of the virtual machine in these situations: n The reservation is changed to a value that the initial host can no longer satisfy.
vSphere Networking n Verify that Network I/O Control is enabled. See Enable Network I/O Control on a vSphere Distributed Switch. n Verify that the virtual machine system traffic has a configured bandwidth reservation. See Configure Bandwidth Allocation for System Traffic. Procedure 1 In the vSphere Web Client, navigate to the distributed switch. 2 On the Configure tab, expand Resource Allocation. 3 Click Network resource pools. 4 Click the Add icon.
vSphere Networking Network I/O Control allocates bandwidth to the virtual machines associated with the distributed port group according to the model implemented in the Network I/O Control version that is active on the distributed switch. See About vSphere Network I/O Control Version 3. Prerequisites n Verify that Network I/O Control is enabled. See Enable Network I/O Control on a vSphere Distributed Switch. Procedure 1 Locate a distributed port group in the vSphere Web Client.
vSphere Networking 5 If you want to configure bandwidth allocation for a new VM network adapter, from the New device drop-down menu select Network and click Add. A New Network section displays options for bandwidth allocation and other network adapter settings. 6 If the VM network adapter is not connected to the distributed port group, select the port group from the drop-down menu next to the Network adapter X or New Network label.
vSphere Networking Procedure 1 In the vSphere Web Client, navigate to the distributed switch. 2 On the Configure tab, expand Resource Allocation. 3 Click Network resource pools. 4 Select a network resource pool. 5 Click Virtual Machines. A list of the VM network adapters that are connected to the selected network resource pool appears. 6 Select the VM network adapters whose settings you want to configure and click Edit.
vSphere Networking 5 In the Reservation quota text box, enter the bandwidth quota for virtual machines from the aggregation of free bandwidth that is reserved for virtual machine system traffic on all physical adapters on the switch. 6 Click OK.
vSphere Networking For example, if the bandwidth allocation on a vSphere Distributed Switch is tailored on top of 10 GbE NICs, you might not be able to add a 1GbE NIC to the switch because it cannot meet the higher allocation requirements configured on the 10GbE NICs. Prerequisites n Verify that the host is running ESXi 6.0 and later. n Verify that vSphere Distributed Switch is version 6.0.0 and later. n Verify that Network I/O Control on the switch is version 3.
MAC Address Management 12 MAC addresses are used in the Layer 2 (Data Link Layer) of the network protocol stack to transmit frames to a recipient. In vSphere, vCenter Server generates MAC addresses for virtual machine adapters and VMkernel adapters, or you can assign addresses manually. Each network adapter manufacturer is assigned a unique three-byte prefix called an Organizationally Unique Identifier (OUI), which it can use to generate unique MAC addresses.
vSphere Networking n Range-based allocation After the MAC address is generated, it does not change unless the virtual machine's MAC address conflicts with that of another registered virtual machine. The MAC address is saved in the configuration file of the virtual machine. Note If you use invalid prefix- or range-based allocation values, an error is logged in the vpxd.log file. vCenter Server does not allocate MAC addresses when provisioning a virtual machine.
vSphere Networking Prefix-based MAC address allocation overcomes the limits of the default VMware allocation to provide unique addresses in larger scale deployments. Introducing an LAA prefix leads to a very large MAC address space (2 to the power of 46) instead of an universally unique address OUI which can give only 16 million MAC addresses. Verify that the prefixes that you provide for different vCenter Server instances in the same network are unique.
vSphere Networking Procedure 1 In the vSphere Web Client, navigate to a vCenter Server instance. 2 On the Configure tab, expand Settings and select Advanced Settings. 3 Click Edit. 4 Add or edit parameters for the target allocation type. Use only one allocation type. n Change to prefix-based allocation. Key Example Value config.vpxd.macAllocScheme.prefixScheme.prefix 005026 config.vpxd.macAllocScheme.prefixScheme.
vSphere Networking 2 Open the vpxd.cfg file. 3 Decide on an allocation type to use and enter the corresponding XML code in the file to configure the allocation type. The following are examples of XML code to use. Note Use only one allocation type.
vSphere Networking n The virtual machine configuration file does not contain the MAC address and information about the MAC address allocation type. MAC Address Format The host generates MAC addresses that consists of the VMware OUI 00:0C:29 and the last three octets in hexadecimal format of the virtual machine UUID. The virtual machine UUID is based on a hash calculated by using the UUID of the ESXi physical machine and the path to the configuration file (.vmx) of the virtual machine.
vSphere Networking You can set a static MAC address that contains the VMware OUI prefix in compliance with the following format: 00:50:56:XX:YY:ZZ where XX is a valid hexadecimal number between 00 and 3F, and YY and ZZ are valid hexadecimal numbers between 00 and FF. To avoid conflict with MAC addresses that are generated by vCenter Server or are assigned to VMkernel adapters for infrastructure traffic, the value for XX must not be greater than 3F.
vSphere Networking Procedure 1 Locate the virtual machine in the vSphere Web Client. a Select a data center, folder, cluster, resource pool, or host and click the VMs tab. b Click Virtual Machines and double-click the virtual machine from the list. 2 Power off the virtual machine. 3 On the Configure tab of the virtual machine, expand Settings and select VM Options. 4 Click Edit and expand Advanced from the VM Options tab within the dialog box displaying the settings.
Configuring vSphere for IPv6 13 Configure ESXi hosts and vCenter Server for operation in a pure IPv6 environment for larger address space and improved address assignment. IPv6 is designated by the Internet Engineering Task Force (IETF) as the successor to IPv4 providing the following benefits: n Increased address length. The increased address space resolves the problem of address exhaustion and eliminates the need for network address translation.
vSphere Networking Table 13‑1.
vSphere Networking IPv6 Connectivity of Virtual Machines Virtual machines can exchange data in the network over IPv6. vSphere supports both static and automatic assignment of IPv6 addresses for virtual machines. Configuring one or more IPv6 addresses is also possible when you customize the guest operating system of a virtual machine. FQDNs and IPv6 Addresses In vSphere, you should use fully qualified domain names (FQDNs) that are mapped to IPv6 addresses on the DNS server.
vSphere Networking n Verify that the hosts have ESXi 6.5 installed. See the vCenter Server Installation and Setup documentation. Procedure 1 In the Direct Console User Interface (DCUI), configure each ESXi host as a pure IPv6 node. a In the DCUI, press F2 and log in to the host. b From the Configure Management Network menu, select IPv6 Configuration and press Enter. c Assign an IPv6 address to the host.
vSphere Networking 2 Configure each ESXi host as a pure IPv6 node. a Open an SSH connection and log in to the ESXi host. b Run the following command: esxcli network ip interface ipv6 set -i vmk0 -e true c Assign an IPv6 address to the management network. Address Assignment Option Description Static address assignment 1 Open an SSH connection and log in to the ESXi host.
vSphere Networking 3 Disable IPv4 configuration for management network a Open an SSH connection and log in to the ESXi host. b Run the following command: esxcli network ip interface ipv4 set -i vmk0 --type=none 4 If vCenter Server uses an external database, configure the database as an IPv6 node. 5 Configure vCenter Server as a pure IPv6 node and restart it. 6 Disable IPv4 on the database server. 7 In the vSphere Web Client, add the hosts to the inventory.
vSphere Networking 2 On the Configure tab, expand Networking and select VMkernel adapters. 3 Select the VMkernel adapter on the target distributed or standard switch and click Edit. 4 In the Edit Settings dialog box, click IPv6 settings. 5 Configure the address assignment of the VMkernel adapter. IPv6 Address Option Description Obtain IPv6 address automatically through DHCP Receive an IPv6 address for the VMkernel adapter from a DHCPv6 server.
vSphere Networking 6 Edit the IPv6 settings. Option Description Obtain IPv6 settings automatically through DHCP Assigns IPv6 addresses to the appliance automatically from the network by using DHCP. Obtain IPv6 settings automatically through Router Advertisement Assigns IPv6 addresses to the appliance automatically from the network by using router advertisement. Static IPv6 addresses Uses static IPv6 addresses that you set up manually. 1 Click the Add icon.
Monitoring Network Connection and Traffic 14 Monitor network connection and packets that pass through the ports of a vSphere Standard Switch or a vSphere Distributed Switch to analyze the traffic between virtual machines and hosts.
vSphere Networking c Make the following changes. true d (Optional) Configure PacketCapture options. Option and Default Value Description 72 On startup delete all pcap and pcap.gz files that were last modified before the specified period of hours and are not part of the current process. /directory_path The directory in which pcap and pcap.gz files are stored. The directory must exist and be accessible.
vSphere Networking What to do next Copy the pcap and pcap.gz files to a system that runs a graphical analyzer tool, such as Wireshark, and examine the packet details. Capturing and Tracing Network Packets by Using the pktcap-uw Utility Monitor the traffic that flows through physical network adapters, VMkernel adapters, and virtual machines adapters, and analyze packet information by using the graphical user interface of network analysis tools such as Wireshark.
vSphere Networking Table 14‑1. pktcap-uw Arguments for Capturing Packets Argument Group Argument Description switch_port_arguments --uplink vmnicX Capture packets that are related to a physical adapter. You can combine the --uplink and --capture options for monitoring packets at a certain place in the path between the physical adapter and the virtual switch. See Capture Packets That Arrive at a Physical Adapter. --vmk vmkX Capture packets that are related to a VMKernel adapter.
vSphere Networking Table 14‑1. pktcap-uw Arguments for Capturing Packets (Continued) Argument Group Argument Description --dir {0|1} Capture packets according to the direction of the flow with regard to the virtual switch. 0 stands for incoming traffic and 1 for outgoing traffic. By default, the pktcap-uw utility captures ingress traffic. Use the --dir option together with the --uplink, --vmk, or --switchport option. --stage {0|1} Capture the packet closer to its source or to its destination.
vSphere Networking pktcap-uw Syntax for Tracing Packets The command of the pktcap-uw utility has the following syntax for tracing packets in the network stack: pktcap-uw --trace filter_options output_control_options Options to the pktcap-uw Utility for Tracing Packets The pktcap-uw utility supports the following options when you use it to trace packets: Table 14‑2.
vSphere Networking Table 14‑3. Options for Output Control That Are Supported by the pktcap-uw Utility (Continued) Option Description {-s | --snaplen} snapshot_length Capture only the first snapshot_length bytes from each packet. If traffic on the host is intensive, use this option to reduce the load on the CPU and storage. To limit the size of captured contents, set a value greater than 24. To capture the complete packet, set this option to 0. -h View help about the pktcap-uw utility.
vSphere Networking Table 14‑4. Filter Options of the pktcap-uw Utility (Continued) Option Description --ip IP_addess Capture or trace packets that have a specific source or destination IPv4 address. --proto 0xIP_protocol_number Capture or trace packets at Layer 3 according to the next level protocol that consumes the payload. For example, to monitor traffic for the UDP protocol, type --proto 0x11. --srcport source_port Capture or trace packets according to their source TCP port.
vSphere Networking 2 In the ESXi Shell to the host, run the pktcap-uw command with the --uplink vmnicX argument and with options to monitor packets at a particular point, filter captured packets and save the result to a file.
vSphere Networking Example: Capture Packets That Are Received at vmnic0 from an IP Address 192.168.25.113 To capture the first 60 packets from a source system that is assigned the IP address 192.168.25.113 at vmnic0 and save them to a file called vmnic0_rcv_srcip.pcap, run the following pktcap-uw command: pktcap-uw --uplink vmnic0 --capture UplinkRcv --srcip 192.168.25.113 --outfile vmnic0_rcv_srcip.
vSphere Networking where the square brackets [] enclose the options of the pktcap-uw --switchport port_ID command and the vertical bars | represent alternative values. If you run the pktcap-uw --switchport port_ID command without options, you obtain the content of packets that are incoming to the standard or distributed switch in the console output at the point when they are switched.
vSphere Networking What to do next If the contents of the packet are saved to a file, copy the file from the ESXi host to the system that runs a graphical analyzer tool, such as Wireshark, and open it in the tool to examine the packet details. Capture Packets for a VMkernel Adapter Monitor packets that are exchanged between a VMkernel adapter and a virtual switch by using the pktcap-uw utility. You can capture packets at a certain capture point in the flow between a virtual switch and a VMkernel adapter.
vSphere Networking You can replace the --vmk vmkX option with --switchport vmkernel_adapter_port_ID, where vmkernel_adapter_port_ID is the PORT-ID value that the network panel of the esxtop utility displays for the adapter. If you run the pktcap-uw --vmk vmkX command without options, you obtain the content of packets that are leaving the VMkernel adapter.
vSphere Networking Procedure 1 In the ESXi Shell to the host, run the pktcap-uw --capture Drop command with options to monitor packets at a particular point, filter captured packets and save the result to a file. pktcap-uw --capture Drop [filter_options] [--outfile pcap_file_path [--ng]] [--count number_of_packets] where the square brackets [] enclose the options of the pktcap-uw --capture Drop command and the vertical bars | represent alternative values.
vSphere Networking Procedure 1 (Optional) To find the name of the DVFilter that you want to monitor, in the ESXi Shell, run the summarize-dvfilter command. The output of the command contains the fast-path and slow-path agents of the DVFilters that are deployed on the host. 2 Run the pktcap-uw utility with the --dvfilter dvfilter_name argument and with options to monitor packets at a particular point, filter captured packets and save the result to a file.
vSphere Networking Using the Capture Points of the pktcap-uw Utility You use the capture points of the pktcap-uw utility to monitor packets when a function handles them at a specific place in the network stack on a host. Overview of Capture Points A capture point in the pktcap-uw utility represents a place in the path between a virtual switch on one side and a physical adapter, VMkernel adapter or a virtual machine adapter on the other.
vSphere Networking Capture Points That Are Relevant to Virtual Machine Traffic The pktcap-uw --switchport vmxnet3_port_ID command supports capture points for functions that handle traffic packets at a specific place and direction in the path between a VMXNET3 adapter and a virtual switch. Capture Point Description Vmxnet3Rx The function in the VMXNET3 backend that receives packets from the virtual switch.
vSphere Networking Capture Point Description VdrRxTerminal Capture packets at the receive terminal I/O chain of a dynamic router in VMware NSX. Use this capture point together with the --lifID option. VdrTxLeaf Capture packets at the transmit leaf I/O chain of a dynamic router in VMware NSX. Use this capture point together with the --lifID option. VdrTxTerminal Capture packets at the transmit terminal I/O chain of a dynamic router in VMware NSX.
vSphere Networking Procedure 1 In the ESXi Shell to the host, run the pktcap-uw --trace command with options to filter traced packets, save the result to a file and limit the number of traced packets. pktcap-uw --trace [filter_options] [--outfile pcap_file_path [--ng]] [--count number_of_packets] where the square brackets [] enclose optional items of the pktcap-uw --trace command and the vertical bars | represent alternative values.
vSphere Networking 3 Type the Collector IP address and Collector port of the NetFlow collector. You can contact the NetFlow collector by IPv4 or IPv6 address. 4 Set an Observation Domain ID that identifies the information related to the switch. 5 To see the information from the distributed switch in the NetFlow collector under a single network device instead of under a separate device for each host on the switch, type an IPv4 address in the Switch IP address text box.
vSphere Networking Table 14‑5. vMotion Interoperability with port mirroring Port mirroring session type Source and destination Interoperable with vMotion Functionality Distributed Port Mirroring Non-uplink distributed port source and destination Yes Port mirroring between distributed ports can only be local. If the source and destination are on different hosts due to vMotion, mirroring between them will not work. However, if the source and destination move to the same host, port mirroring works.
vSphere Networking TSO and LRO TCP Segmentation Offload (TSO) and large receive offload (LRO) might cause the number of mirroring packets to not equal to the number of mirrored packets. When TSO is enabled on a vNIC, the vNIC might send a large packet to a distributed switch. When LRO is enabled on a vNIC, small packets sent to it might be merged into a large packet.
vSphere Networking 2 Click the Configure tab and expand Settings. 3 Select the Port mirroring option and click New. 4 Select the session type for the port mirroring session. 5 Option Description Distributed Port Mirroring Mirror packets from a number of distributed ports to other distributed ports on the same host . If the source and the destination are on different hosts, this session type does not function.
vSphere Networking Select Port Mirroring Sources To continue creating a port mirroring session, select sources and traffic direction for the new port mirroring session. You can create a port mirroring session without setting the source and destinations. When a source and destination are not set, a port mirroring session is created without the mirroring path. This allows you to create a port mirroring session with the correct properties set.
vSphere Networking Procedure 1 Select the destination for the port mirroring session. Depending on which type of session you chose, different options are available. Option Description Select a destination distributed port Click Select distributed ports to select ports from a list, or click Add distributed ports to add ports by port number. You can add more than one distributed port.
vSphere Networking Procedure 1 In the vSphere Web Client, navigate to the distributed switch. 2 On the Configure tab, expand Settings and click Port mirroring. 3 Select a port mirroring session from the list and click Edit. 4 On the Properties page, edit the session properties. Depending on the type of port mirroring session being edited, different options are available for configuration.
vSphere Networking 6 In the Destinations section, edit the destinations for the port mirroring session. Depending on the type of port mirroring session being edited, different options are available for configuration. Option Description Select a destination distributed port Click the Select distributed ports… button to select ports from a list, or click the Add distributed ports… button to add ports by port number. You can add more than one distributed port.
vSphere Networking Required Configuration on the Distributed Switch Configuration Error Health Check The VLAN trunk ranges configured on the distributed switch do not match the trunk ranges on the physical switch. Checks whether the VLAN settings on the distributed switch match the trunk port configuration on the connected physical switch ports. At least two active physical NICs The MTU settings on the physical network adapters, distributed switch, and physical switch ports do not match.
vSphere Networking Prerequisites Verify that health check for VLAN and MTU, and for teaming policy is enabled on the vSphere Distributed Switch. See Enable or Disable vSphere Distributed Switch Health Check. Procedure 1 In the vSphere Web Client, navigate to the distributed switch. 2 On the Monitor tab, click Health. 3 In the Health Status Details section, examine the overall, VLAN, MTU and teaming health of the hosts connected to the switch.
vSphere Networking 5 6 From the Operation drop-down menu, select the operational mode of the ESXi hosts connected to the switch. Option Description Listen ESXi detects and displays information about the associated Cisco switch port, but information about the vSphere Distributed Switch is not available to the Cisco switch administrator.
vSphere Networking View Switch Information When Cisco Discovery Protocol (CDP) or Link Layer Discovery Protocol (LLDP) is enabled on the distributed switch and the hosts connected to the switch are in Listen or Both operational mode, you can view physical switch information from the vSphere Web Client. Procedure 1 In the vSphere Web Client, navigate to the host. 2 On the Configure tab, expand Networking and click Physical adapters.
Configuring Protocol Profiles for Virtual Machine Networking 15 A network protocol profile contains a pool of IPv4 and IPv6 addresses that vCenter Server assigns to vApps or to virtual machines with vApp functionality that are connected to port groups associated with the profile. Network protocol profiles also contain settings for the IP subnet, DNS, and HTTP proxy server.
vSphere Networking Add a Network Protocol Profile A network protocol profile contains a pool of IPv4 and IPv6 addresses. vCenter Server assigns those resources to vApps or to virtual machines with vApp functionality that are connected to port groups associated with the profile. Network protocol profiles also contain settings for the IP subnet, DNS, and HTTP proxy server.
vSphere Networking 4 Select the Enable IP Pool check box to specify an IP pool range. 5 If you enable IP Pools, enter a comma-separated list of host address ranges in the IP pool range field. A range consists of an IP address, a pound sign (#), and a number indicating the length of the range. The gateway and the ranges must be within the subnet. The ranges that you enter in the IP pool range field cannot include the gateway address. For example, 10.20.60.4#10, 10.20.61.
vSphere Networking Specify Network Protocol Profile DNS and Other Configuration When you create a network protocol profile, you can specify the DNS domain, DNS search path, a host prefix, and HTTP proxy. Procedure 1 Enter the DNS domain. 2 Enter the host prefix. 3 Enter the DNS search path. The search paths are specified as a list of DNS domains separated by commas, semi-colons, or spaces. 4 Enter the server name and port number for the proxy server.
vSphere Networking 4 On the Set association type page of the Associate Network Protocol Profile wizard, select Use an existing network protocol profile and click Next. If the existing network protocol profiles do not contain settings suitable for the vApp virtual machines in the port group, you must create a new profile. 5 Select the network protocol profile and click Next. 6 Examine the association and settings of the network protocol profile, and click Finish.
Multicast Filtering 16 In vSphere 6.0 and later, vSphere Distributed Switch supports basic and snooping models for filtering of multicast packets that are related to individual multicast groups. Choose a model according to the number of multicast groups to which the virtual machines on the switch subscribe. n Multicast Filtering Modes In addition to the default basic mode for filtering multicast traffic, vSphere Distributed Switch 6.0.
vSphere Networking Basic Multicast Filtering In basic multicast filtering mode, a vSphere Standard Switch or vSphere Distributed Switch forwards multicast traffic for virtual machines according to the destination MAC address of the multicast group. When joining a multicast group, the guest operating system pushes the multicast MAC address of the group down to the network through the switch. The switch saves the mapping between the port and the destination multicast MAC address in a local forwarding table.
vSphere Networking Use multicast snooping if virtualized workloads on the switch subscribe to more than 32 multicast groups or must receive traffic from specific source nodes. For information about the multicast filtering modes of vSphere Distributed Switch, see Multicast Filtering Modes. Prerequisites Verify that vSphere Distributed Switch is version 6.0.0 and later. Procedure 1 In the vSphere Web Client, navigate to the distributed switch. 2 From the Actions menu, select Settings > Edit Settings.
vSphere Networking 4 Click Edit and enter a new value between 1 and 32 for the setting. 5 Click OK. VMware, Inc.
Stateless Network Deployment 17 Stateless is a mode of execution for ESXi hosts with no local storage that formerly would save configuration or state. Configurations are abstracted into a host profile, which is a template that applies to a class of machines. Stateless allows easy replacement, removal, and addition of failed hardware, and improves the ease of scaling a hardware deployment. Every stateless ESXi boot is like a first boot.
vSphere Networking n Creates all standard switch instances, along with port groups. It selects uplinks based on policy. If the policy is based on the VLAN ID, there is a probing process to gather relevant information. n For VMkernel network adapters connected to the standard switch, it creates VMkernel network adapters and connects them to port groups.
Networking Best Practices 18 Consider these best practices when you configure your network. n To ensure a stable connection between vCenter Server, ESXi, and other products and services, do not set connection limits and timeouts between the products. Setting limits and timeouts can affect the packet flow and cause services interruption. n Isolate from one another the networks for host management, vSphere vMotion, vSphere FT, and so on, to improve security and performance.
vSphere Networking n Physical network adapters connected to the same vSphere Standard Switch or vSphere Distributed Switch should also be connected to the same physical network. n Configure the same MTU on all VMkernel network adapters in a vSphere Distributed Switch. If several VMkernel network adapters, configured with different MTUs, are connected to vSphere distributed switches, you might experience network connectivity problems. VMware, Inc.
Troubleshooting Networking 19 The troubleshooting topics about networking in vSphere provide solutions to potential problems that you might encounter with the connectivity of ESXi hosts, vCenter Server and virtual machines.
vSphere Networking Guidelines for Troubleshooting To troubleshoot your implementation of vSphere, identify the symptoms of the problem, determine which of the components are affected, and test possible solutions. Identifying Symptoms A number of potential causes might lead to the under-performance or nonperformance of your implementation. The first step in efficient troubleshooting is to identify exactly what is going wrong.
vSphere Networking Recognizing the characteristics of the software and hardware elements and how they can impact the problem, you can explore general problems that might be causing the symptoms. n Misconfiguration of software settings n Failure of physical hardware n Incompatibility of components Break down the process and consider each piece and the likelihood of its involvement separately.
vSphere Networking Common Logs The following logs are common to all deployments on Windows or Linux. Table 19‑1.
vSphere Networking Table 19‑2. Management Node Log Directories (Continued) Log Directory Description vsphere-client VMware vSphere Web Client vcha VMware High Availability Service (Linux only) Platform Services Controller Logs You can examine the following logs if a Platform Services Controller node deployment is chosen. Table 19‑3.
vSphere Networking n A virtual machine has been transferred in power-off state from one vCenter Server instance to another in the same network, for example, by using shared storage, and a new virtual machine network adapter on the first vCenter Server receives the freed MAC address. Solution n Change the MAC address of a virtual machine network adapter manually. If you have an existing virtual machine with a conflicting MAC address, you must provide a unique MAC address in the Virtual Hardware settings.
vSphere Networking n If the vCenter Server instance generates the MAC addresses of virtual machines according to the default allocation, VMware OUI, change the vCenter Server instance ID or use another allocation method to resolve conflicts. Note Changing the vCenter Server instance ID or switching to a different allocation scheme does not resolve MAC address conflicts in existing virtual machines.
vSphere Networking n Enforce MAC address regeneration when transferring a virtual machine between vCenter Server instances by using the virtual machine files from a datastore. a Power off a virtual machine, remove it from the inventory, and in its configuration file (.vmx), set the ethernetX.addressType parameter to generated. X next to ethernet stands for the sequence number of the virtual NIC in the virtual machine.
vSphere Networking Problem In the vSphere Web Client, after you assign a MAC address within the range 00:50:56:40:YY:ZZ – 00:50:56:7F:YY:ZZ to a virtual machine, attempts to power the virtual machine on fail with a status message that the MAC address is in conflict. 00:50:56:XX:YY:ZZ is not a valid static Ethernet address. It conflicts with VMware reserved MACs for other usage.
vSphere Networking Solution Problem Solution Cannot remove a host from a distributed switch 1 In the vSphere Web Client, navigate to the distributed switch. 2 On the Configure tab, select More > Ports. 3 Locate all ports that are still in use and check which VMkernel or virtual machine network adapters on the host are still attached to the ports. 4 Migrate or delete the VMkernel and virtual machine network adapters that are still connected to the switch.
vSphere Networking After the host re-connects to vCenter Server, the vSphere Web Client displays a warning that some hosts on the switch have different networking configuration from the configuration stored in vSphere distributed switch. 2 In the vSphere Web Client, configure the distributed port group for the management network with correct settings. Situation Solution You have altered the port group configuration only once You can roll the configuration of the port group back one step.
vSphere Networking Problem After you change the networking configuration of a port group on a vSphere Distributed Switch 5.0 or earlier that contains the VMkernel adapters for the management network, the hosts on the switch lose connectivity to vCenter Server. In the vSphere Web Client the status of the hosts is nonresponsive. Cause On a vSphere Distributed Switch 5.
vSphere Networking 5 In the vSphere Web Client, configure the distributed port group for the management network with correct settings. 6 Migrate the VMkernel adapter for the management network from the standard switch to a port on the distributed switch by using the Add and Manage Hosts wizard. For information about the Add and Manage Hosts wizard, see the vSphere Networking documentation.
vSphere Networking Problem After you rearrange the uplinks in the failover groups for a distributed port group in vCenter Server, for example, by using the vSphere Web Client, some virtual machines in the port group can no longer access the external network. Cause After changing the failover order, many reasons might cause virtual machines to lose connectivity to the external network.
vSphere Networking b Select Virtual Switches and select the distributed proxy switch. c Click Manage the physical network adapters connected to the selected switch, and move the NIC to the active uplink Unable to Add a Physical Adapter to a vSphere Distributed Switch That Has Network I/O Control Enabled You might be unable to add a physical adapter with low speed, for example, 1 Gbps, to a vSphere Distributed Switch that has vSphere Network I/O Control version 3 configured.
vSphere Networking SR-IOV Enabled Workload Cannot Communicate After You Change Its MAC Address After you change the MAC address in the guest operating system of an SR-IOV enabled virtual machine, the virtual machine loses connectivity. Problem When you connect the network adapter of a virtual machine to an SR-IOV virtual function (VF), you create a passthrough network adapter for the virtual machine.
vSphere Networking A Virtual Machine that Runs a VPN Client Causes Denial of Service for Virtual Machines on the Host or Across a vSphere HA Cluster A virtual machine sending Bridge Protocol Data Unit (BPDU) frames, for example, a VPN client, causes some virtual machines connected to the same port group to lose connectivity. The transmission of BPDU frames might also break the connection of the host or of the parent vSphere HA cluster.
vSphere Networking Solution n If the VPN software must continue its work on the virtual machine, allow the traffic out of the virtual machine and configure the physical switch port individually to pass the BPDU frames. Network Device Configuration Distributed or standard switch Set the Forged Transmit security property on the port group to Accept to allow BPDU frames to leave the host and reach the physical switch port.
vSphere Networking Low Throughput for UDP Workloads on Windows Virtual Machines When a Windows virtual machine in vSphere transmits large UDP packets, the throughput is lower than expected or is oscillating even when other traffic is negligible. Problem When a Windows virtual machine transmits UDP packets larger than 1024 bytes, you experience lower than expected or oscillating throughput even when other traffic is negligible. In case of a video streaming server, video playback pauses.
vSphere Networking X next to ethernet stands for the sequence number of the vNIC in the virtual machine. For more information about configuring parameters in the .vmx file, see the vSphere Virtual Machine Administrationdocumentation. n Modify ESXi host coalescing settings. This approach affects all virtual machines and all virtual machine NICs on the host.
vSphere Networking n The physical NICs on the hosts that are assigned to the active or standby uplinks reside in different VLANs on the physical switch. The physical NICs in different VLANs cannot see each other and thus cannot communicate with each other. Solution n In the topology of the distributed switch, check which host does not have physical NICs assigned to an active or standby uplink on the distributed port group.
vSphere Networking n A port group that has the same name and is associated with another protocol profile might already exist in the target data center, and vApps and virtual machines might be connected to this group. Replacing the protocol profiles for the port group might affect the connectivity of these vApp and virtual machines.
vSphere Networking Solution n Use the vSphere Web Client to increase the timeout for rollback on vCenter Server. If you encounter the same problem again, increase the rollback timeout with 60 seconds incrementally until the operation has enough time to succeed. a On the Configure tab of a vCenter Server instance, expand Settings. b Select Advanced Settings and click Edit. c If the property is not present, add the config.vpxd.network.rollbackTimeout parameter to the settings.
