Objective 2 – Configure and Administer Advanced vSphere Networking

Objective 2.1: Configure Advanced Policies/Features and Verify Network Virtualization Implementation

Knowledge

  • Explain the behavior of a virtual Distributed Switch (vDS) Auto-Rollback
    • Enabled by default after 5.1, can disable/enable thought
      • vcenter advanced settings -> config.vpxd.network.rollback. Restart vcenter to apply the change.
      • If automatic rollback is disabled and a configuration change causes a loss of connectivity, can rollback the vDS via the DCUI.
        • Network restore options -> restore vDS. Config & apply.
        • Creates a local ephemeral port and moves the VMK associated with the management net. Now you can reconnect the host to vCenter, fix the bad config and migrate the VMK back to the vDS.
    • Strictly for management ports
      • Can backup your vDS, then use the back to restore switch or port group from a backup. Can also create a new switch or port group from the backup. Can revert to previous port group configuration after changes are made.
    • By default rollback is enabled. vDS and esxi server monitor connection to vcenter.
    • Rollback provides the following options to recover from management network misconfigurations.
      • Automatic rollback if misconfiguration is detected.
      • DCUI to recover management network
    • Two types of rollback
      • Distributed switch-level rollback occurs after the user updates distributed switch related objects such as port groups or distributed ports
      • Host-level rollback triggered when there is a change to the host networking configurations such as physical nic speed change, mtu or IP settings
  • Determine and configure appropriate port group PVLAN settings for VMs given communication requirements
  • Determine physical network connectivity requirements between ESXi Hosts to support vDS deployments
  • Configure LACP on vDS given design parameters
    • Up to 64 LAGs on a dVS, up to 32 LAG’s on a host. Number of ports in a LAG depend on capabilities of the physical switches port channel abilities
    • Number of ports on the LACP port channel must be equal to the number of physical NIC’s being grouped on a host. LAG on host must have at least 2 ports.
    • Hashing algorithm of the LACP port channel must match that of the LAG.
    • All physical nic’s must have same speed/duplex.
    • Load balancing, only a single LAG can be active at a time, no additional uplinks or LAG’s can be active. Standby LAG’s are not supported (only during traffic migration to the LAG to ensure connectivity).. All standalone or additional LAG’s must be set to unused.
    • Pre-req’s to upgrade dVS to Enhanced LACP support
      • Switch is v 5.5 or 6
      • No ports have an override for uplink teaming
      • Only one uplink port group on the dVS
      • dvPort group.Modify  &   Host.Configuration.Modify
    • Creating a LAG & migrating traffic to it
      • Create the LAG
      • Set the LAG as a standby uplink of dPortgroups
      • Reassign physical adapters to the LAG.
      • Set the LAG as active, and move all else to unused.
    • LACP limitations
      • iSCSI multipathing not supported
      • Host profiles not supported
      • Does not work with dump collector
      • Does not work with port mirroring
  • Configure vDS across multiple vCenter Servers to support Long Distance vMotion

Objective 2.2: Configure Network I/O Control (NIOC)

Knowledge

    • Configure NIOC shares/limits based on VM requirements
      • If you use Network I/O Control version 2 on a vSphere Distributed Switch, you can specifically associate a distributed port with a user-defined resource pool to use Network I/O Control for control over the bandwidth provided to the virtual machine connected to the port.
      • NIOC imposes a limit of 75% of the total uplink capacity. Ex. 2 x10Gb uplinks = 20GB uplink capacity. No reservation can be more than 15Gb (75%).
    • Explain the behavior of a given NIOC setting
      • Traffic filtering and marking policy replaces CoS tagging that was associated with a NIOC2 user-defined network resource pool.
        • Traffic filtering and monitoring is defined in the settings of a dPortGroup
      • Unused reservations return the unused portion to the pool, effectively meaning that NIOC reservations only come into play during resource contention.
      • VM network resource pools use a quota based system. Review page 167 of the network guide
        • Before using resource pools, you must have a reservation set for VM system traffic
        • When you create a reservation on a VM in a network resource pool, that VM reservation must not exceed that of the pool quota
    • Determine Network I/O Control requirements
      • NIOC is only available on distributed switches. Load balancing requires DRS to be enabled.
      • In vspher6 NIOC v2 & v3 can coexist. V2 configures bandwidth allocation for VM’s at the physical adapter, whereas v3 lets you setup bandwidth allocation for VM’s at the dVS.
      • Upgrading a dVS NIOC is also upgraded to v3 unless using features that are not available in NIOC3
      • NIOC2 uses network resource pools + COS tagging.
      • When upgrading to NIOC3, existing NIOC2 configurations are converted into constructs of shares/reservations/limits. This is disruptive activity. vDS must be at v6
      • For a VM to use NIOC, it must be on a dvSwitch
      • You can exclude a host’s nic from being included in the scope of NIOC
        • host->manage->settings->advanced-> use a comma separated list to add nic’s to the Net.IOControlPnicOptOut parameter.
    • Differentiate Network I/O Control capabilities
      • “System traffic is strictly associated with an ESXi host. The network traffic routes change when you migrate a virtual machine across the environment. To provide network resources to a virtual machine regardless of its host, in Network I/O Control you can configure resource allocation for virtual machines that is valid in the scope of the entire distributed switch.” – netowkring guide, page 160
      • shares/limits/reservations all exist as they do for other resources.
        • Shares: proportional distribution
        • Reservation: minimum bandwidth to guarantee
        • Limit: cap on the amount of resource
      • The capacity of the physical adapters determines the bandwidth that you can guarantee.
      • Bandwidth reservations for classes of traffic: Management, Fault Tolerance, iSCSI,  NFS, Virtual SAN, vMotion, vSphere Replication, vSphere Data Protection Backup, Virtual machine
        • VM get 100 shares by default, all else get 50.
      • Bandwidth reservations for VM’s. Can get down to a per VM granularity or create Network resources pools which can be associated w/ dvPortGroup
      • DRS accounts for bandwidth reservations in addition to CPU & Memory, for both power on (admission control) and load balancing.
      • NIOC can introduce additional CPU load due to how the network scheduler handles VM interrupts. This CPU overhead can be decreased by changing the VM interrupt interval, but may cause increased latency for small packets.
        • 1. Go to VM Settings  VM Options  Configuration Parameters  Edit Configuration.
        • 2. Add an entry for ethernetX.coalescingParams (where X is the interface number, for example, ethernet0). The default interrupt rate is 4000 per second.
      • “Handling Latency Sensitive Traffic – While NetIOC does a great job in limiting the increase in latency of a network port due to competing traffic from other network ports, there is still some increase due to sharing of resources. Experiment 2 demonstrated that the latency increase for a Web server is not more than 15% once NetIOC bandwidth reservations are in place. This latency increase should be in the tolerance zone for most applications. However, in certain situations, an application can be extremely latency sensitive. In such cases, you can limit the impact to latency from competing VMs by reducing the maximum time for which the uplink device can delay transmit completions. By default, the value of this parameter is set to 33 us. The parameter is Net.NetSchedCoalesceTxUsecs in the Advanced System Settings tab for the host. Note that changing this parameter value adjusts the corresponding physical NIC interrupt throttle rate. Lowering Net.NetSchedCoalesceTxUsecs leads to a higher interrupt throttle rate, and thereby higher CPU utilization. Lowering Net.NetSchedCoalesceTxUsecs below a certain value will also limit the NIC from reaching line rate throughput. Please remember to keep Net.NetSchedCoalesceTxUsecs consistent across all hosts in the cluster.” – page 12, Performance Evaluation of Network I/O Control in VMware vSphere 6
  • Enable/Disable Network I/O Control
    • SR-IOV is not available for VM’s configured to use NIOC v3
    • NIOC is enabled by default
    • To enable/disable. dVS->actions->edit settings. Select enabled/disabled from the Network I/O Control drop-down menu.
    • To create a network resource pool, networking->DVswitch->manage->resource allocation-> network resource pools. Add resource pool and specify reservation.
    • To manage system shares/reservations, networking->DVswitch->manage->resource allocation-> system traffic. Click the type of traffic and edit