HP Virtual Connect – “NO_COMM” state

These are my comments on HP Customer Advisory c02720395 where the HP Virtual Connect Manager (VCM) on Flex-10 Ethernet modules looses communication with the active On Board Administrator in HP c7000 blade chassis.

Symptoms

The first sign of trouble would be the red X for the domain status when you login to the VCM.  When you view the domain status or Flex-10 Ethernet Module you will see a message similar to this:

Resolution

This issue has been resolved in VCM firmware version 3.17 and higher.  The most recent firmware for Virtual Connect Flex-10 and Virtual Connect FC modules can be found here.

Workaround

This issue is directly related to the way the module queries DNS.  The workaround is to remove the DNS server settings for the affected I/O modules (Flex-10).  I have not seen this issue affect the Fibre Channel (FC) modules.

The instructions in the advisory are very detailed.  Generally, for VCM version 3.0 and higher you will simply remove the DNS server addresses for all Flex-10 I/O modules in the chassis.  You can access these settings by clicking on Enclosure Bay IP Addressing in the navigation pane of the OBA then selecting the Innerconnect Bays tab.  In this example they are in Bay 1 and 2:

If your current version of VCM is earlier than 3.0 you will also need to remove the DNS server settings for all of the iLO bays on the Device Bays tab.

This workaround can help in preventing the loss of network connectivity to the blades in the affected chassis however it is recommended that the OBA and VCM firmware be upgraded as soon as possible.

-James

Posted in Server Hardware by James. No Comments

PowerCLI for setting multipathing policies

The plan is that most of my posts here will be rooted in actual challenges, and this one is just that.  It became necessary to have a process in place that will guarantee the configuration of individual storage LUNs attached to ESX/ESXi hosts is the same for all host in the cluster.  Consistently has always been important but with the recent growth this environment has seen manual configuration is just simply not reliable or efficient.

Several different settings have been tested and in this environment we are best served by using the following settings:

  • LUNs hosted on active/active storage arrays we will use the VMware round-robin Path Selection Policy (PSP)
  • The round-robin PSP will switch paths after every 10 IOPS.

Previously these properties were being set with an esxcli command for each target device.  This is not efficient for me because the vMA still has to connect to each host individually.  I knew I had seen the PSP property values while viewing storage devices on hosts using the Get-ScsiLun cmdlet so I started here to explore all of the available properties.

1
2
$luns = Get-ScsiLun -LunType "disk"
$luns | Get-Member

Get-ScsiLun Get-Member output

Here I saw the property ‘CommandsToSwitchPath’ however when I tried to set this property an obscure error was returned.  A visit to the VMTN forums uncovered that this seems to be a deprecated property (not confirmed) as it does not exist in ESXi 4.0.  While on the forums LucD pointed me to a cmdlet that is new to PowerCLI 4.1.1, Get-Esxcli.

You can find an article on this cmdlet here.  There is also a good blog post here.

With this information I had all I needed to complete the script.  The working part of the script is simply this:

1
2
## Create the remote ESXCLI object
$esxcli = Get-EsxCli $myHost

$luns = Get-VMHost $VMHost | Get-ScsiLun -LunType “disk”
ForEach ($lun in $luns)
{
## Get the device ID to configure
$device = $lun.CanonicalName

## Set the multipath policy to round-robin
$esxcli.nmp.device.setpolicy($null,$device,”VMW_PSP_RR”)

## Set the iops policy
$esxcli.nmp.roundrobin.setconfig(0,$device,[long] 10,”iops”,$false)
}

 

This code sample will set this policy for every LUN attached to the specified host.  If you have LUNs from multiple arrays make sure the policies you are setting are compatible with all of the arrays.  You can also filter the output of the Get-ScsiLun cmdlet by reading the Model property of the LUN.

1
$luns = Get-VMHost $VMHost | Get-ScsiLun -LunType "disk" | Where {$_.Model -Match "array model name"}

 

-James

Posted in PowerShell / PowerCLI by James. No Comments

Is Hyperthreading Active?

While troubleshooting some guest performance issues we looked at the Processors page from the Configuration tab in vCenter and found the status for hyperthreading to be disabled on a particular host. Now, enabling hyperthreading was no big deal. It did require rebooting the host but that was easily done after hours when the load on the cluster had gone down. The challenge was checking the other 270+ ESX/ESXi hosts to see if hyperthreading was disabled.

My friend PowerCLI had the answer in this command:

Get-Host | Select-Object Name,HyperthreadingActive | Where-Object {$_.HyperthreadingActive -eq $false}

This will return a nice table with the name of all of the host on the currently connected vCenter that have hyperthreading disabled.

Posted in PowerShell / PowerCLI by James. No Comments

Hello world!

Hello and welcome to VMJim.  My name is James and I am a Senior Systems Engineer for Windows and VMware products.  I have been working in the IT field for over 11 years and decided to start this blog for two reasons.  First I thought this would be a nice place to document issues and challenges that I come across so others (and I) can refer to them.  Second, I commonly find myself in discussions with co-workers and peers from other companies about various topics related to Windows Administration and virtualization with VMware and this would be a good forum to document and investigate some of these conversations.

I view my knowledge and experience as a journey to be the best I can be at what I do.  I believe it is necessary to know who to go to for help as well as be able to be a resource for others.  That is what I hope to accomplish here.

Welcome to my journey!

-James

Posted in General Discussions by James. No Comments