How-To

Troubleshooting Trouble in 2018 with Networking, VMs and VMware View

Rick Vanover works through an odd issue after a Windows Update reboot.

• By Rick Vanover
• 04/18/2018

More than nine years ago, fellow Virtualization & Cloud Review contributor Chris Wolf wrote a blogpost that stuck with me for a long time: "Troubleshooting Trouble."  When I read the post then, I was nodding my head in agreement. Oddly, just recently I had a situation happen and I could not help but remember his post. In this article, I'll share an interesting situation and the progression I went through to resolve it.

The symptom was that after a reboot from Windows Update on a collection of VMs, VMware View wasn't able to communicate with the VMs. It gave a message I hadn't seen before, "Invalid IP," as shown in Figure 1.

I had never seen this message, but it seemed to match the symptom of the user not being able to connect to the VMware View VM. At first, I wanted to blame Windows Update -- but that wasn't the case. I wanted to check the IP configuration and that's where it got interesting. I couldn't determine what was going wrong, and I even went to Twitter for help. I was grateful as more than 20 people responded with ideas, but I still couldn't determine why this system, which had a static IP previously, was reporting the Windows automatic IP private addressing (the 169) as its address. Even the ipconfig /all option was not helping or making sense. The screenshot in Figure 2 is what was shown on the impacted system.

At that point, troubleshooting needed to kick in before it really became a sticky situation. I found that after each boot, VMware View would report that each VM had an Invalid IP setup, and each had taken the automatic IP private addressing scheme and disregarded the static IP that was in place previously. The troubleshooting process involved each of the following technologies:

• Double-checking Windows networking configuration
• Double-checking VMware vSphere configuration for the VM
• Double-checking VMware vSphere configuration for the distributed switch
• Double-checking VMware View Server configuration
• Re-installing the VMware View Agent
• Uninstalling the latest Windows Updates
• Reading some suggested registry changes to disable automatic private IP addressing

None of these resolved the issue. Additionally, deploying new templates also had the issue. This was really frustrating! Those of you who know me know that I will have a backup. Restoring the backup brought the same behavior, as well! I knew something was amiss at this point.

After a good night's sleep (this is a lab environment), Anthony on my team tried to tackle the same issue.  He found this post where an additional registry was suggested to solve the issue -- and it worked. Turns out that a switch on the "external" interface of this VMware View environment was updated with some sort of setting (still investigating!) that basically caused static IP addresses to not function -- yet the registry entry for [HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters] that had ArpRetryCount (32-bit DWORD) to 0 restored the connectivity and VMware View was happy again.

This was one of the oddest things I have ever had to troubleshoot, and nine years later, I can confirm that Chris Wolf's prediction came true!