VMware ESX Error (Unable to connect to the MKS vmx connection handshake failed for mks)

Posted on 03 Mar 2010 by Ray Heffer

When you power on a virtual machine with VMware ESX VI3 or vSphere it may reach 95% progress in the tasks, but when you open the console you’ll notice the error:

Unable to connect to the MKS: vmx connection handshake failed for mks of /vmfs/volumes...

The MKS is the virtual Mouse, Keyboard, Screen (aka. KVM - Keyboard, Video, Mouse), and the error is simply unable to map this to the guest OS, hence the power on will fail.

This issue isn’t very common, but should be relatively easy to resolve. First check the following:

1. Edit the settings of the virtual machine, click the Options tab and make sure that the guest operating system version is correct. This can cause this error, so make sure you select the correct OS. E.g. “Microsoft Windows Server 2008 R2 (64-bit)”.
2. Try removing the virtual machine from the vCenter inventory and re-adding it.
3. Try restarting the VMware management agents (service mgmt-vmware restart and service vmware-vpxa restart).

I have only had this issue occur twice and both times it was related to the guest operating system type being set incorrectly.