How To Fix Host Not Responding Error with VMware ESX, vSphere in vCenter

Posted on 05.Feb 2010 by in Virtualisation, VMware

Virtualcenter looses connectivity to an ESX or vSphere host, and all of the virtual machines that are running on the host show as ‘disconnected’. You will also see that the host has ‘not responding’ in brackets next to it’s name.

This one is very simple to fix, as it is usually caused by the host agent service (mgmt-vmware) failing due to a dead process.

First, try and restart the mgmt-vmware service:

# service mgmt-vmware restart

If you find this is hanging when trying to restart the host agent, then you’ll need to kill off the process causing the issue. Open another console session and do the following:

# ps -ef | grep hostd

This will output a list of processes using hostd similar to the following:

root 23955 1 0 10:42 pts/1 00:00:00 /bin/sh /usr/bin/vmware-watchdog -s hostd -u 60 -q 5 -c /usr/sbin/vmware-hostd-support /usr/sbin/vmware-hostd -u
root 23961 23955 4 10:42 ? 00:00:15 /usr/lib/vmware/hostd/vmware-hostd /etc/vmware/hostd/config.xml -u
root 24211 23422 0 10:48 pts/1 00:00:00 grep hostd

If you look at the output carefully you’ll see that the first process is using the vmware-watchdog, this is fine, but the second line is using hostd (config.xml -u). This is the culprit, so lets kill the process. By the way, your virtual machines will continue to run so don’t worry about that.

# kill -9 23961

You’ll now find that the hostd service will start and after a few seconds your host and virtual machines will become available again in vCenter.

Thanks for reading! Please comment or Tweet this page (see below)

Tags , , , , , ,

14 Responses

  1. mrteenwolf says:
    January 14, 2011 at 14:36

    You saved me from pulling out the remainder of my hair.

    vCenter lost connectivity to an ESX host every 90 seconds following a host reboot and the VMware KBs provided only a hint towards the solution. After finding the process ID and killing it with your command above the host stays connected.

    Thank you!

    Reply
  2. Anahita says:
    January 16, 2011 at 11:47

    Hi,
    Thanks for the good information but I still has a problem relating to mgmt-vmware service. I have an ESX 4 server and I can not connect to it using Vsphere client getting this error:” vSphere Client could not connect to [ip address of the ESX server]. A connection failure occurred (unable to connect to the remote server]. After some investigation I understood that the mgmt-vmware service (vmware-hosted) is stopped. when I restart the server it is in a running status for a few seconds and then be stopped automatically. I searched a lot but could not find any solution. Would you mind helping me?
    Regards,

    Reply
    • Ray Heffer says:
      January 16, 2011 at 12:03

      If I understand your situation correctly, hostd isn’t running (mgmt-vmware service) and is stopped even after rebooting the ESX host? You can check this with service mgmt-vmware status. Also check the hostd process using ps -ef | grep vmware-hostd to see if it’s running.

      If it keeps doing this, even after a reboot then I would check the following logs: /var/log/messages, /var/log/vmkernel and /var/log/vmkwarning

      Ray

      Reply
  3. Anahita says:
    January 16, 2011 at 12:49

    Thanks a lot for the response. I have rebooted the main server physically and the problem did not solved. also I can restart the mgmt-vmware and after that when I use the I recieve this information: ps -ef | grep vmware-hostd
    root 26055 20965 0 16:11 pts/0 00:00:00 grep vmware-hostd
    But as usaual after a few seconds the vmware-hosted is being stopped.
    I saw a post at http://communities.vmware.com/thread/77006?tstart=4095; it said the problem was because a syntax error in a file. could my situation be the same? should I send the log files?

    Reply
  4. Ray Heffer says:
    January 16, 2011 at 12:59

    It might be worth posting this on the VMware community forums, and include the logs. I think it’s unlikely to be a syntax error in one the system files unless they’ve been edited. Is this is live ESX server or just a test box?

    The results of the grep vmware-hostd verify that hostd is not running. The result you see is the grep command.

    Also, check that your ESX host hasn’t run out of disk space by using df -h.

    Have you checked the steps in this KB: http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1002849

    Reply
  5. Anahita says:
    January 16, 2011 at 13:17

    Thanks again for all your repsonses. I have checked the site but again I will do it more accuratly and also inform you the results.

    Kind regards,

    Reply
  6. Bob says:
    May 13, 2011 at 11:51

    I have the same problem described above. Manually restarting the mgmt-vmware service works fine however I am trying to troubleshoot why it is not starting when the host starts. I have rebooted the host numerous times and everything else seems fine except this service. I am very new to VMWare (and Linux) so am at a bit of a loss as to why this would not start with the host. I really appreciate any assistance you can provide and this post was a big help to at least get connectivity back to the VSphere client. I did a lot of research on this before stumbling accross this and your post was the only one I found that got me close to resolving the issue. Thank you very much.

    Reply
  7. Bob says:
    May 31, 2011 at 12:32

    Hello Ray, I just wanted to follow up and see if you had any ideas/suggestions about my problem above? Thanks again.

    Reply
    • Ray Heffer says:
      May 31, 2011 at 14:05

      Hello Bob. What version of ESX are you running and is it a single host or multiple hosts with this issue? This should certainly start with the host unless it fails each time, in which case sounds like an issue with your particular ESX host. If you can provide more details I can see if I can help you.

      Ray

      Reply
  8. Bob says:
    May 31, 2011 at 18:48

    It is the latest released version (ESX v4.1 Update 1.) Single Host experiencing the problem. The Host fails to start each time on reboot. Manually restarting the Host Agent “service mgmt-vmware restart” has worked everytime. Once restarted, it continues running just fine until next Host reboot.

    Reply
  9. Bob says:
    May 31, 2011 at 19:41

    No 3rd party agents are installed. Yes, it is a little strange to me too. I will look at the link, thanks.

    Reply
  10. Allen White says:
    September 5, 2011 at 14:54

    all, we had this issue on esxi use the menu system to reset the mgmt services

    http://www.techieshelp.com/esxi-host-disconnected-in-vcentre/

    Reply
  11. Sachin Shinde says:
    December 23, 2011 at 06:12

    if you still facing such problem, no issues solution is there :-)

    Simply log in your VCenter Sevrver>

    go to Services.msc>

    find VMware VirtualCenter Server> Service would be stopped just go and start the service.

    Thats it :-)

    now you should able to log in VSphere Client without any probelm thats for sure….

    incase of any issues please tell me on sachin200682@gmail.com

    Reply

Leave a Reply