Troubleshooting HCX

VMware HCX was announced at VMworld Europe 2017.  Use cases for the solution include Infrastructure Hybridity & Workload Mobility.   See the official product page and this blog post or more information


In this post I’m going to walk through some basic steps to validate and troubleshoot NSX Hybrid Connect, a.k.a HCX utilizing diagnostic tools built into the product

Utilizing the VMware Hands On Lab environment for this.  Specifically HOL-1981-02-HBD – Modernize your SDDC with NSX Hybrid Connect

If you haven’t seen HCX before, I definitely recommend walking through the HOL yourself to give it a try and see how this technology can benefit your business.

Checking HCX Manager Resource Utilization & Services

  1. Open a browser and connect to the HCX Manager Appliance UI https://hcxmanager-fqdn:9443
  2. Login as admin, with the password that you set during deployment(NOTE, deployment is not covered in this post)


  3. Here you can review CPU, Memory & Storage utilization


  4. Click Appliance Summary to review Services Status(NOTE,  SNMP & SSH are optional services, and may be stoped in your environment).  Validate the other services are running, else you can click the Start button




  5. You can also check status, stop, start services CLI if you prefer.  For this I’ll click the Start button next to the SSH Service
  6. I can now SSH to the HCX Manager Appliance. Login as admin, then su – to root


  7. Some examples are
    1. systemctl status web-engine will check the status of the web-engine service.
    2. systemctl restart web-engine will restart the web-engine service


Central CLI (CCLI)

The Central CLI on HCX allows you to run commands available centrally on the HCX Manager to query all your HCX services, including the HCX Interconnect and Network Extension Service. The Central CLI reduces troubleshooting time by providing centralized diagnostics and increases the security of the WAN Interconnect service appliances by eliminating the need to run SSH

  1. I’ll use the CCLI to check ipsec tunnel status
  2. From a SSH session on the HCX Manager, type ccli to enable central CCLI.
    1. Then list to show the various HCX components in your environment.
    2. Type help to show available commands


    3. In my example i’ll type go 1 to connect to the Network Extension Service
    4. The command prompt should update to show the name of the Network Extension Service. In this example it’s L2C-HT-RegionA01-vDS-COMP-CTVL


    5. I’ll then type show ipsec status, and I can hit enter to scroll down and see the tunnel status.  In my case I have 5 tunnels up an established 4 hours ago


    6. Type q to return to the ccli. Some other examples are as follows, explore the help command yourself  to view all available
      1. show log alert
      2. show log event
      3. show service ipsec
      4. show system uptime
    7. Another option with CCLI, to to ssh to another component security from the HCX manager appliance
    8. See example.  go 1, debug remoteaccess enable, ssh


    9. From this session I can then run command ip tunnel and ping remote end to validate connection


Downloading Log Bundles

If you need to open a SR with VMware Global Support Services, be sure to include support logs to expedite the troubleshooting process.

You can get these either from the HCX Manager Appliance, or within the vSphere HCX Plug-In

  1. From the vSphere HCX Plug-In select Administration, Troubleshooting and select at least Collect Core HCX Logs & click Request then Download once complete


  2. Or from the HCX Manager Appliance, go to Administration, Troubleshooting, Technical Support Logs, Generate, then Download once complete


  3. Once complete the log bundle will be available for you to download and upload to your SR for analysis by support



Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s