Troubleshooting NSX Hybrid Connect/HCX

VMware HCX was announced at VMworld Europe 2017.  Use cases for the solution include Infrastructure Hybridity & Workload Mobility.   See the official product page and this blog post or more information

architecture.png

In this post I’m going to walk through some basic steps to validate and troubleshoot NSX Hybrid Connect, a.k.a HCX utilizing diagnostic tools built into the product

Utilizing the VMware Hands On Lab environment for this.  Specifically HOL-1981-02-HBD – Modernize your SDDC with NSX Hybrid Connect

If you haven’t seen HCX before, I definitely recommend walking through the HOL yourself to give it a try and see how this technology can benefit your business.

Checking HCX Manager Resource Utilization & Services

  1. Open a browser and connect to the HCX Manager Appliance UI https://hcxmanager-fqdn:9443
  2. Login as admin, with the password that you set during deployment(NOTE, deployment is not covered in this post)HCX-1
  3. Here you can review CPU, Memory & Storage utilization

    HCX-3

  4. Click Appliance Summary to review Services Status(NOTE,  SNMP & SSH are optional services, and may be stoped in your environment).  Validate the other services are running, else you can click the Start button

    HCX-6

    HCX-7

    HCX-8

  5. You can also check status, stop, start services CLI if you prefer.  For this I’ll click the Start button next to the SSH Service
  6. I can now SSH to the HCX Manager Appliance. Login as admin, then su – to root

    HCX-9

  7. Some examples are
    1. systemctl status web-engine will check the status of the web-engine service.
    2. systemctl restart web-engine will restart the web-engine service

      HCX-10

Central CLI (CCLI)

The Central CLI on HCX allows you to run commands available centrally on the HCX Manager to query all your HCX services, including the HCX Interconnect and Network Extension Service. The Central CLI reduces troubleshooting time by providing centralized diagnostics and increases the security of the WAN Interconnect service appliances by eliminating the need to run SSH

  1. I’ll use the CCLI to check ipsec tunnel status
  2. From a SSH session on the HCX Manager, type ccli to enable central CCLI.
    1. Then list to show the various HCX components in your environment.
    2. Type help to show available commands

      HCX-11

    3. In my example i’ll type go 1 to connect to the Network Extension Service
    4. The command prompt should update to show the name of the Network Extension Service. In this example it’s L2C-HT-RegionA01-vDS-COMP-CTVL

      HCX-12

    5. I’ll then type show ipsec status, and I can hit enter to scroll down and see the tunnel status.  In my case I have 5 tunnels up an established 4 hours ago

      HCX-19.png

    6. Type q to return to the ccli. Some other examples are as follows, explore the help command yourself  to view all available
      1. show log alert
      2. show log event
      3. show service ipsec
      4. show system uptime
    7. Another option with CCLI, to to ssh to another component security from the HCX manager appliance
    8. See example.  go 1, debug remoteaccess enable, ssh

      HCX-14

    9. From this session I can then run command ip tunnel and ping remote end to validate connection

      HCX-15.png

Downloading Log Bundles

If you need to open a SR with VMware Global Support Services, be sure to include support logs to expedite the troubleshooting process.

You can get these either from the HCX Manager Appliance, or within the vSphere HCX Plug-In

  1. From the vSphere HCX Plug-In select Administration, Troubleshooting and select at least Collect Core HCX Logs & click Request then Download once complete

    HCX-17

  2. Or from the HCX Manager Appliance, go to Administration, Troubleshooting, Technical Support Logs, Generate, then Download once complete

    HCX-18

  3. Once complete the log bundle will be available for you to download and upload to your SR for analysis by support

 

 

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s