• #17
    Open

    vmbase_init.service fails to start due to VC Cert Name Mismatch, default self signed cert and disable TLS verification TRUE

    Aug 30, 2023

    Hi All,

    I'm testing this in an internal lab due to customer interest in this Fling. I've encountered a persistent certificate based error, which is not adding up due to appliance configuration.

    Essentially, the plugin never registers in vCenter due to the vmbase_init.service persistently crashing. It's crash with debug level logs is detailed below:

    ***

    VC Version: vCenter Server 7.0 Update 3j Build 20990077
    Fling OVA used: VMware-Power-Actions-1.0.2.0-22232535_OVF10.ova

    ***

    Service status:

    root@VMware-Power-Actions [ ~ ]# systemctl status vmbase_init.service
    ● vmbase_init.service - Appliance initialization script
    Loaded: loaded (/lib/systemd/system/vmbase_init.service; disabled; vendor preset: enabled)
    Active: failed (Result: exit-code) since Mon 2023-08-28 19:41:03 UTC; 1 day 20h ago
    Process: 1754 ExecStart=/usr/lib/vmbase_init/vmbase_init (code=exited, status=1/FAILURE)
    Main PID: 1754 (code=exited, status=1/FAILURE)

    Aug 28 19:41:01 VMware-Power-Actions systemd[1]: Starting Appliance initialization script...
    Aug 28 19:41:03 VMware-Power-Actions systemd[1]: vmbase_init.service: Main process exited, code=exited, status=1/FAILURE
    Aug 28 19:41:03 VMware-Power-Actions systemd[1]: vmbase_init.service: Failed with result 'exit-code'.
    Aug 28 19:41:03 VMware-Power-Actions systemd[1]: Failed to start Appliance initialization script.

    ***

    Service restart output:

    root@VMware-Power-Actions [ ~ ]# systemctl restart vmbase_init.service
    Job for vmbase_init.service failed because the control process exited with error code.
    See "systemctl status vmbase_init.service" and "journalctl -xe" for details.

    ***

    /var/log/bootstrap (same output at journalctl -xe) error:

    **** NOTE: Acknowledgement of configuration to disable TLS verification, to be used when using self-signed certificates on vCenter per documentation ***

    16:21:22:22 [INFO] ${VC_DISABLE_TLS_VERFIVICATION}: True

    *** vCenter registration configuration ***

    16:21:22:22 [INFO] ${POST_DEPLOY_ENDPOINT}: https://vcsa-01a.corp.local/api/ui/solutioninstall
    16:21:22:22 [INFO] ${VC_ADDRESS}: vcsa-01a.corp.local

    *** Error message ***

    ++ echo '16:21:23:23 [INFO] Setting up hostname to '
    16:21:23:23 [INFO] Setting up hostname to
    ++ SRSA_HOSTNAME=VMware-Power-Actions
    ++ log_info 'Validate vCenter certificate'
    ++ log 'Validate vCenter certificate' INFO '\033[0;32m'
    +++ date +%T:%S
    ++ local TIME=16:21:23:23
    ++ echo -e '16:21:23:23 [\033[0;32mINFO\033[0m] Validate vCenter certificate'
    ++ echo '16:21:23:23 [INFO] Validate vCenter certificate'
    16:21:23:23 [INFO] Validate vCenter certificate
    +++ openssl s_client -connect vcsa-01a.corp.local:443
    +++ openssl x509 -subject -noout -in /dev/stdin
    +++ awk 'match($0, /CN ?= ?.*[,/]+/) {print substr($0, RSTART, RLENGTH-1)}'
    +++ cut -d = -f2
    depth=0 CN = vcsa-01a.corp.local, C = US, ST = California, L = Palo Alto, O = VMware, OU = VMware Engineering
    verify error:num=20:unable to get local issuer certificate
    verify return:1
    depth=0 CN = vcsa-01a.corp.local, C = US, ST = California, L = Palo Alto, O = VMware, OU = VMware Engineering
    verify error:num=21:unable to verify the first certificate
    verify return:1
    DONE
    ++ CERT_COMMON_NAME=vcsa-01a.corp.local/C
    ++ '[' vcsa-01a.corp.local '!=' vcsa-01a.corp.local/C ']'
    ++ log_error 'vCenter address does not match it'\''s certificate common name'
    ++ log 'vCenter address does not match it'\''s certificate common name' ERROR '\033[0;31m'
    +++ date +%T:%S
    ++ local TIME=16:21:23:23
    ++ echo -e '16:21:23:23 [\033[0;31mERROR\033[0m] vCenter address does not match it'\''s certificate common name'
    ++ echo '16:21:23:23 [ERROR] vCenter address does not match it'\''s certificate common name'
    16:21:23:23 [ERROR] vCenter address does not match it's certificate common name

    Certificate output when queried via openssl via the VMware-Power-Actions Appliance:

    root@VMware-Power-Actions [ ~ ]# openssl s_client -connect vcsa-01a.corp.local:443 -showcerts
    CONNECTED(00000003)
    depth=0 CN = vcsa-01a.corp.local, C = US, ST = California, L = Palo Alto, O = VMware, OU = VMware Engineering
    verify error:num=20:unable to get local issuer certificate
    verify return:1
    depth=0 CN = vcsa-01a.corp.local, C = US, ST = California, L = Palo Alto, O = VMware, OU = VMware Engineering
    verify error:num=21:unable to verify the first certificate
    verify return:1
    ---
    Certificate chain
    0 s:/CN=vcsa-01a.corp.local/C=US/ST=California/L=Palo Alto/O=VMware/OU=VMware Engineering
    i:/CN=vcsa-01a/DC=vsphere/DC=local/C=US/ST=California/O=vcsa-01a.corp.local/OU=VMware Engineering

    Server certificate
    subject=/CN=vcsa-01a.corp.local/C=US/ST=California/L=Palo Alto/O=VMware/OU=VMware Engineering
    issuer=/CN=vcsa-01a/DC=vsphere/DC=local/C=US/ST=California/O=vcsa-01a.corp.local/OU=VMware Engineering

    ***
    For what it's worth, I've recreated the vCenter certificate twice in this environment, using the fixcerts.py script, to receive a new certificate, each time with a CN that matches the "vcsa-01a.corp.local" CN that I've configured for connection

    Ask to VMware Power Actions Community:

    Is this by chance a bug in the fling? Does anything stick out as environmentally incorrect? On the second point, I've looked this over carefully and do not see a particular problem, outside of using a self-signed certificate, which I believe is the intention of allowing users to disable TLS verification in their OVA deployment.

    Thanks in advance!

  • Sep 01, 2023

    There is a bug in the vCenter certificate check. It will be fixed in the next release.