Breadcrumb Build – Region A VVD Virtual Infrastructure on VxRail Part 6: Deploying the Shared Edge/ Compute VxRail

Welcome to Part 6 of my VVD on VxRail Breadcrumb Build Series

• Part 1: Preparation
• Part 2: Deploying the Management VxRail
• Part 3: Adjusting the Management VxRail
• Part 4: Installing NSX for Management VxRail
• Part 5: Logical Networking for Management VxRail
• Part 6: Deploying the Shared Edge/Compute VxRail
• Part 7: Adjusting the Shared Edge/Compute VxRail
• Part 8: Installing NSX for Shared Edge/Compute VxRail
• Part 9: Logical Networking for Shared Edge/Compute VxRail
• Part 10: Conclusion

Congratulations on making it this far. Post 5 was a hard slog to write, so I’m sure it took some time to consume / use it too. The good news is that from here on in, you’ve more or less seen it all before, so the next few posts should be a bit familiar to you. This post is going to get the second VxRail (Shared Edge/Compute) up and running. I’ll refer to it mostly as the SEC VxRail….its just shorter to type. Let’s crack on!

Anything in the format [input_value] represents a value from your preparation that you should insert (without the square brackets of course)

Deploying the Workload vCenter

• Phase I
  • Mount the vCenter ISO -> Run UI Installer -> Install -> vCenter Server (Requires External Platform Services Controller)
    • vCenter Server name: [mgmt-vcenter-fqdn]
    • HTTPS Port: 443
    • Username: administrator@vsphere.local
    • Password: [sso-admin-password]
  • Set up appliance VM
    • VM Name: [sec-vcenter-vmname]
    • Passwords: [sec-vcenter-root-password]
  • Deployment Size: Large
    • Network: [mgmt-portgroup]
    • IP version: IPv4
    • IP assignment: Static
    • System name: [sec-vcenter-fqdn]
    • IP address: [sec-vcenter-ip]
    • Subnet mask or prefix length: [mgmt-mask]
    • Default gateway: [mgmt-gateway]
    • DNS servers: [dns-server]
• Phase II
  • Appliance configuration
    • Time synchronization mode: Synchronize time with NTP servers
    • NTP servers: [ntp-ip]
    • SSH Access: Selected
  • SSO configuration
    • PSC: [psc-lb-virtual-fqdn]
    • HTTPS port: 443
    • SSO domain name: vsphere.local
    • SSO password: [sso-admin-password]
• Add [sec-vcenter-vmname] to DFW Exclusion List
  • Networking & Security -> Firewall Settings -> Exclusion List

Replace the Certificate of the SEC vCenter Server

• Ensure bash is set a default shell
  • SSH to [sec-vcenter-ip]
  • shell
  • chsh -s “/bin/bash” root
• mkdir -p /root/certs
• WinSCP files to /root/certs
  • [sec-vcenter-hostname].1.cer
  • [sec-vcenter-hostname].key
  • Root64.cer
• /usr/lib/vmware-vmca/bin/certificate-manager
  • Option 1
  • Enter IP of [sec-psc-ip]
  • Option 2
• Enter file names
• service vami-lighttp restart
• cd /root/certs
• rm*

SEC VxRail Pre-Requisites

• Create Datacenter in [sec-vcenter-fqdn]
  • Log into [sec-vcenter-fqdn]
  • Hosts & Clusters -> Select [sec-vcenter-fqdn] -> New Datacenter -> [sec-datacenter]
• Create User
  • Home -> Administration -> Single Sign-on -> Users and Groups -> Users
  • Add User
    • Username: [sec-vxrm-managment-user]
    • Password: [sec-vxrm-managment-password]

Factory Reset (as Required)

This only needs to be done if the Rail was previously used and/or you want to wipe the nodes clean / install and updated image from the Local SD card on each host.

• Connect to iDRAC
• Launch Virtual Console
  • Menu -> Next Boot -> Local SD Card
  • Power -> Warm Reset
• At SD Menu
  • 1 (Factory Reset)
  • Y Continue
• Reboot

VxRail Bring Up With Non-Native VLAN (Optional)

If the management network used for ESXi hosts and VxRail Manager is not the native VLAN on your switches, then you need to do a little bit of prep before these components will be able to talk to each other when running the VxRail Deployment wizard. If this is for you , perform the following on each ESXi Node (replacing the IPs/VLANs accordingly)

• Enable Shell (found this to be done already)
  • F2 – > Login as root / Passw0rd!
  • Troubleshooting Options -> Enter
  • Enable ESXi Shell -> Enable ->ESC
• Set VLANs
  • Alt-F1 to access Shell
  • Login root / Passw0rd!
  • Execute
    • esxcli network vswitch standard portgroup set -p “Management Network” -v [mgmt-vlan]
    • esxcli network vswitch standard portgroup set -p “VM Network” -v [mgmt-vLAN]
    • /etc/init.d/loudmouth restart
  • Verify VLAN ID
    • esxcli network vswitch standard port group list
  • Set IP/VLAN on VxRail Manager
    • Login with root / Passw0rd!
    • Open Xterm (replace IP info accordingly)
      • systemctl status vmware-marvin
      • systemctl stop vmware-marvin
      • ip addr add [sec-vxrm-ip]/[mgmt-prefix-length] brd [mgmt-broadcast] dev eth0
      • ip addr del 192.168.10.200/24 brd dev eth0
      • ip route add default via [mgmt-gateway]
      • ip link set eth0 down
      • ip link set eth0 up
      • ip a
      • /opt/vmware/share/vami/vami_set_network eth0 STATICV4 [sec-vxrm-ip] [mgmt-mask] [mgmt-gateway]
      • systemctl restart vmware-marvin
      • sytemctl restart vmware-loudmouth
      • systemctl restart network

Deploy SEC VxRail

Back to Automation for a spell!

• Browse to [sec-vxrm-fqdn] -> Get Started -> Accept
• Discover 4 Nodes
• Confirm the list of Nodes
• Choose Step-By-Step
• System
  • Globals
    • NTP: [ntp-ip]
    • NIC Configuration [use default]
  • Networks
    • Top Level Domain
      • Domain: [ad-domain]
    • ESXi Host Names
      • Prefix: [sec-esxi-hostname-prefix]
    • ESXi Host IP Pool
      • Starting IP: [sec-esxi-ip-range-start]
      • Ending IP: [sec-esxi-ip-range-end]
    • vCenter Server
      • Check Join Existing vCenter Server
      • Check External Platform Services Controller
      • Hostname: [sec-vcenter-fqdn]
      • Username: administrator@vsphere.local
      • Password: [sso-admin-password]
      • Management Username: [sec-vxrm-managment-user]
      • Management Password:  [sec-vxrm-managment-password]
    • Platform Services Controller
      • Hostname: [psc-lb-virtual-fqdn]
    • VxRail Manager
      • Hostname: [sec-vxrm-hostname]
      • IP Address: [sec-vxrm-ip]
    • Networking
      • Subnet Mask: [mgmt-mask]
      • Gateway: [mgmt-gateway]
      • DNS: [dns-server]
  • vSphere vMotion
    • Starting IP: [sec-vmotion-ip-range-start]
    • Ending IP: [sec-vmotion-ip-range-end]
    • Subnet Mask: [sec-vmotion-mask]
    • VLAN ID: [sec-vmotion-vlan]
  • vSAN
    • Starting IP: [sec-vsan-ip-range-start]
    • Ending IP: [sec-vsan-ip-range-end]
    • Subnet Mask: [sec-vsan-mask]
    • VLAN ID: [sec-vsan-vlan]
  • VM Networks
    • VM Network Name: [sec-vmnetwork-name]
    • VLAN ID: [sec-vmnetwork-vlan]
  • Solutions
    • None required. Specify as needed
  • Accounts
    • VxRail Root Password: [sec-vxrail-root-password]
    • ESXi Root Password: [sec-esxi-root-password]
    • Mgmt Username: [sec-esxi-mgmt-username]
    • Mgmt Password: [sec-esxi-mgmt-password]
• Validate
• Deploy

That’s the second VxRail done. You should now have two vCenters in your SSO domain, and one cluster in each. Next step, align the SEC cluster to VVD standards.