Breadcrumb Build – Region A VVD Virtual Infrastructure on VxRail Part 1: Preparation

Welcome to Part 1 of my VVD on VxRail Breadcrumb Build Series

Note: If you’ve read this before, you might want to rescan it. I’ve revamped how I laid out the data you need to gather as I felt this integrated better with the rest of the posts in the series

Having spent the past five years as chief architect on DellEMC’s Enterprise Hybrid Cloud and having worked very closely with the VMware Validated Designs (VVD) team when they were first getting started, you could say that VVD has a special place in my heart. For that reason, I thought it would be an interesting exercise to carry out a VVD build on VxRail.

If you’ve read my post on documentation styles here then you’ll know the type of documentation I like to use personally, so when I do a build as lengthy as as VVD 4.3 on VxRail, I like to create my own Breadcrumb documentation for my own future reference. This series of posts is going to capture that, and hopefully you guys will find use in it too.

Disclaimer: this is not intended to replace the existing documentation, which is very comprehensive. Nor will this be updated regularly if issues are found with settings from the original release. Think of it as a cheat sheet for repeat builds

Relevant Resources

DellEMC VVD 4.3 on VxRail Region A Deployment Guide

https://community.emc.com/servlet/JiveServlet/download/66332-26-147683/VVD_on_VxRail_Install_Guide_RegionA_Version4.3_final.pdf

Official VMware Region A Deployment Guide

https://docs.vmware.com/en/VMware-Validated-Design/4.3/com.vmware.vvd.sddc-deploya.doc/GUID-657DB777-D919-4C23-BA5E-B98D8A91CA8B.html

Preparation

Before you do anything, get your ducks in a row. Fail to prepare, prepare to fail and all that. Here the are the key areas I mapped out before starting

Licenses

Have licenses together for the following items

  • vSphere 6.5
  • VSAN
  • NSX

Certificates

Certs. So important, so complex (sometimes). Thanks to the clever people I’ve worked with (special shout out to a good friend Brian O’Connell over at LifeOfBrianOC and the similarly clever people he’s recently joined forces with within the hallowed halls of VMware) certs need not be so scary. So pluck up the courage and use the CertGenVVD tool and get all your self-signed certs ready to rock before you start. Make sure you have all your FQDNs and let the cert tool take care of the hard work.

Common Parameters

Supporting Infrastructure

Parameter KeyNotesExample
[ntp-ip]
Time Sync192.168.76.32
[ad-domain]Domain For Region Aregiona.domain.local
[dns-server]Name Resolution192.168.5.9

Active Directory Security Objects

Parameter KeyNotesExample
[sddc-admins]
Used to admin the SDDCSDDC Admins
[vcenter-admins-group]
Used to admin the vCentervCenter Admins
[ad-psc-bind-username]
Account used to connect SSO to Active Directoryad-psc-bind
[ad-psc-bind-password]
VMw@r3!!
[svc-nsxmanager-username]
Account to connect NSX to vCenter
[svc-nsxmanager-password]
VMw@r3!!
[nsx-admin]NSX Enterprise Admin
[nsx-admin-password]VMw@r3!!

SSO Credential Information

Parameter KeyNotesExample
[mgmt-shared-password]

VMw@r3!!
[sso-admin-password]Derived from [mgmt-shared-password]VMw@r3!!

VLAN Information

Know the VLANs you intended to use (and have already been setup on your TORs switches) for each of the following areas across the two VxRails.

Management VLANs

Parameter KeyNotesExample
[mgmt-vlan]
1101
[mgmt-vmotion-vlan]
1102
[mgmt-vsan-vlan]
1103
[mgmt-vmnetwork-vlan]
1101
[mgmt-uplink01-vlan]
1106
[mgmt-uplink02-vlan]
1109
[mgmt-vxlan-vlan]1104

SEC VLANS

Parameter KeyNotesExample
[sec-vmotion-vlan]
1102
[sec-vsan-vlan]
1103
[sec-vmnetwork-vlan]
1101
[sec-uplink01-vlan]
1108
[sec-uplink02-vlan]
1110
[sec-vxlan-vlan]1107

In my case I put the vCenters and hosts for both VxRails on the same VLAN -hence there being no explicit [sec-management-vlan]. You are free to keep them separate as you see fit. Just make sure that the relevant subnets are routable.

vMotion and vSAN while numerically the same were actually isolated and non-routable. Uplinks were independent and routable naturally. I piggy-backed my VM network on the 1101 with management as it was only a test build – do with those what you will.

Physical ESXi Node Information

You’ll need the know the DRAC IPs for all the ESXi nodes if you are going to do a factory reset as part of the process

Parameter KeyNotesExample
[mgmt-esxi01-drac-ip]192.168.45.10
[mgmt-esxi02-drac-ip]192.168.45.11
[mgmt-esxi03-drac-ip]192.168.45.12
[mgmt-esxi04-drac-ip]192.168.45.13
[sec-esxi01-drac-ip]192.168.45.14
[sec-esxi03-drac-ip]192.168.45.15
[sec-esxi03-drac-ip]192.168.45.16
[sec-esxi04-drac-ip]192.168.45.17

VxRail Manager Information

Management VxRail Manager

Parameter KeyNotesExample
[mgmt-vxrm-fqdn]
vxrm01.regiona.domain.local
[mgmt-vxrm-ip]
192.168.5.30
[mgmt-vxrm-hostname]
vxrm01
[mgmt-vxrm-vmname]
Referenced when converting internal vCenter to externalVXRM01
[mgmt-vxrm-root-password]VMw@r3!!

SEC VxRail Manager

Parameter KeyNotesExample
[sec-vxrm-fqdn]
vxrm02.regiona.domain.local
[sec-vxrm-ip]
192.168.5.5
[sec-vxrm-hostname]
vxrm01
[sec-vxrm-root-password]VMw@r3!!

Virtual Networks

Parameter KeyNotesExample
[mgmt-portgroup]
Created by VxRail deployment toolManagement Network-c1628d89-021e-41d5-87c5-7be8f57ded1d
[mgmt-broadcast]
192.168.5.127
[mgmt-gateway]
192.168.5.1
[mgmt-mask]
255.255.255.128
[mgmt-prefix-length]25
[mgmt-vmotion-ip-range-start]
Assigned by VxRail deployment tool192.168.200.10
[mgmt-vmotion-ip-range-end]Assigned by VxRail deployment tool192.168.200.13
[mgmt-vmotion-mask]255.255.255.0
[mgmt-vsan-ip-range-start]
Assigned by VxRail deployment tool192.168.250.10
[mgmt-vsan-ip-range-end]Assigned by VxRail deployment tool192.168.250.13
[mgmt-vsan-mask]255.255.255.0
[mgmt-vmnetwork-name]Supplied by user. Created by VxRail deployment toolVM Network
[sec-vmotion-ip-range-start]

Assigned by VxRail deployment tool192.168.200.14
[sec-vmotion-ip-range-end]Assigned by VxRail deployment tool192.168.200.17
[sec-vmotion-mask]255.255.255.0
[sec-vsan-ip-range-start]
Assigned by VxRail deployment tool192.168.250.14
[sec-vsan-ip-range-end]Assigned by VxRail deployment tool192.168.250.17
[sec-vsan-mask]255.255.255.0
[sec-vmnetwork-name]Supplied by user. Created by VxRail deployment toolVM Network

Platform Services Controller Information

Parameter KeyNotesExample
[mgmt-psc-fqdn]
psc01.regiona.domain.local
[mgmt-psc-hostname]
psc01
[mgmt-psc-ip]
192.168.5.32
[sec-psc-fqdn]
psc02.regiona.domain.local
[sec-psc-hostname]
psc02
[sec-psc-ip]
192.168.5.33
[sec-psc-vmname]
PSC02
[sec-psc-root-password]
VMw@r3!!
[psc-lb-virtual-fqdn]
ra-psc.regiona.domain.local
[psc-lb-virtual-ip]
192.168.5.34
[psc-lb-edgename]RA-PSC

vCenter Information

Management vCenter

Parameter KeyNotesExample
[mgmt-vcenter-hostname]
vc01
[mgmt-vcenter-vmname]
Assigned by VxRail Deployment toolVMware vCenter Server Appliance
[mgmt-vcenter-fqdn]
vc01.regiona.domain.local
[mgmt-vcenter-ip]
192.168.5.31
[mgmt-vcenter-root-password]Derived from [mgmt-shared-password]

SEC vCenter

Note: Additional parameters as vCenter is not wizard deployed

Parameter KeyNotesExample
[sec-vcenter-hostname]
vc02.regiona.domain.local
[sec-vcenter-vmname]
VC02
[sec-vcenter-fqdn]
vc02.regiona.domain.local
[sec-vcenter-ip]
192.168.5.41
[sec-vcenter-root-password]VMw@r3!!
[sec-vxrm-managment-user]
admin
[sec-vxrm-managment-password]VMw@r3!!

ESXi Host Information

Management Hosts

Parameter KeyNotesExample
[mgmt-esxi-root-password]
VMw@r3!!
[mgmt-vxrm-esxi-username]
admin
[mgmt-vxrm-esxi-password]
VMw@r3!!
[mgmt-esxi-hostname-prefix]
mgmt-esxi
[mgmt-esxi-ip-range-start]
192.168.5.15
[mgmt-esxi-ip-range-end]192.168.5.18

SEC Hosts

Parameter KeyNotesExample
[sec-esxi-root-password]
VMw@r3!!
[sec-vxrm-esxi-username]
admin
[sec-vxrm-esxi-password]
VMw@r3!!
[sec-esxi-hostname-prefix]
wld-esxi
[sec-esxi-ip-range-start]
192.168.5.21
[sec-esxi-ip-range-end]192.168.5.24

vSphere Objects

Management Objects

Parameter KeyNotesExample
[mgmt-datacenter]
VxRail-Datacenter
[mgmt-cluster]
Assigned by VxRail deployment toolVxRail-VSAN-Cluster-4fb389d5-3dde-4079-b25e-7633c28a20b9
[mgmt-vds]
Name to rename vds toMgmt-VDS
[mgmt-vsan-datastore]
Name to rename datastore toMgmt-VSAN
[mgmt-generated-random-uid]https://www.uuidgenerator.net/33b60ab9-464b-4890-bf3e-a8a1d83e1060

SEC Objects

Parameter KeyNotesExample
[sec-datacenter]
VxRail-SEC-Datacenter
[sec-cluster]
VxRail-SEC-Cluster
[sec-vds]
Name to rename vds toSEC-VDS
[sec-vsan-datastore]
Name to rename datastore toSEC-VSAN
[sec-generated-random-uid]https://www.uuidgenerator.net/99f67f0e-0f2e-4a17-82bf-b239195187a0

Management NSX Basic Infrastructure

Parameter KeyNotesExample
[mgmt-nsx-manager-vmname]
MGMT-NSX
[mgmt-nsx-manager-fqdn]
mgmt-nsx.regiona.domain.local
[mgmt-nsx-manager-ip]
192.168.5.35
[mgmt-nsx-manager-admin-password]
VMw@r3!!
[mgmt-nsx-manager-hostname]
mgmt-nsx.regiona.domain.local
[mgmt-nsx-controller-complex-password]
VMw@r3!!VVD4
[mgmt-nsx-controller-ip-range]
192.168.5.36 – 192.168.5.38
[mgmt-nsx-controller01-name]
mgmt-nsx-controller01
[mgmt-nsx-controller02-name]
mgmt-nsx-controller02
[mgmt-nsx-controller03-name]
mgmt-nsx-controller03
[mgmt-nsx-segment-id-pools]
5000 – 5200
[mgmt-nsx-multicast-addresses]
239.1.0.0 – 239.1.255.255
[mgmt-nsx-universal-segment-id-pool]
30000 – 39000
[mgmt-nsx-universal-multicast-addresses]239.2.0.0 – 239.2.255.255

SEC NSX Basic Infrastructure

Parameter KeyNotesExample
[sec-nsx-manager-vmname]
WLD-NSX
[sec-nsx-manager-fqdn]
wld-nsx.regiona.domain.local
[sec-nsx-manager-ip]
192.168.5.42
[sec-nsx-manager-admin-password]
VMw@r3!!
[sec-nsx-manager-hostname]
wld-nsx.regiona.domain.local
[sec-nsx-controller-complex-password]
VMw@r3!!VVD4
[sec-nsx-controller-ip-range]
192.168.5.43 – 192.168.5.45
[sec-nsx-controller01-name]
wld-nsx-controller01
[sec-nsx-controller02-name]
wld-nsx-controller02
[sec-nsx-controller03-name]
wld-nsx-controller03
[sec-nsx-segment-id-pools]
5300 – 9000
[sec-nsx-multicast-addresses]
239.3.0.0 – 239.3.255.255
[sec-nsx-universal-segment-id-pool]
20000 – 29000
[sec-nsx-universal-multicast-addresses]239.4.0.0 – 239.4.255.255

Management NSX Logical Infrastructure

Management North/South Edges

Parameter KeyNotesExample
[mgmt-nsx-edge-complex-password]
VMw@r3!!VVD4
[mgmt-esg01-edgename]
Mgmt-ESG01
[mgmt-esg01-uplink01-ip]
192.168.5.149
[mgmt-esg01-uplink02-ip]
192.168.5.197
[mgmt-esg01-utn-ip]
192.168.10.1
[mgmt-esg01-local-as]
65003
[mgmt-esg01-bgp-password]
VMw@r3!!
[mgmt-esg02-edgename]
Mgmt-ESG02
[mgmt-esg02-uplink01-ip]
192.168.5.150
[mgmt-esg02-uplink02-ip]
192.168.5.198
[mgmt-esg02-utn-ip]
192.168.10.2
[mgmt-esg02-local-as]
65003
[mgmt-esg02-bgp-password]VMw@r3!!

Management UDLR

Parameter KeyNotesExample
[mgmt-udlr-utn-uplink-ip]
192.168.10.3
[mgmt-udlr-bgp-protocol-ip]
192.168.10.4
[mgmt-udlr-local-as]
65003
[mgmt-udlr-bgp-password]
Password needed to peer with UDLRVMw@r3!!
[mgmt-udlr-cross-region-vxlan-ip]
192.168.11.1
[mgmt-udlr-regiona-vxlan-ip]
192.168.31.1

Management TORs

Parameter KeyNotesExample
[tor1-bgp-address]
192.168.5.148
[tor1-local-as]
65001
[tor1-bgp-password]
Password needed to peer with TOR1VMw@r3!!
[tor2-bgp-address]
192.168.5.196
[tor2-local-as]
65001
[tor2-bgp-password]
Password needed to peer with TOR2VMw@r3!!

Management Load Balancer

Parameter KeyNotesExample
[mgmt-lb-virutal-fqdn]
mgmt-lb.regiona.domain.local
[mgmt-lb-virtual-ip]
192.168.11.2
[mgmt-lb-edgename]
MGMT-LB

Other Management Network Details

Parameter KeyNotesExample
[mgmt-uplink01-prefix-length]
28
[mgmt-uplink02-prefix-length]
28
[mgmt-utn-prefix-length]
24
[mgmt-cross-region-vxlan-network]
192.168.11.0
[mgmt-cross-region-vxlan-prefix-length]
24
[mgmt-regiona-vxlan-network]
192.168.31.0
[mgmt-regiona-vxlan-prefix-length]
24

SEC NSX Logical Infrastructure

SEC North/South Edges

Parameter KeyNotesExample
[sec-nsx-edge-complex-password]
VMw@r3!!VVD4
[sec-esg01-edgename]
SEC-ESG01
[sec-esg01-uplink01-ip]
192.168.5.181
[sec-esg01-uplink02-ip]
192.168.5.213
[sec-esg01-utn-ip]
192.168.100.1
[sec-esg01-gtn-ip]192.168.101.1
[sec-esg01-local-as]
65000
[sec-esg01-bgp-password]
VMw@r3!!
[sec-esg02-edgename]
SEC-ESG02
[sec-esg02-uplink01-ip]
192.168.5.182
[sec-esg02-uplink02-ip]
192.168.5.214
[sec-esg02-utn-ip]
192.168.100.2
[sec-esg02-gtn-ip]192.168.101.2
[sec-esg02-local-as]
65000
[sec-esg02-bgp-password]VMw@r3!!

SEC UDLR

Parameter KeyNotesExample
[sec-udlr-utn-uplink-ip]
192.168.100.3
[sec-udlr-bgp-protocol-ip]
192.168.100.4
[sec-udlr-local-as]
65000
[sec-udlr-bgp-password]
Password needed to peer with UDLRVMw@r3!!
[sec-udlr-compute-network-ip]
192.168.20.2 **

** Not specified by VVD, and not referenced in posts. But is the next logical step

SEC DLR

Parameter KeyNotesExample
[sec-dlr-gtn-uplink-ip]
192.168.101.3
[sec-dlr-bgp-protocol-ip]
192.168.101.4
[sec-dlr-local-as]
65000
[sec-dlr-bgp-password]
Password needed to peer with UDLRVMw@r3!!
[sec-dlr-compute-network-ip]
192.168.25.2 **

** Not specified by VVD, and not referenced in posts. But is the next logical step

SEC TORs

Parameter KeyNotesExample
[tor1-bgp-address]
192.168.5.180
[tor1-local-as]
65001
[tor1-bgp-password]
Password needed to peer with TOR1VMw@r3!!
[tor2-bgp-address]
192.168.5.212
[tor2-local-as]
65001
[tor2-bgp-password]
Password needed to peer with TOR2VMw@r3!!

Other Management Network Details

Parameter KeyNotesExample
[sec-uplink01-prefix-length]
28
[sec-uplink02-prefix-length]
28
[sec-utn-prefix-length]
24
[sec-gtn-prefix-length]24
[sec-udlr-compute-network]
192.168.11.0
[sec-udlr-compute-network-prefix-length]
24
[sec-dlr-compute-network]
192.168.31.0
[sec-dlr-compute-network-prefix-length]
24

Additional IPS needed for Firewall Rules

These are referenced during firewall configuration and needed for vRealize deployment much further down the line

Parameter KeyNotesExample
[vra_appliance01-ip]
192.168.50
[vra_appliance02-ip]
192.168.51
[vra_appliance03-ip]
192.168.52
[vra-dem01-ip]
192.168.60
[vra-dem02-ip]
192.168.61
[vra-manager01-ip]
192.168.62
[vra-manager02-ip]
192.168.63
[vra-web01-ip]
192.168.64
[vra-web01-ip]
192.168.65
[vra-agent01-ip]
192.168.66
[vra-agent02-ip]
192.168.67
[vrb-ip]
192.168.53
[vrb-collector01-ip]
192.168.54
[vdp-ip]
192.168.55
[vrops-ip]
192.168.56
[vrops-collector01-ip]
192.168.57
[vrli-ip]
192.168.58
[vrlcm-ip]
192.168.59
[update-manager-ip]
192.168.80

If you’ve collected all of the above detail you should be in pretty good shape to move forward.

3 thoughts on “Breadcrumb Build – Region A VVD Virtual Infrastructure on VxRail Part 1: Preparation”

Add yours

  1. Pingback: Breadcrumb Build: Region A VVD on VxRail Part 2: Deploying the Management VxRail – FearDamhan
  2. Pingback: Breadcrumb Build: Region A VVD on VxRail Part 3: Adjusting the Management VxRail – FearDamhan
  3. Pingback: Breadcrumb Build – Region A VVD Virtual Infrastructure on VxRail Part 10: Conclusion – FearDamhan

Leave a Reply

Fill in your details below or click an icon to log in:

Gravatar
WordPress.com Logo

You are commenting using your WordPress.com account. ( Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. ( Log Out /  Change )

Cancel

Connecting to %s

Blog at WordPress.com.

Up ↑

%d bloggers like this: