Breadcrumb Build – Region A VVD Virtual Infrastructure on VxRail Part 10: Conclusion

Welcome to Part 10 of my VVD on VxRail Breadcrumb Build Series

• Part 1: Preparation
• Part 2: Deploying the Management VxRail
• Part 3: Adjusting the Management VxRail
• Part 4: Installing NSX for Management VxRail
• Part 5: Logical Networking for Management VxRail
• Part 6: Deploying the Shared Edge/Compute VxRail
• Part 7: Adjusting the Shared Edge/Compute VxRail
• Part 8: Installing NSX for Shared Edge/Compute VxRail
• Part 9: Logical Networking for Shared Edge/Compute VxRail
• Part 10: Conclusion

Initial Thoughts

Firstly, this build was a lot of fun. I’ve spent so long over the past few years working on automation and being knee deep in the minutiae of getting the smallest of tasks done via API etc that its been really refreshing to build a system end to end by hand.

That said….it’s not something you’d want to do every week, and if you did you would very quickly start looking for an automation tool. Hence the work that Enterprise Hybrid Cloud did with its AIT (Automated Installation Tool) and VMware have done with the likes of Cloud Builder. It’s tools that these that I love to create and work with….it adds incredible value IMHO as it’s such a hard thing to build two systems of this scale exactly to the same spec.

Ultimately there is always a place in the world for step-through documentation like this series of posts. You need it as the basis for automation, and I say the basis as quite often automation is far more exhaustive than even the last 9 posts have been. Things that you do with one-click in UI can have several layers of API automation that in themselves need sequencing before that single objective gets done.

Having working on Enterprise Hybrid Cloud build documentation and subsequently the automation of same, I take my hat off to the guys in both the VVD and VxRail teams for a great bit of work. Its not often you can get through 200 pages of build tasks, and not feel like there were too many areas that were open to interpretation.

So with that said, here are some of the things that I stumbled on (through not having enough coffee), things I interpreted differently or things I just feel are worth pointing out.

Observations

• When deploying a VxRail, we all know that you need a minimum of three hosts to proceed. In my experience, if there are many nodes on the the same VLAN as your VxRail manager you will only find hosts with the same spec as the host that VXRM is running on. This caught me out as one of my nodes had some failed RAM….and so it didn’t show up. You can skip that node and add it afterwards. The add node to cluster automation is not as picky. Good to know!
• You can change the starting suffix number for an ESXi hostname when deploying a VxRail, but you cannot go back later to fill in hostnames with suffix numbers that start before that original starting prefix. So start where you really would like to start. I tried to exclude my host with the failed RAM and come back to make that ‘esxi01’ later on. No dice.
• I really can’t stress enough the importance of being well prepared. There are so many settings, that having them nailed down in an easy to reference format really helped me. I did it in a Text Editor, so that I could copy and paste without any formatting issues. I tried to reproduce the essence of that in the first post in the series. Feedback welcome!
• The VVD 4.3 build process will configure static routes on the Shared Edge/Compute ESGs for networks south of the UDLR and DLR respectively, but doesn’t actually go about putting interfaces on the UDLR or DLR to act as gateways for workloads on those networks. I’m not sure if this is an omission that will be corrected later, or because those networks are considered ‘too customer specific’. Thats a valid point of view. Just remember that the static routes you configure are kind of let hanging there until that is done at some point in the future.
• There is a section of the VVD doc that indicates you should modify the system created anti-affinity rule for the UDLR VMs and add the N/S ESGs to the rule. What I missed was that you should only do this if the cluster had more than four hosts.
  • I missed it as I was primarily using the VxRail doc as reference. It doesn’t mention the ‘more than 4 nodes caveat’ (as its predicated on 4 nodes builds) and when my system threw an error about conflicting anti-affinity rules I jumped straight to the same spot in the VVD docs to confirm. However, a small formatting quirk in the VVD doc had the warning embedded in the previous bullet, so I didn’t see it.
• Be careful when setting up the firewall rules on the management cluster. Between a combination of some quirky cache with the vSphere flash client, and needing to check a series of buttons to add and ip set to security group, I managed bork my access to the system when I set the default firewall rule to Block. I worked around it though, and there’s a ProTip here if you manage to do the same

I might think of a few other things as I go back through my notes, and if I do I’ll update the post, but in the meantime, I’m off to upload another Breadcrumb Build on the VMware HCX configuration I build on top of this system. Check back soon for that.