Nested vSphere and NSX-T Deployed With Ansible – April Update

Some weeks ago I introduced you to my GitHub repository containing a set of Ansible playbooks helping people deploy a highly customizable vSphere 6.7/7.0 with NSX-T 2.5/3.0 nested lab environment.

As I mentioned in the “launch” post, this project is a work in progress and during the last couple of weeks I’ve been spending many hours on trying to improve bits and pieces of the deployment process. I’m also learning more and more about working with Git and Ansible which is a great added bonus.

After reaching somewhat of a milestone the other day, I thought I’d write a short blog post on what’s new and improved. So let’s have a look.

What’s New

Python 3

No more Python 2 code or dependencies! As a matter of fact, I carry out all testing from an Ubuntu 18.04 VM with only Python 3 installed.

VyOS router

A VyOS router (VM) is now part of the default deployment.

VIFs on the router’s internal interface are default gateways for the VLANs within the nested environment. The public interface should be connected to your physical network so that traffic can be routed in and out of the nested environment. Furthermore, BGP is configured for peering with the Tier-0 Gateway and NAT is enabled for the nested environments management VLAN.

NSX-T logical networking

Leveraging VMware’s new NSX-T 3.0 Ansible modules, the default deployment now provisions NSX-T logical networking.

A Tier-0 Gateway with two external interfaces and BGP configuration for peering with the VyOS router. Of course, everything from AS numbers to IP subnets is customizable.

vCenter not required/used

The entire deployment is now carried out against a standalone physical ESXi host. vCenter is not required and not used.

Miscellaneous improvements

  • Only Ansible and VMware supported modules are used by the deployment. Custom modules have been removed.
  • Improved answerfile.yml (I’m still trying to find the perfect balance between customizability and ease of use).
  • Added undeploy.yml for easy removal of the deployed components.
  • An updated README.md now contains clearer instructions and more information including some diagrams.

Summary

While I’m afraid that this project will never be finished, I am happy enough with the latest improvements to call it a “milestone”.

There are more areas that need attention, but there’s a foundation at least. The playbooks certainly help me when I need to spin up different vSphere/NSX-T environments for testing.

Thanks for reading.

10 Comments

  1. Lalit
    Permalink

    When i am running deploy.yml. Its stucking here for a long time.

    TASK [Perform vCenter CLI-based installation] **********************************
    task path: /root/vsphere-nsxt-lab-deploy/playbooks/deployVC.yml:32
    Wednesday 29 April 2020 11:20:42 +0000 (0:00:00.165) 0:03:06.096 *******
    ESTABLISH LOCAL CONNECTION FOR USER: root
    EXEC /bin/sh -c ‘echo ~root && sleep 0’
    EXEC /bin/sh -c ‘( umask 77 && mkdir -p “` echo /root/.ansible/tmp `”&& mkdir /root/.ansible/tmp/ansible-tmp-1588159243.1004767-4152-106655190239341 && echo ansible-tmp-1588159243.1004767-4152-106655190239341=”` echo /root/.ansible/tmp/ansible-tmp-1588159243.1004767-4152-106655190239341 `” ) && sleep 0’
    Using module file /usr/local/lib/python3.6/dist-packages/ansible/modules/commands/command.py
    PUT /root/.ansible/tmp/ansible-local-356458vwl3fy/tmpnop2q_ty TO /root/.ansible/tmp/ansible-tmp-1588159243.1004767-4152-106655190239341/AnsiballZ_command.py
    EXEC /bin/sh -c ‘chmod u+x /root/.ansible/tmp/ansible-tmp-1588159243.1004767-4152-106655190239341/ /root/.ansible/tmp/ansible-tmp-1588159243.1004767-4152-106655190239341/AnsiballZ_command.py && sleep 0’
    EXEC /bin/sh -c ‘/usr/bin/python3 /root/.ansible/tmp/ansible-tmp-1588159243.1004767-4152-106655190239341/AnsiballZ_command.py && sleep 0’

    Like

    Reply
  2. Darek
    Permalink

    Hello Rutger,
    Great idea. I was looking for that. Thank you πŸ™‚
    I’m struggling to run all updated scripts – without success -still. I don’t know the Ansible: (… Are you able to identify my configuration issue ? Is it related to Ansible / Python or mistake in configuration of VyOS?

    TASK [Deploy VyOS router
    Traceback (most recent call last):
    File “/root/.ansible/tmp/ansible-tmp-1588074996.8718963-9270-263842927263233/AnsiballZ_vmware_deploy_ovf.py”, line 102, in
    _ansiballz_main()
    File “/root/.ansible/tmp/ansible-tmp-1588074996.8718963-9270-263842927263233/AnsiballZ_vmware_deploy_ovf.py”, line 94, in _ansiballz_main
    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)
    File “/root/.ansible/tmp/ansible-tmp-1588074996.8718963-9270-263842927263233/AnsiballZ_vmware_deploy_ovf.py”, line 40, in invoke_module
    runpy.run_module(mod_name=’ansible.modules.cloud.vmware.vmware_deploy_ovf’, init_globals=None, run_name=’__main__’, alter_sys=True)
    File “/usr/lib/python3.6/runpy.py”, line 205, in run_module
    return _run_module_code(code, init_globals, run_name, mod_spec)
    File “/usr/lib/python3.6/runpy.py”, line 96, in _run_module_code
    mod_name, mod_spec, pkg_name, script_name)
    File “/usr/lib/python3.6/runpy.py”, line 85, in _run_code
    exec(code, run_globals)
    File “/tmp/ansible_vmware_deploy_ovf_payload_vj6v0cn6/ansible_vmware_deploy_ovf_payload.zip/ansible/modules/cloud/vmware/vmware_deploy_ovf.py”, line 704, in
    File “/tmp/ansible_vmware_deploy_ovf_payload_vj6v0cn6/ansible_vmware_deploy_ovf_payload.zip/ansible/modules/cloud/vmware/vmware_deploy_ovf.py”, line 696, in main
    File “/tmp/ansible_vmware_deploy_ovf_payload_vj6v0cn6/ansible_vmware_deploy_ovf_payload.zip/ansible/modules/cloud/vmware/vmware_deploy_ovf.py”, line 484, in upload
    File “/tmp/ansible_vmware_deploy_ovf_payload_vj6v0cn6/ansible_vmware_deploy_ovf_payload.zip/ansible/modules/cloud/vmware/vmware_deploy_ovf.py”, line 389, in get_lease
    File “/tmp/ansible_vmware_deploy_ovf_payload_vj6v0cn6/ansible_vmware_deploy_ovf_payload.zip/ansible/modules/cloud/vmware/vmware_deploy_ovf.py”, line 361, in get_objects
    KeyError: ‘network’
    fatal: [127.0.0.1 -> localhost]: FAILED! => {
    “changed”: false,
    “module_stderr”: “Traceback (most recent call last):\n File \”/root/.ansible/tmp/ansible-tmp-1588074996.8718963-9270-…………..

    Thank you in advance.
    Best Regards
    Darek

    Like

    Reply
  3. Darek
    Permalink

    Rutger,
    Thank you.
    It was. Currently vcenter (installed on this host) is down. Only DC (nested)an Linux (VMNetwork) with Ansible are working.

    Like

    Reply
  4. Kate
    Permalink

    Hello Rutger,

    I need a little bit help πŸ™‚

    I stopped adding license

    TASK [Add NSX-T license]
    fatal: [localhost]: FAILED! => {“changed”: false, “msg”: “Failed to add license. Request body [{\”license_key\”: \”********\”}]. Error[].”}

    Is this a regular txt file -nsxlicense.txt ?
    The key included should look like XXXX-XXXX-………. without any additional characters ?

    Thank you in advance

    Kate

    Like

    Reply
    • rutgerblom
      Permalink

      Hi Kate,
      That’s correct. Just a text file with the key, no spaces. FYI We’re working on an improved process for handling licenses. 😊

      Like

      Reply
  5. Kate
    Permalink

    Hello Rutger,

    Thank you for confirmation.
    Problem with uploading the license (in my case) was related to temporary lack of resources (NSX VM was lstarting services too long)

    Next question…
    vsan: true # vSAN requires at least three ESXi hosts in the cluster – OK
    but each host shoud have 1 cache and 2 capacity (minimum) disks to build vSAN storage.

    Here we have only 1+1 declared

    boot_disk_size: “8”
    vsan_cache_size: “20”
    vsan_capacity_size: “180”

    If it’s initial preparation – that’s OK

    Thank you in advance

    Like

    Reply
    • rutgerblom
      Permalink

      Hi Kate,
      vSAN requires 1 cache and 1 capacity disk minimum. The default deployment takes care of this.

      Cheers

      Like

      Reply
  6. Kate
    Permalink

    Hi Rutger,

    Yes, of course ! Sorry :).
    I don’t know where this idea came from πŸ˜›

    BR

    Like

    Reply
  7. SDDC.Lab v2 |
    Permalink

    […] after publishing my second post about this project Luis Chanu, who already was somewhat of a “trusted advisor”, stepped […]

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.