Automate Ubuntu Server 20.04 Installation with Ansible

Posted by

And now something completely different. Recently, while working on a project, I had to come up with a way to automate Ubuntu Server 20.04 VM installations on vSphere. Utilizing Ubuntu’s new autoinstall method together with some Ansible code I managed to get something up and running. Decent enough to share it with you.

Overview

The Playbook I’m showcasing in this article will carry out the following operations:

  1. Download the Ubuntu Server 20.04 ISO
  2. Modify ISO contents to enable unattended installation
  3. Upload the modified ISO to a vSphere datastore
  4. Create a VM and install Ubuntu Server 20.04 using the modified ISO
  5. Configure static IP on the Ubuntu Server

Step 5 is required because for some reason Ubuntu’s autoinstall reverts the network configuration to DHCP after rebooting the server. I hope this will be fixed in a future release.

Requirements

To run this Playbook you will need the following:

  • An Ubuntu 20.04 machine with the following:
    • sudo apt update && sudo apt install python3 python3-pip git xorriso
    • git clone https://github.com/rutgerblom/ubuntu-autoinstall.git ~/git/ubuntu-autoinstall
    • pip3 install –upgrade -r ~/git/ubuntu-autoinstall/pip_requirements.txt
    • ansible-galaxy collection install –upgrade -r ~/git/ubuntu-autoinstall/requirements.yml
    • Access to your VMware vSphere environment
  • vSphere 6.7 or higher

Playbook

Below are the contents of the Ansible Playbook for reference. This together with the supporting files is actually better viewed and cloned on Github.

---
- hosts: localhost
  name: DeployUbuntu.yml
  gather_facts: false
  vars:
    workingdir: "{{ lookup('env','HOME') }}/ubuntu-autoinstall"   # Temp directory on the Ansible Controller
    ubuntuiso: ubuntu-20.04.2-live-server-amd64.iso               # Ubuntu ISO filename
    ubuntuiso_url: http://old-releases.ubuntu.com/releases/20.04/ # Ubuntu ISO URL
    ubuntunewiso: ubuntu.iso                                      # Ubuntu custom ISO name
    vcenterserver: vcenter.demo.local                             # vCenter FQDN
    vcenteruser: administrator@vsphere.local                      # vCenter username
    vcenterpassword: VMware1!                                     # vCenter password
    datacenter: SDDC                                              # vCenter datacenter
    vspherecluster: Lab-Cluster                                   # vCenter cluster
    vmfolder: /                                                   # vCenter VM folder
    datastore: Shared_VMs                                         # vSphere datastore
    datastoredir: /ISO                                            # vSphere datastore ISO directory
    ubuntuvmname: ubuntu-server                                   # Ubuntu VM name of the virtual machine
    ubuntuvmdisksize: 50                                          # Ubuntu VM disksize in gigabytes
    ubuntuvmmemorysize: 2048                                      # Ubuntu VM memory size in megabytes
    ubuntuvvmcpus: 1                                              # Ubuntu VM number of CPUs
    ubuntuvmcpucores: 1                                           # Ubuntu VM number of cores
    ubuntuvmportgroup: VLAN-301                                   # Ubuntu VM vSphere portgroup
    ubuntuoslocale: en_US                                         # Ubuntu OS locale
    ubuntuoskeyboardlayout: en                                    # Ubuntu OS keyboard layout
    ubuntuoskeyboardvariant: us                                   # Ubuntu OS keyboard variant
    ubuntuosipv4address: 10.203.0.50/24                           # Ubuntu OS IPv4 address
    ubuntuosipv4gateway: 10.203.0.1                               # Ubuntu OS IPv4 gateway
    ubuntuosipv4dns: 10.203.0.5                                   # Ubuntu OS DNS server
    ubuntuossearchdomain: sddc.lab                                # Ubuntu OS DNS search domain
    ubuntuoshostname: ubbuntu-server                              # Ubuntu OS hostname
    ubuntuosuser: ubuntu                                          # Ubuntu OS username
    ubuntuospassword: VMware1!                                    # Ubuntu OS password

  tasks:
    - name: Create working directory on Ansible Controller
      ansible.builtin.file:
        path: "{{ workingdir }}"
        state: directory
        mode: "755"


    - name: Check if ubuntu ISO exists locally on Ansible Controller
      ansible.builtin.stat:
        path: "{{ workingdir }}/{{ ubuntuiso }}"
      register: installerfilecheck


    - name: Download ubuntu ISO (if ISO file doesn't exist locally)
      ansible.builtin.get_url:
        url: "{{ ubuntuiso_url }}{{ ubuntuiso }}"
        dest: "{{ workingdir }}/{{ ubuntuiso }}"
        mode: "755"
      when:
        - not installerfilecheck.stat.exists


    - name: Extract Ubuntu ISO
      ansible.builtin.command: "xorriso -osirrox on -indev {{ workingdir }}/{{ ubuntuiso }} \
                               -extract / {{ workingdir }}/iso"
      changed_when: false


    - name: Add write permission to extracted files
      ansible.builtin.command: "chmod -R +w {{ workingdir }}/iso"    # Using chmod as Ansible (Python) can't handle the recursion depth on the Ubuntu ISO
      changed_when: false


## Start workaround issue with Ubuntu autoinstall
## Details of the issue and the workaround: https://askubuntu.com/questions/1394441/ubuntu-20-04-3-autoinstall-with-embedded-user-data-crashing-i-got-workaround

    - name: Extract the Packages.gz file on Ubuntu ISO
      ansible.builtin.command: "gunzip -f {{ workingdir }}/iso/dists/focal/main/binary-amd64/Packages.gz --keep"
      changed_when: false

## End workaround issue with Ubuntu autoinstall


    - name: Copy txt.cfg from ubuntu ISO
      ansible.builtin.copy:
        src: "{{ workingdir }}/iso/isolinux/txt.cfg"
        dest: "{{ workingdir }}/isocopy/isolinux/"
        mode: "775"


    - name: Edit txt.cfg to modify append line
      ansible.builtin.replace:
        dest: "{{ workingdir }}/isocopy/isolinux/txt.cfg"
        regexp: 'append   initrd=/casper/initrd quiet  ---'
        replace: 'append   initrd=/casper/initrd quiet --- autoinstall ds=nocloud;s=/cdrom/autoinstall/'


    - name: Create directory to store user-data and meta-data
      ansible.builtin.file:
        path: "{{ workingdir }}/isocopy/autoinstall"
        state: directory
        mode: "755"


    - name: Copy user-data file to directory
      ansible.builtin.template:
        src: ./Ubuntu_user-data.j2
        dest: "{{ workingdir }}/isocopy/autoinstall/user-data"
        mode: "755"


    - name: Create empty meta-data file in directory
      ansible.builtin.file:
        path: "{{ workingdir }}/isocopy/autoinstall/meta-data"
        state: touch
        mode: "755"


    - name: Create custom ubuntu ISO
      ansible.builtin.command: "xorrisofs -relaxed-filenames -J -R -o {{ workingdir }}/{{ ubuntunewiso }} \
                               -b isolinux/isolinux.bin -c isolinux/boot.cat -no-emul-boot -boot-load-size 4 -boot-info-table \
                               {{ workingdir }}/iso/ {{ workingdir }}/isocopy/"
      args:
        chdir: "{{ workingdir }}/isocopy/"
      changed_when: false


    - name: Upload the custom ubuntu ISO to the vSphere datastore
      community.vmware.vsphere_copy:
        hostname: "{{ vcenterserver }}"
        username: "{{ vcenteruser }}"
        password: "{{ vcenterpassword }}"
        validate_certs: false
        datacenter: "{{ datacenter }}"
        src: "{{ workingdir }}/{{ ubuntunewiso }}"
        datastore: "{{ datastore }}"
        path: "{{ datastoredir }}/{{ ubuntunewiso }}"


    - name: Deploy ubuntu VM
      community.vmware.vmware_guest:
        hostname: "{{ vcenterserver }}"
        username: "{{ vcenteruser }}"
        password: "{{ vcenterpassword }}"
        validate_certs: false
        name: "{{ ubuntuvmname }}"
        state: poweredon
        guest_id: ubuntu64Guest
        cluster: "{{ vspherecluster }}"
        datacenter: "{{ datacenter }}"
        folder: "{{ vmfolder }}"
        disk:
          - size_gb: "{{ ubuntuvmdisksize }}"
            type: thin
            datastore: "{{ datastore }}"
        hardware:
          memory_mb: "{{ ubuntuvmmemorysize }}"
          num_cpus: "{{ ubuntuvvmcpus }}"
          num_cpu_cores_per_socket: "{{ ubuntuvmcpucores }}"
          scsi: paravirtual
        networks:
          - name: "{{ ubuntuvmportgroup }}"
            device_type: vmxnet3
        cdrom:
          - controller_number: 0
            unit_number: 0
            type: iso
            iso_path: "[{{ datastore }}] {{ datastoredir }}/{{ ubuntunewiso }}"
            state: present
        annotation: |
                    *** Auto-Deployed by Ansible ***
                    Username: {{ ubuntuosuser }}
                    Password: {{ ubuntuospassword }}


    - name: Start checking if the ubuntu VM is ready
      community.vmware.vmware_guest_info:
        hostname: "{{ vcenterserver }}"
        username: "{{ vcenteruser }}"
        password: "{{ vcenterpassword }}"
        datacenter: "{{ datacenter }}"
        validate_certs: false
        name: "{{ ubuntuvmname }}"
        schema: vsphere
      register: vm_facts
      until: vm_facts.instance.guest.hostName is search(ubuntuoshostname)
      retries: 30
      delay: 60


    - name: Set password for the ubuntu user
      community.vmware.vmware_vm_shell:
        hostname: "{{ vcenterserver }}"
        username: "{{ vcenteruser }}"
        password: "{{ vcenterpassword }}"
        validate_certs: false
        vm_id: "{{ ubuntuvmname }}"
        vm_username: "{{ ubuntuosuser }}"
        vm_password: VMware1!
        vm_shell: /usr/bin/echo
        vm_shell_args: "'{{ ubuntuosuser }}:{{ ubuntuospassword }}' | sudo chpasswd"


    - name: Copy network configuration file to working directory
      ansible.builtin.template:
        src: ./ubuntu_Netplan.j2
        dest: "{{ workingdir }}/00-installer-config.yaml"
        mode: "755"


    - name: Copy network configuration file to ubuntu VM
      community.vmware.vmware_guest_file_operation:
        hostname: "{{ vcenterserver }}"
        username: "{{ vcenteruser }}"
        password: "{{ vcenterpassword }}"
        validate_certs: false
        vm_id: "{{ ubuntuvmname }}"
        vm_username: "{{ ubuntuosuser }}"
        vm_password: VMware1!
        copy:
          src: "{{ workingdir }}/00-installer-config.yaml"
          dest: "/home/{{ ubuntuosuser }}/00-installer-config.yaml"


    - name: Move network configuration file to right location on ubuntu VM
      community.vmware.vmware_vm_shell:
        hostname: "{{ vcenterserver }}"
        username: "{{ vcenteruser }}"
        password: "{{ vcenterpassword }}"
        validate_certs: false
        vm_id: "{{ ubuntuvmname }}"
        vm_username: "{{ ubuntuosuser }}"
        vm_password: VMware1!
        vm_shell: /usr/bin/sudo
        vm_shell_args: "mv /home/{{ ubuntuosuser }}/00-installer-config.yaml /etc/netplan/00-installer-config.yaml"


    - name: Appply the network configuration on ubuntu VM
      community.vmware.vmware_vm_shell:
        hostname: "{{ vcenterserver }}"
        username: "{{ vcenteruser }}"
        password: "{{ vcenterpassword }}"
        validate_certs: false
        vm_id: "{{ ubuntuvmname }}"
        vm_username: "{{ ubuntuosuser }}"
        vm_password: VMware1!
        vm_shell: /usr/bin/sudo
        vm_shell_args: netplan apply


    - name: Delete working directory on Ansible Controller
      ansible.builtin.file:
        path: "{{ workingdir }}"
        state: absent

Most of the tasks here are pretty self-explanatory. I’ve also tried to use descriptive names for each task to help you understand what is happening.

You will change some of the values of the variables defined under vars: so that they match your environment. Besides that it’s pretty much good to go.

Usage

Modify DeployUbuntu.yml so that the values of the variables match your environment and requirements:

Now run the Playbook with:

ansible-playbook DeployUbuntu.yml

The deployment kicks off and will take about 15 minutes depending on your environment:

And the result is a shiny new Ubuntu Server 20.04 VM:

Not too hard. If Ubuntu’s autoinstall would leave the networking configuration in place after reboot, I would be able to shave off 5 tasks and more than 10 minutes from this deployment. Perhaps somebody out there already knows of a way to handle this? Let me know in the comments.

Summary

In this article I showcased a very basic single-purpose Ansible Playbook for unattended deployment of an Ubuntu Server 20.04 VM on VMware vSphere. This can of course be expanded upon and become part of a larger (automation) process. Hopefully you’ll find this useful and can serve as some inspiration for your projects.

Thanks for reading.

References and resources:

22 comments

  1. thanks for the post i tried the same it was work fine i tried the same with the desktop version but it was not working as expected even i tried with pressed files but it was also not working if you can help me on the same it will be great i tried on Ubuntu forums but i’m not able to find any solution for the same

    Like

  2. Hi,
    a noob question: how are the LOCAL_UbuntuOS* variables passed inside the VM with LOCA_ stripped away, since they are used within the .j2 files for netplan and user data? BTW, nice guide!

    Like

  3. It work for a few year but now with exsi 7.0 is does’t work anymore. @Rutger blom. Does It stil works for you?
    During handling of the above exception, another exception occurred:

    Like

      1. Thanks for replying, but, there is no specific folder and all my VMs are in the root directory. I don’t see any directories. How can I find it? Thanks for your help.

        Like

  4. Sorry for taking your time. I did not HA cluster configuration and only there are 3 ESXi hosts managed by Vcenter server 7. I cannot see any folder name and before you refer to the like, I’ve already seen it and am still confused about the Folder name. 😦

    Like

    1. They changed the layout of the ISO in 22.04 and the custom ISO needs to be prepared differently.
      I haven’t had the time to look into it.
      To be clear, Ubuntu AutoInstall works fine in 22.04 it’s just my script is not compatible. 😊

      Like

  5. this is a great guide, thanks. on the point of running this just for an esxi host i have the same issue as Mahmood, no matter what the folder var is set to /ha-datacenter/vm is always prepended to the file path. Can you share a solution for this @Mahmood? the error is “msg”: “No folder /demo-server matched in the search path : /ha-datacenter/vm/demo-server”. absolute filepath to vm is /vmfs/datastore1/demo-server

    Like

    1. Hi Hoang, the current data structure and VM provisioning task only support one VM to be deployed at a time, but this could easily be updated so that the data is organized in a list or dictionary containing multiple VM entries. The VM provisioning task would then loop through these entries.
      If I find the time I will update the script and data so that it supports deployment of multiple VMs.

      Like

      1. Many thanks to Rutgerblom. I finished auto deploy VMs using Gitlabci and ansible 😀
        If anyone has some issues, i will support :D.
        I have some notes about this
        – I use https://github.com/rutgerblom/ubuntu-autoinstall to make VM and convert to template
        – After that , i deployed multiple VM from template with parameter template (ansible module vmware_guest already support deploy from template)
        – Problem with VMs when you deploy with template you can get is: Network adapter will be disconnected ( i wrote playbook change state from disconnected to connected using powershell )
        – When i upload ISO to VMware Vsan datastore , i cannot browse this ISO file but VMs can see and deploy it (this thing is inconvenient so i decide to deploy base VM on ESX host and convert it to template).

        Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.