Moving your VMWare virtual machines to the GIG.tech edge cloud

Pre-face

In this tutorial we will highlight the differrent steps needed for migrating VMWare virtual machines to the GIG.tech edge cloud:

  • Preparing the VM with the necessary drivers needed for running in the GIG.tech edge cloud.
  • Exporting it from VMWare.
  • Uploading it to a MinIO S3 appliance running in the GIG.tech edge cloud.
  • Importing it into the GIG.tech edge cloud using the cloudapi from the command line.

Acknowledgements

  • This tutorial has not been tested for 32-bit virtual machines

Prepare VM

"Virtio drivers are paravirtualized device drivers for KVM virtual machines."

We need to install Virtio drivers on the virtual machines, while they are still running on the VMWare hypervisor. Most virtual machines do not have these drivers installed by default. Having the Virt-IO drivers installed is needed to run virtual machine on the KVM hypervisor used by the GIG.tech edge cloud.

Windows virtual machines

To make life simple for you we created a Powershell script that automatically uninstalls the VMWare tools and installs the Virt-IO drivers needed for running your Windows virtual machine in the GIG.tech edge cloud.

Open a Powershell window in your windows virtual machine and paste the following lines to get your windows machine ready for running in the GIG.tech edge cloud.

PS C:\Users\Administrator> Invoke-RestMethod -Uri https://raw.githubusercontent.com/gig-tech/utils/master/windows/scripts/win12-16-19/vmware-prepare.ps1 -Method Get -OutFile prepare.ps1
PS C:\Users\Administrator> .\prepare.ps1

The script will then ask you to specify the version of your Windows virtual machine. If your version is not mentioned, it means that the script is still under development for the Windows version of your virtual machine. When writing this tutorial the following versions of Windows were supported:

  • Windows Server 2012
  • Windows Server 2012 R2
  • Windows Server 2016

After the script has finished you should see something as follows in your powershell:

powershell

Linux virtual machines

Preparing linux virtual machines is not really needed as they come with the Virt-IO drivers as part of the linux kernel. If your linux virtual machine would be running a special linux kernel without the Virt-IO drivers, you would need to install them first before exporting them to OVA. How to do this is beyond the scope of this tutorial as it will be very specific for every linux distribution.

Windows & Linux

Now shut down the virtual machine to make it ready to be exported to OVA (Open Virtual Appliance).

Convert images

Once all necessary drivers are installed VMs are ready for export and can be converted to the platform-independent format OVA. Note that all VMs should be powered off prior to converting. One way to convert VMs to an OVA virtual appliance is it use the GUI VMware tools vSphere Client. Another way is to use the command based OVF tool utility. For user guide and downloads see OVF Tool Documentation. OVF tool converts .vmx files of the VM as an argument into the OVA format:

For this tutorial we are using a single ESXi server. The VMWare software allows us to export the VM via the web browser:

export

export popup

After the download is complete we go to work with ovftool to create the ova file:

Geerts-Air:~ geert$ 
Geerts-Air:~ geert$ cd Downloads/
Geerts-Air:Downloads geert$ ls
disk-1.vmdk    disk-2.vmdk    win.ova        win16.mf    win16.nvram    win16.ovf
Geerts-Air:Downloads geert$ touch win16.nvram
Geerts-Air:Downloads geert$ ovftool win16.ovf win.ova
Opening OVF source: win16.ovf
The manifest validates
Warning:
 - Line -1: Unsupported value 'tools.guest.desktop.autolock' for attribute 'key' on element 'ExtraConfig'.
 - Line -1: Unsupported value 'pciBridge0.present' for attribute 'key' on element 'ExtraConfig'.
 - Line -1: Unsupported value 'svga.present' for attribute 'key' on element 'ExtraConfig'.
...
 - Line -1: Unsupported value 'vmware.tools.requiredversion' for attribute 'key' on element 'ExtraConfig'.
 - Line -1: Unsupported value 'migrate.hostLog' for attribute 'key' on element 'ExtraConfig'.
Opening OVA target: win.ova
Writing OVA package: win.ova
Transfer Completed                    
Warning:
 - No manifest entry found for: 'win16.ovf'.
 - File is missing from the manifest: 'win16.ovf'.
 - No manifest entry found for: 'win16.nvram'.
 - Wrong file size specified in OVF descriptor for 'disk-1.vmdk' (specified: 0, actual 9485831168).
 - Wrong file size specified in OVF descriptor for 'disk-2.vmdk' (specified: 0, actual 73216).
 - No manifest entry found for: 'win16.nvram'.
Completed successfully
Geerts-Air:Downloads geert$ ls
disk-1.vmdk    disk-2.vmdk    win.ova        win16.mf    win16.nvram    win16.ovf
Geerts-Air:Downloads geert$

Alright, now that we have the virtual machine in OVA format, we can go to the next step.

Upload images

OpenvCloud (the GIG.tech software running on the G8's that are delivering the IAAS stack of the GIG.tech edge cloud) can import images from a webdav server but also from S3. Importing from webdav has already been supported for a long time, but we found that when the OVA images get bigger, customers tend to suffer from timeouts while importing their VM's into the GIG.tech edge cloud. To address this we added support for importing VM's from S3 starting from OpenvCloud version 2.5.1. The advantage of S3 is that, under the hood, it downloads large files in chunks and also provides resumable downloads in case of network interruptions.

If you do not have an S3 server at your disposal, you can easily deploy a MinIO appliance on one of the G8s. Deployment steps are covered in tutorial Setup MinIO Appliance on GIG Edge Cloud with Terraform and Ansible

For this tutorial we will use a MinIO appliance deployed in the turorial mentioned above using the mc cli utility you can download from https://min.io/download

We start by creating a bucket on the minio appliance:

Geerts-Air:Downloads geert$ mc mb myMinioOnGIG/vmware-exports
Bucket created successfully `myMinioOnGIG/vmware-exports`.

Then we copy the ova file to the bucket:

Geerts-Air:Downloads geert$ mc cp win.ova myMinioOnGIG/vmware-exports

Note The tutorial performs as much as possible via cli, because this makes you able to script the different actions to automate large migrations of virtual machines to the GIG.tech edge cloud.

Importing the OVA into the GIG.tech edge cloud

Pre-requisites for this step

The next steps of the tutorial are being done from the cli. Note that importing an OVA is also possible from the end user portal of OpenvCloud.

Prepare the environment in your shell.

  • Export your JWT as $JWT in the environment variables of your shell
  • Export the url to the G8 you are importing to into the $URL environment variable. In my case this is:
    export URL="https://ch-lug-dc01-002.gig.tech"
  • Install the jq cli utility for parsing JSON output from the API. See https://stedolan.github.io/jq/

Getting the cloudspace id

Before you can start importing you need to know the id of your cloudspace. You can list the cloudspaces to which your JWT has access using the following command:

Geerts-Air:Downloads geert$ curl --silent -X POST -H "Authorization: bearer ${JWT}" -F "includedeleted=False" "${URL}/restmachine/cloudapi/cloudspaces/list" | jq -r '.[] | "\(.id)\t\(.accountName)/\(.name)"'
1958    minio/minio
1964    minio/vmware
Geerts-Air:Downloads geert$

Note Try the previous command without piping the output of the curl command to the jq cli utility. It will show you the complete output of the API result.

Let's also discuss the parameters we passed to the curl command in our shell:

  • --silent to omit curl to output any progress or other non important information
  • -X POST to make curl do an http POST request
  • -H "Authorization: bearer ${JWT}" passes an authentication header using our JWT
  • -F "includedeleted=False" passes the includedeleted argument to the API call

All these arguments are then POST'ed to the URI of the API resource to list cloudspaces, which is "${URL}/restmachine/cloudapi/cloudspaces/list"

As you can see in the output of the API call, the cloudspace vmware, which I created for the purpose of this tutorial has id 1964.

Now that we know the id of our cloudspace we can go ahead and trigger the import. For my case I performed the following API call:

Geerts-Air:Downloads geert$ curl --silent -X POST -H "Authorization: bearer ${JWT}" -F "link=https://s3.myminio-on-gig.com" -F "key=*********" -F "secret=******************" -F "region=us-east-1" -F "bucket=vmware-exports" -F "objectName=win.ova" -F "cloudspaceId=1964" -F "name=Windows Server 2016" -F "memory=8192" -F "vcpus=8" -F "privateip=10.101.108.66" -F "_async=True" "${URL}/restmachine/cloudapi/machines/importS3"

"ee5a0252-4b95-4d32-93d7-da15322db4f4"

This command launches the import process and returns a taskguid allowing us to poll for its completion.

Before we proceed lets discuss the arguments passed into the API call:

  • -F "link=https://s3.myminio-on-gig.com" passes the link to the MinIO appliance on which I previously uploaded my OVA image.
  • -F "key=*********" passes the key for authenticating to my MinIO appliance. This key should have at least read rights to the bucket in which the OVA file is uploaded.
  • -F "secret=******************" passes the secret for authenticating to the api matching the key passed in the previous argument.
  • -F "region=us-east-1" passes the region of the S3 bucket. For a MinIO appliance this is us-east-1 by default.
  • -F "bucket=vmware-exports" passes the name of the bucket in which I previously uploaded my OVA image.
  • -F "objectName=win.ova" passes the name of the file I uploaded to the bucket.
  • -F "cloudspaceId=1964" passes the id of the cloudspace to which the virtual machine should be imported.
  • -F "name=Windows Server 2016" passes the name for the imported virtual machine.
  • -F "memory=8192" passes how much memory the imported virtual machine should have in MiB.
  • -F "vcpus=8" passes how much virtual cpu's the imported virtual machine should have.
  • -F "privateip=10.101.108.66" passes the ip address that should be provisioned to the virtual machine via dhcp. Note that this is optional.
  • -F "_async=True" makes the API call asynchronous which is desired for long running tasks like this one.

So now we have to wait until the import process finishes. Checking the status is done with the following system API call using the taskguid. Check the result of the previous API call and you'll notice that it returned "ee5a0252-4b95-4d32-93d7-da15322db4f4". This is the taskguid of the ongoing import process which we now can use to check the status of the import:

Geerts-Air:Downloads geert$ curl --silent -X POST -H "Authorization: bearer ${JWT}" -F "taskguid=ee5a0252-4b95-4d32-93d7-da15322db4f4" "${URL}/restmachine/system/task/get"
Geerts-Air:Downloads geert$

As you can see we get an empty result. This means that the import process is still ongoing. Keep polling this API until you actually get a result returned from the API call. If you get something like the following, it means that the import process has finished:

Geerts-Air:Downloads geert$ curl --silent -X POST -H "Authorization: bearer ${JWT}" -F "taskguid=ee5a0252-4b95-4d32-93d7-da15322db4f4" "${URL}/restmachine/system/task/get"
[true,26902]

So we get an array with two values returned. The first value will always be either true or false indicating success or failure of the API call and the second value is the actual result of the API call, which is in this case the id of the newly imported vm.

Checking the end-user portal gives me the following: console

Conclusion

We moved a Windows VM from an ESXi server to the GIG.tech edge cloud. After the import process we see that the VM is running properly in it's new home. We made use of the command line interface as much as possible which gives you the ability to take the steps of this tutorial and script them to move entire VMWare deployments into the GIG.tech edge cloud.