Automated Virtual Data Center Deployment with Terraform on the GIG Edge Cloud

Introduction

GIG Edge Cloud consists of a number of interconnected G8s - infrastructure platform units, providing capacity on the edge, relying on the OpenvCloud (OVC) software. In similar fashion as for other cloud providers like AWS, Azure etc., infrastructure deployed to OVC can be managed with Terraform. OVC supports Terraform by means of the custom OVC provider allowing creating/updating/destroying such cloud resources like virtual data centers (also referred as cloud spaces), virtual machines, port forwards and virtual disks.

This guide covers how to interact with a G8 unit using Terraform.

Prerequisites

  • Itsyou.online Account. Authentication to OVC is done with itsyou.online identity server. In order to gain access to a G8 you should have an itsyou.online account. To interact with G8 you need either a valid JWT token or Application ID (or client ID) and Secret pair. To generate these credentials on the page of your itsyou.online account create a new API key. In order to generate a token use the following curl request:

    CLIENT_ID="<Your Client ID>"
    CLIENT_SECRET="<Your Client Secret>"
    export TF_VAR_client_jwt=$(curl --silent -d 'grant_type=client_credentials&client_id='"$CLIENT_ID"'&client_secret='"$CLIENT_SECRET"'&response_type=id_token&scope=offline_access' https://itsyou.online/v1/oauth/access_token)
  • Access on the G8 Account. Ensure that your itsyou.online user is added to the G8 account where you are going to deploy your infrastructure. Request your account administrator to provide you with access.

Step 1 - Install Terraform

Terraform version used in this tutorial: 0.11.14, OVC provider version: 1.0.0. Please make sure that you always use compatible Terraform and OVC provider binaries.

Official documentation for Terraform installation, see this getting started guide. Download Terraform package appropriate for your system. If needed, download one of the previous versions. For Ubuntu the installation procedure is the following:

curl -O https://releases.hashicorp.com/terraform/0.11.14/terraform_0.11.14_linux_amd64.zip
unzip terraform_0.11.14_linux_amd64.zip -d /usr/bin/
  1. Extract Terraform to a directory.
  2. Add the directory to the PATH.

Verify installation by executing terraform. You should see the Terraform help page appear in your terminal.

Step 2 - Install OVC Terraform Provider

Terraform uses Provider Plugins to interact with specific cloud infrastructures.

The Terraform provider is a binary used as a plugin for Terraform. All providers should be placed, by default, in the plugin folder

~/.terraform.d/plugins

There are two options for obtaining the OVC Terraform provider.

Option 1. Download the released binary from here.

Option 2. Compile the latest version:

  1. Install Go version 1.11 or higher, set up GOPATH:

    export GOPATH=$HOME/go
    export PATH=${GOPATH}/bin:${PATH}
  2. Clone OVC provider and Shared Go code for OVC provider repositories and build the binary

    mkdir -p $GOPATH/src/github.com/gig-tech/
    git clone git@github.com:gig-tech/terraform-provider-ovc.git $GOPATH/src/github.com/gig-tech/terraform-provider-ovc
    cd $GOPATH/src/github.com/gig-tech/terraform-provider-ovc
    make build
  3. Move the binary to the Terraform plugin folder

    mkdir -p ~/.terraform.d/plugins
    mv terraform-provider-ovc ~/.terraform.d/plugins

Step 3 - Terraform Configuration

Configuration files

Terraform configuration can reside in any directory of choice, for example

mkdir ~/terraform_config

Note that each configuration directory can contain only a single set of configuration files. Make sure no irrelevant files with extension .tf are present in the directory, since Terraform will consume all .tf files.

In detail you can learn about Terraform configuration files and Terraform configuration language in the official docs.

In this tutorial we use three types of configuration files:

  • terraform.tfvars - contains the settings we want to use
  • variables.tf - contains definition of the variables and their defaults
  • main.tf - contains the configuration of the infrastructure we want to set up, consists of blocks of different types. In this tutorial we use the following constructs:
    • provider block describes provider configuration, resource types, their arguments and attributes. Each configuration should contain a provider block.
    • resource block describes resource of given type. By adding/changing/deleting this block we can create/update/destroy the resource it is pointing at.
    • data block describes a data resource. This types of blocks is used to export info of one or several existing resources to the local Terraform module where it can be used in other blocks.

Terraform commands

Initialize a working directory. This is a first step after creating new terraform configuration

terraform init

Verify provider. Once configuration folder is created you can verify provider installation, Terraform should list installed provider plugins:

cd ~/terraform_config
$ terraform providers
.
└── provider.ovc

Verify configuration. Configuration can be validated by Terraform within the configuration directory

terraform validate

Plan Execution. Generate and show the execution plan

terraform plan

Apply Changes. Build or change infrastructure

terraform apply

List states. Terraform is stateful, by default state of each resource is stored locally in the configuration directory in file terraform.tfstate. To list all the resources Terraform keeps track of execute

terraform state list

Remove List. Tampering with infrastructure bypassing Terraform can cause conflicts between local state and real state of the system. Sometimes it is necessary to delete saved states and manually delete cloud resources before reapplying the config. To remove all saved states delete file terraform.tfstate; to delete state information of a certain resource execute

terraform state rm <state_name>

Environmental variables

Terraform makes use of a number of environmental variables to customize some aspects of Terraform behavior and to set input variables. For example, if a variable name is expected to be a part of configuration, but is not defined in configuration files Terraform will try to fetch the value from the environmental variable TF_VAR_name.

OVC configuration

Authentication

There are two options how to configure your credentials with OVC provider.

Option 1. (Recommended)

Set JWT as environmental variable

export TF_VAR_client_jwt=""

This option is recommended since does not require exposing client ID and secret combination to your system. Examples in this tutorial assume this way of Authentication.

Option 2. (NOT recommended)

Alternatively it is possible to set client ID and secret as environmental variables, then terraform provider will request JWT token based on these credentials.

export TF_VAR_client_id=""
export TF_VAR_client_secret=""

This option is NOT recommended since requires exposing client ID and secret combination to your system.

Note: Either JWT token or Client ID + Secret should be specified, these options are mutually exclusive. If both options are provided in environmental variables or in the configuration files, Terraform will produce an error.

Using the OVC provider

Below we include several examples how to define OVC resources. Complete description of the OVC resources, their attributes, defaults and usage examples can be found here.

Cloud Space

Create Cloud Space

To create a Cloud Space (CS) Terraform files should include the following configuration:

terraform.tfvars
    server_url = "https://be-g8-demo.gig.tech"
account = "test_account"
cloudspace = "test_cloudspace"
  
variables.tf
    variable "server_url" {
  description = "API server URL"
}
variable "client_jwt" {
  description = "Client JWT"
}
variable "account" {
  description = "account name"
}
variable "cloudspace" {
  description = "cloudspace name"
}
  
main.tf
    provider "ovc" {
  server_url = "${var.server_url}"
  client_jwt="${var.client_jwt}"
}
resource "ovc_cloudspace" "cs" {
  account = "${var.account}"
  name = "${var.cloudspace}"
}
  

View execution plan:

$ terraform plan
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.


------------------------------------------------------------------------

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  + ovc_cloudspace.cs
      id:                  <computed>
      account:             "test_account"
      description:         <computed>
      external_network_ip: <computed>
      location:            <computed>
      name:                "test_cloudspace"
      private_network:     <computed>
      status:              <computed>


Plan: 1 to add, 0 to change, 0 to destroy.

Apply configuration:

$ terraform apply
  An execution plan has been generated and is shown below.
  Resource actions are indicated with the following symbols:
    + create

  Terraform will perform the following actions:

    + ovc_cloudspace.cs
        id:                  <computed>
        account:             "test_account"
        description:         <computed>
        external_network_ip: <computed>
        location:            <computed>
        name:                "test_cloudspace"
        private_network:     <computed>
        status:              <computed>


  Plan: 1 to add, 0 to change, 0 to destroy.

  Do you want to perform these actions?
    Terraform will perform the actions described above.
    Only 'yes' will be accepted to approve.

    Enter a value:

Terraform will always ask to confirm changes, presented in the execution plan. After changes are confirmed, Terraform updates the infrastructure to the desired state and returns the list of changes:

ovc_cloudspace.cs: Creating...
  account:             "" => "test_account"
  description:         "" => "<computed>"
  external_network_ip: "" => "<computed>"
  location:            "" => "<computed>"
  name:                "" => "test_cloudspace"
  private_network:     "" => "<computed>"
  status:              "" => "<computed>"
ovc_cloudspace.cs: Still creating... (10s elapsed)
ovc_cloudspace.cs: Creation complete after 18s (ID: 18290)

Update Cloud Space

To update the CS we need to change and reapply configuration. For example, to set CS limits and change CS name we need to update the block describing CS resource with limit values and new CS name:

terraform.tfvars
    ...
cloudspace = "new_cs_name"
...
  
main.tf
    ...
resource "ovc_cloudspace" "cs" { account = "${var.account}" name = "${var.cloudspace}" resource_limits { max_memory_capacity = 10 max_disk_capacity = 50 max_cpu_capacity = 5 max_num_public_ip = 3 } } ...

View execution plan:

Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.

ovc_cloudspace.cs: Refreshing state... (ID: 18290)

------------------------------------------------------------------------

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  ~ update in-place

Terraform will perform the following actions:

  ~ ovc_cloudspace.cs
      name:                                "test_cloudspace" => "new_cs_name"
      resource_limits.%:                   "0" => "4"
      resource_limits.max_cpu_capacity:    "" => "5"
      resource_limits.max_disk_capacity:   "" => "50"
      resource_limits.max_memory_capacity: "" => "10"
      resource_limits.max_num_public_ip:   "" => "3"


Plan: 0 to add, 1 to change, 0 to destroy.

------------------------------------------------------------------------

Note: You didn't specify an "-out" parameter to save this plan, so Terraform
can't guarantee that exactly these actions will be performed if
"terraform apply" is subsequently run.

For these changes to come into affect, execute again

terraform apply

Viewing Terraform execution plan without changing configuration, should indicate that infrastructure in up-to-date

$ terraform plan
No changes. Infrastructure is up-to-date.

This means that Terraform did not detect any differences between your
configuration and real physical resources that exist. As a result, no
actions need to be performed.

Destroy Cloud Space

Destroying resources with Terraform is done by removing block describing the resource for deletion and reapplying the configuration. In this example we remove CS by removing ovc_cloudspace.cs block. After changes are applied Terraform reports completed actions:

$ terraform apply
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  - destroy

Terraform will perform the following actions:

  - ovc_cloudspace.cs


Plan: 0 to add, 0 to change, 1 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes
ovc_cloudspace.cs: Destroying... (ID: 18290)
ovc_cloudspace.cs: Destruction complete after 8s

Apply complete! Resources: 0 added, 0 changed, 1 destroyed.

Virtual Machine

Create Virtual Machine on existing Cloud Space, preload ssh key, choose image by name

The following set of configuration files serves as a minimal example for creating a Virtual Machine (VM) from a recent Ubuntu 18.04 image with preloaded ssh-key and a port forward:

terraform.tfvars
    # G8 api url
server_url = ""

# OVC Account name, your IYO account must have access to it.
account = "test_account"

# name of a CS on the account where VM should be created
cloudspace = "test_cloudspace"

# VM name
vm_name = "test_vm"

# The description of the VM
vm_description = "test creating VM with Terraform"

# Memory to be provisioned for the VM
memory = 2048

# Number of CPUs to be provisioned for the VM
vcpus = 2

# boot disk size in GB
disksize = 10

# image name regex string
image_name_regex = "(?i).*ubuntu.*18.04"

# User data, contains users and SSH keys to be added to the VM
userdata = "users: [{name: root, shell: /bin/bash, ssh-authorized-keys: [, ]}]"
    
  
variables.tf
    variable "server_url" {
  description = "API server URL"
}
variable "client_jwt" {
  description = "Client JWT"
}
variable "account" {
  description = "account"
}
variable "cloudspace" {
  description = "cloudspace name"
}
variable "vm_name" {
  description = "cloudspace name"
}
variable "vm_description" {
  description = "Description of the VM"
}
variable "image_regex_string" {
  description = "Image name or a regex string to much image name"
}
variable "memory" {
  description = "memory provisioned for the VM"
  default     = 1024
}
variable "vcpus" {
  description = "number of CPUs provisioned for the VM"
  default     = 2
}
variable "disksize" {
  description = "disksize"
  default     = 20
}
variable "userdata" {
  description = "user data"
}
  
main.tf
    provider "ovc" {
  server_url = "${var.server_url}"
  client_jwt="${var.client_jwt}"
}

# Data definition for the cloudspace
# To be able to get the ip address
  data "ovc_cloudspace" "cs" {
  account = "${var.account}"
  name = "${var.cloudspace}"
}

# Data definition for image
  data "ovc_image" "im"{
  most_recent = true
  name_regex = "${var.image_regex_string}"
}

# Definition of the vm to be created with the settings defined in terraform.tfvars
resource "ovc_machine" "mymachine" {
  cloudspace_id = "${data.ovc_cloudspace.cs.id}"
  image_id      = "${data.ovc_image.im.id}"
  disksize      = "${var.disksize}"
  memory        = "${var.memory}"
  vcpus           = "${var.vcpus}"
  name          = "${var.vm_name}"
  description   = "${var.vm_description}"
  userdata      = "${var.userdata}"
}

# Definition of the portforward to be created for the VM
resource "ovc_port_forwarding" "ssh" {
  cloudspace_id = "${data.ovc_cloudspace.cs.id}"
  public_ip     = "${data.ovc_cloudspace.cs.external_network_ip}"
  public_port   = "${var.public_port}"
  machine_id    = "${ovc_machine.mymachine.id}"
  local_port    = "${var.local_port}"
  protocol      = "tcp"
  depends_on    = ["ovc_machine.mymachine"]
}
  

Data source ovc_cloudspace.cs is used to fetch CS ID and CS external IP address. In similar fashion image ID is fetched by image name using ovc_image.im data resource definition.

Resource ovc_port_forwarding.ssh is defined to add a port forward to the VM. Since port forward can be added only after VM was successfully deployed, explicit dependency is added depends_on = ["ovc_machine.mymachine"]. depends_on attribute can be used to serialize creation of resources that have behavioral dependencies. Similar to resources, explicit dependencies can be defined for data resources, but this is not recommended.

Note that most of the time explicit dependencies are not needed, since Terraform can automatically detect dependencies via interpolation expressions and define correct order of execution. Learn more about resource dependencies.

Virtual Disk

Add disks to Virtual Machine

Example below shows how to add two disks to an existing VM. Configuration should include the following blocks:

main.tf
    # Data resource definition of CS to fetch CS ID
data "ovc_cloudspace" "cs" {
   account = "${var.account}"
   name = "${var.cloudspace}"
}

# Data resource definition of VM to fetch VM ID
data "ovc_machine" "machine" {
   cloudspace_id = "${data.ovc_cloudspace.cs.id}"
   name = "${var.machine}"
}

# Definition of the the disk1
resource "ovc_disk" "disk1" {
  machine_id = "${data.ovc_machine.machine.id}"
  disk_name = "terraform_disk_1"
  description = "Disk created by terraform"
  size = 10
  type = "D"
  iops = 2000
}

# Definition of the the disk2
resource "ovc_disk" "disk2" {
  machine_id = "${data.ovc_machine.machine.id}"
  disk_name = "terraform_disk_2"
  description = "Disk created by terraform"
  size = 20
  type = "D"
  iops = 3000
  depends_on = ["ovc_disk.disk1"] # Explicit dependency to serialize disk creation.
}
  

Note that OVC allows only adding one disk at a time to the VM, explicit dependency is used to ensure that disks are added sequentially.

Result of terraform apply:

ovc_disk.disk1: Creating...
  description: "" => "Disk created by terraform"
  disk_name:   "" => "terraform_disk_1"
  iops:        "" => "2000"
  machine_id:  "" => "19471"
  size:        "" => "10"
  type:        "" => "D"
ovc_disk.disk1: Still creating... (10s elapsed)
ovc_disk.disk1: Still creating... (20s elapsed)
ovc_disk.disk1: Still creating... (30s elapsed)
ovc_disk.disk1: Still creating... (40s elapsed)
ovc_disk.disk1: Creation complete after 43s (ID: 26389)
ovc_disk.disk2: Creating...
  description: "" => "Disk created by terraform"
  disk_name:   "" => "terraform_disk_2"
  iops:        "" => "3000"
  machine_id:  "" => "19471"
  size:        "" => "20"
  type:        "" => "D"
ovc_disk.disk2: Still creating... (10s elapsed)
ovc_disk.disk2: Creation complete after 15s (ID: 26390)

Apply complete! Resources: 2 added, 0 changed, 0 destroyed.

To update attributes of the disks, remove corresponding resources from the configuration and reapply configuration.

To destroy one or both disks added to the VM, remove corresponding resources from the configuration and reapply configuration.

Conclusion

This tutorial breaks down how to use Terraform to manage your OVC infrastructure. Practical examples discussed here illustrate basic use cases of cloud deployment. Given Terraform configurations can be easily extended to a larger and more complex systems by adding multiple resources and deploying desired software in an automated way.

To gain more knowledge about automated IaC deployment on GIG Edge Cloud with Terraform please get familiar with Terraform documentation and OVC Provider documentation.