Categories

How To Use Private and Nested Cloudspaces to Deploy a Custom Virtual Firewall

You are here:
  • Main
  • Networking
  • How To Use Private and Nested Cloudspaces to Deploy a Custom Virtual Firewall
< All Topics

Introduction

GIG Cloud-as-a-Service (GIG CaaS) introduces private and nested Cloudspaces to unlock more advanced networking and support adding custom virtual firewalls. A short overview:

  • Public cloudspaces, the only cloudspaces that were available until now and still are the default option, have a public Internet routable IP address and can expose services from Virtual Machines (VMs) via portforwards. The virtual firewall of the public cloudspace also takes care of
    • handing out IP addresses to the VMs via dhcp
    • bootstrapping VMs at first boot through cloudinit
    • acts as the default gateway for the VMs inside the cloudspace
  • Private cloudspace, does the same as the public cloudspace except that it does not have a public Internet routable IP address. Hence it will not provide portforwards nor will it act as a default gateway for the VMs inside the cloudspace. However, just like the public cloudspace, it does hand out IP addresses to the VMs and bootstraps VMs via cloudinit.
  • Nested cloudspace, also does the same as the public cloudspace with the difference that the "public" interface of its virtual firewall is in the private network from its parent cloudspace rather than publicly exposed to the Internet.

This makes it possible for users of the GIG CaaS to introduce a virtual network appliance of their choice to implement the security (anti-ddos, ids, virus scanning, etc.) levels needed, apply advanced firewall configurations and specific routing policies. A user now can deploy a virtual firewall from an image of choice and gain full access to the appliance.

This tutorial shows how to use the new networking features to deploy a Cloudspace using a private Cloudspace and a nested Cloudspace with a custom virtual firewall using an openwrt VM.

Further the tutorial explains how to setup the cloudspaces with a custom virtual gateway using Terraform and to configure the openwrt VM to route external traffic to the other components.

Prerequisites

Terraform version used in this tutorial: 0.11.14, OVC provider version: 1.3.1.

To deploy the infrastructure on the GIG Edge Cloud with Terraform you will need to ensure access to an account, install Terraform and install OVC Terraform Provider. Make sure to use compatible versions of the provider and Terraform. For detailed information on installation, generating access token and creating configurations with Terraform see this tutorial.
If needed, contact the administrator of your account to ensure that all required images are accessible:

  • Template image of OpenWRT to deploy the VFW VM on the private cloudspace (qcow2)
  • Template image of choice to deploy the machines on the nested cloudspace (qcow2)

In case your G8 doesn't already have an openwrt image, you can create it yourself by following the next few steps.

Preparing required image

Prepare the environment in your shell.

  • Export your JWT as $JWT in the environment variables of your shell

  • Export the url to the G8 you are importing to into the $URL environment variable. In my case this is:

    export URL="https://ch-lug-dc01-001.gig.tech"
  • Install the jq cli utility for parsing JSON output from the API. See https://stedolan.github.io/jq/

Getting your Account ID

Before you can start creating an image, you need to know your accountID. You can list the accounts to which your JWT has access using the following command:

kasem@kasem$ curl --silent -X POST -H "Authorization: bearer ${JWT}" "${URL}/restmachine/cloudapi/accounts/list" | jq -r '.[] | "\(.id)\t\(.name)"'
497 Kasem development
kasem@kasem$

Note Try the previous command without piping the output of the curl command to the jq cli utility. It will show you the complete output of the API result.

Let's also discuss the parameters we passed to the curl command in our shell:

  • --silent to omit curl to output any progress or other non important information

  • -X POST to make curl do an http POST request

  • -H "Authorization: bearer ${JWT}" passes an authentication header using our JWT

    All these arguments are then POST'ed to the URI of the API resource to list cloudspaces, which is "${URL}/restmachine/cloudapi/accounts/list"

Now head on to your user portal on your G8 and choose Machine API from the top bar
file

Find your way to the API endpoint by the name /cloudapi/images/create under cloudapi__images and enter the values for following parameters. Needless to say, your values will most likely differ.

file

Parameters:

NOTE: You only need to change the accountID in this API call, however you can use the parameters to customize the image for your needs based on the explanation stated

name: This is the name of the image and typically can be any name of your choice. However you'll need to keep it with the same name that's referenced in the terraform files, which in this case is openwrt so you can leave that as it is.

url: This is where the api will try to fetch the actual image to be used for the machine. We provide you with an url where we host an openwrt-19.07.2 image. You can use this value: https://gig.tech/wp-content/uploads/2020/05/openwrt-19.07.2-x86-64-combined-ext4.qcow2

bootDiskSize: This value should also match the data in the terraform specification. For our case 1 GB should be enough boot disk space for the openwrt image.

imagetype: This Value should be either of Linux, Windows, Darwin, Unix, Other. For our case we know that openwrt is a Linux image so we use Linux.

memory: This is the minimum memory requirement for the image. 64 mb should be enough

boottype: In our case we need bios boot.

accountID: This must be the correct ID of the account that's tied to your user on the G8. This will be different for you; in my case it's 497. In your case this will be the output of the previous step.

Architecture

file

We choose the following design to implement the openwrt VM as the virtual firewall for our virtual datacenter.

The OpenWRT machine is a virtual machine deployed from an openwrt image and attached to an external network. The machine is set as a default gateway of the private cloudspace, acts as a firewall and forwards traffic from the outside to the other components of the deployment.

The Private cloudspace is a cloudspace without a public interface, created to host the OpenWRT machine and in our case a nested cloudspace within its private network. Additional services can run in VMs deployed on private cloudspace.

The Nested cloudspace is a cloudspace with the private cloudspace as its parent. This cloudspace manages its own private network and its virtual firewall has an IP address on the parent network (the private cloudspace). Machines of the nested cloudspace gain access to the Internet via the virtual firewall of the nested cloudspace. The virtual firewall on its own turn forwards the traffic from the Internet to it's own default gateway which is set to the private IP address of the OpenWRT machine. The VMs in the nested cloudspace are meant to run the actual services and applications delivered by this virtual datacenter.

To provide Internet access for the machines in the nested cloudspace, we need to set some rules and redirects on the OpenWRT vm firewall as well as configuring the OpenWRT VM to be the default Gateway of the private cloudspace. We will dive into the details of that later on in the tutorial.

Example

For the sake of illustration let's consider a practical example with the following components:

  • Private cloudspace with its private network deployed behind the virtual gateway
  • Custom network appliance VM based on OpenWRT OS
  • One or several VMs on the private cloudspace in DMZ network
  • Nested cloudspace
  • One or several server VMs on the nested cloudspace

Configuration requirements:

  • Internet traffic is forwarded to the machines on the network of the nested cloudspaces by openwrt machine
  • VMs on private and nested cloudspaces are reachable via SSH

file

Below we provide steps how to deploy and configure such an infrastructure.

Deploy infrastructure with Terraform

Terraform configuration

Find below an example of the Terraform configuration for a cloudspace with a custom virtual gateway. The configuration consists of three files:

  • main.tf describes all the resources that will be deployed and data sources that will be fetched to provide the necessary information like image IDs, external network ID.
  • variables.tf describes all variables used in main.tf
  • terraform.tfvars sets values for all variables. Instead of setting variables in a file, you can set them as environmental variables passed to the terraform process. For example use environmental variable TF_VAR_machine_name to set variable machine_name
main.tf
    
provider "ovc" {
  server_url = "${var.server_url}"
  client_jwt = "${var.client_jwt}"
}
# Data source of server image
data "ovc_image" "default" {
  account = "${var.account}"
  most_recent = true
  name_regex = "(?i).*ubuntu.*18.04.*"
}
data "ovc_image" "openwrt" {
  account = "${var.account}"
  most_recent = true
  name_regex = "openwrt"
}
# Resource definition for the private cloudspace
resource "ovc_cloudspace" "private" {
  account = "${var.account}"
  private_network = "192.168.22.0/24"
  name = "${var.private_cs_name}"
  mode = "private"
}
# Resource definition for the nested cloudspace
resource "ovc_cloudspace" "nested" {
  account = "${var.account}"
  name = "${var.nested_cs_name}"
  mode = "nested"
  private_network = "192.168.2.0/24"
  external_network_id = "${ovc_cloudspace.private.id}"
}
# Resource definition for the openwrt machine
resource "ovc_machine" "openwrt" {
  cloudspace_id = "${ovc_cloudspace.private.id}"
  image_id = "${data.ovc_image.openwrt.id}" # image used for the first boot
  memory        = "${var.openwrt_memory}"
  vcpus         = "${var.openwrt_vcpus}"
  disksize      = "${var.openwrt_disksize}"
  name          = "${var.openwrt_machine_name}"
  description   = "OpenWRT machine deployed with Terraform"
  userdata      = "${var.userdata}"
  act_as_default_gateway = true
}
# Resource definition for a machine on the nested private cloudspace
resource "ovc_machine" "vm_in_dmz" {
  cloudspace_id = "${ovc_cloudspace.private.id}"
  image_id = "${data.ovc_image.default.id}"
  memory = "${var.server_memory}"
  vcpus = "${var.server_vcpus}"
  disksize = "${var.server_disksize}"
  name = "VM-in-DMZ"
  description = "machine deployed on private CS in DMZ network"
}
# Resource definition for a server machine on the nested cloudspace
resource "ovc_machine" "server" {
  cloudspace_id = "${ovc_cloudspace.nested.id}"
  image_id      = "${data.ovc_image.default.image_id}"
  memory        = "${var.server_memory}"
  vcpus         = "${var.server_vcpus}"
  disksize      = "${var.server_disksize}"
  name          = "${var.server_machine_name}"
  description   = "Server machine deployed with Terraform"
  userdata      = "${var.userdata}"
}
# Resource definition for port forward
resource "ovc_port_forwarding" "port_forward_server" {
  cloudspace_id = "${ovc_cloudspace.nested.id}"
  public_ip = "${ovc_cloudspace.nested.external_network_ip}"
  public_port = 2201
  machine_id = "${ovc_machine.server.id}"
  local_port = 22
  protocol = "tcp"
}
output "openwrt-machineId" {
  value = "${ovc_machine.openwrt.id}"
}
output "openwrt-internal-ip" {
  value = "${ovc_machine.openwrt.ip_address}"
}
output "private-cs Id" {
  value = "${ovc_cloudspace.private.id}"
}
output "nested-cs Id" {
  value = "${ovc_cloudspace.nested.id}"
}
output "private-network-private-cs" {
  value = "${ovc_cloudspace.private.private_network}"
}
    
  
variables.tf
    
variable "server_url" {
  description = "API server URL"
}
variable "client_jwt" {
  description = "itsyouonline jwt token"
}
variable "account" {
  description = "account"
}
variable "private_cs_name" {
  description = "name of private cloudspace"
  default = "PRIVATE-CS"
}
variable "nested_cs_name" {
  description = "name of nested cloudspace"
  default = "NESTED-CS"
}
variable "server_disksize" {
  description = "data disk size"
  default = 10
}
variable "server_machine_name" {
  description = "server machine memory"
  default = "SERVER-VM"
}
variable "server_memory" {
  description = "server machine memory"
  default = 2048
}
variable "server_vcpus" {
  description = "number of virtual CPUs on the server machine"
  default = 1
}
variable "openwrt_disksize" {
  description = "data disk size"
  default = 2
}
variable "openwrt_machine_name" {
  description = "server machine memory"
  default = "OpenWRT-VM"
}variable "openwrt_memory" {
  description = "openwrt machine memory"
  default = 64
}
variable "openwrt_vcpus" {
  description = "number of virtual CPUs on the openwrt machine"
  default = 1
}
variable "userdata" {
  description = "user info"
  default = "users: [{name: root, shell: /bin/bash, ssh-authorized-keys: []}]"
}
    
  
terraform.tfvars
    
# OVC Account name, your IYO account must have access to it.
account = ""
# G8 api url
server_url= ""
# User data to be added to the VM 
userdata = "users: [{name: user, shell: /bin/bash, ssh-authorized-keys:[]},{name: root, shell: /bin/bash, ssh-authorized-keys: []}]"
    
  

After applying the configuration, Terraform will print the interfaces of the openwrt machine in the cli output.

Sample output:

nested-cs Id = 1041
openwrt-internal-ip = 192.168.22.254
openwrt-machineId = 21651
private-cs Id = 1040
private-network-private-cs = 192.168.22.0/24

We will need this data in the next steps

Configure OpenWRT VM

Configure network

Now head on to your user portal on your G8 and choose Machine API from the top bar
file

Now go to /cloudapi/cloudspaces/setDefaultGateway under cloudapi__cloudspaces.

Enter cloudspaceID for your Private Cloudspace and your openwrt-internal-ip. In my case, these values are the following.

file

Now go to cloudapi__machines and find the API endpoint by the name /cloudapi/machines/attachExternalNetwork

Enter your openwrt-machineID for machineID parameter. In my case it was 21644. The rest of the parameters are optional and we will not use them.

file

Now your OpenWRT VM has been assigned an external Network IP however this has not been set to the interface on your machine

Get ExternalNetwork Data of OpenWRT VM

We need to obtain 2 pieces of information which are the externalNetwork IP address as well as the gateway. You can get those via the following API call

file

The highlighted area contains that information.

From that we infer that the IP Address = 185.15.201.74

and the gateway should be 185.15.201.65

OpenWRT VM Configurations

Now head to your OpenWRT console on your user portal of the G8 and enter the following commands. You should enter your own values of course. In my case I enter the following based on my data.

$ uci set network.wan.proto='static'
$ uci set network.wan.netmask='255.255.255.192'
$ uci set network.wan.gateway='185.15.201.65'
$ uci set network.wan.ipaddr='185.15.201.74'

$ uci set network.lan.proto='static'
$ uci set network.lan.netmask='255.255.255.0'
$ uci set network.lan.ipaddr='192.168.22.253'

$ uci commit network 
$ service network restart
...
...
$ ping 1.1.1.1
64 bytes from 1.1.1.1: icmp_seq=1 ttl=56 time=212 ms
64 bytes from 1.1.1.1: icmp_seq=2 ttl=56 time=236 ms
64 bytes from 1.1.1.1: icmp_seq=3 ttl=56 time=157 ms
64 bytes.....

your ping command should be successful since now your openWRT has access to externalNetwork.

Now if you do the ping 1.1.1.1 command to check internet access on any VM in NESTED-CS or PRIVATE-CS, it should succeed too since OpenWRT VM is the gateway for the private cs and the private cs is the gateway for the nested cs

Now we have achieved our first requirement. However you won't be able to SSH into your VMs just yet. For that, we need port forwards.

Port Forwarding on OpenWRT

Go back again to the console of your OpenWRT VM and enter the following commands.

Adding a portforward to a neighboring VM is easy since you only need to specify a destination ip, destination port and specify a port on the openWRT as src port. To do that use the following commands. Note that your IP and ports may change depending on your use case. In my case I need ssh so I'll forward openWRT port 2222 to

$ uci add firewall redirect
$ uci set firewall.@redirect[-1].name='ssh-to-neighbor-vm'
$ uci set firewall.@redirect[-1].src='wan'
$ uci set firewall.@redirect[-1].dest_ip='192.168.22.254'
$ uci set firewall.@redirect[-1].dest_port='22'
$ uci set firewall.@redirect[-1].src_dport='2222'
$ uci set firewall.@redirect[-1].proto='tcpudp'

$ uci commit firewall
$ service firewall restart

In order to have ssh access on VMs that are part of the nested cloudspace, you'll need to do the following:

  1. Create a portforward from your VM to your nested cloudspace
  2. Forward a port from OpenWRT to the other end of the port on the cloudspace.

For example.

Nested-VM port :22 <<<----- FORWARDED TO ----->>> NESTED-CS port :2201 <<<----- FORWARDED TO ----->>> openWRT port :2201

Luckily the previous terraform configuration should have created a port forward for one VM inside the Nested CS. Now you only need to forward a port from openWRT VM to the Nested-CS IP with a specific port. To do that you can do the following commands on your OpenWRT VM console.

In my case the nested cloudspace has ip 192.168.22.252

$ uci add firewall redirect
$ uci set firewall.@redirect[-1].name='ssh-to-vm-in-nested-cs'
$ uci set firewall.@redirect[-1].src='wan'
$ uci set firewall.@redirect[-1].dest_ip='192.168.22.252'
$ uci set firewall.@redirect[-1].dest_port='2201'
$ uci set firewall.@redirect[-1].src_dport='2201'
$ uci set firewall.@redirect[-1].proto='tcpudp'

$ uci commit firewall
$ service firewall restart

In order to test your ssh access to the VMs you can use the following command

$ ssh root@185.15.201.74 -p 2201

Conclusion

This tutorial describes how to exploit private and nested cloudspaces to deploy a private network behind a fully customizable virtual firewall. The firewall configuration is illustrated by the example of the OpenWRT OS. The configuration example shows how to enable the Internet access and to allow SSH connection on the virtual machines on both private and nested cloudspaces. Using architectural principles described in this tutorial, users can create various complex network architectures, by adding more nested cloudspaces on a private cloudspace or leveraging from double nested cloudspaces. OpenWRT was used as an example, while any other virtual firewall software can be installed and configured in this setup.

Table of Contents