Terraforming Rancher Deployments on Digital Ocean

on Nov 6, 2017

Having a cool deployment system is pretty neat, but one thing every engineer learns one way or another is that manual processes aren’t processes, they’re chores. If you have to do something more than once, you should automate it if you can. Of course, if the task of automating the process takes longer than the total projected time you’ll spend executing the process, you shouldn’t automate it.

 

XKCD 1205 - Is It Worth the Time?

XKCD 1205 – Is It Worth the Time?

With Rancher, the server can automate many traditionally manual tasks, saving you a lot of time in the long run.  If you’re running Rancher in different environments or even testing it out with different domains or instance sizes, however, you’ll want to automate even the setup and teardown of your Rancher server.

In a prior blog post, I walked through how to build a personal cloud on Digital Ocean with Rancher. In this article, I’m going to outline the advantages of automating a Rancher installation and how to Terraform a Rancher server on Digital Ocean.

Although it is possible to create more complicated configurations using Terraform, this tutorial will follow the basic steps for bootstrapping a docker host on Digital Ocean using Terraform. We will be deploying a single docker container—in this case, Rancher—using one simple line of provisioning. In essence, this tutorial is more generally about Terraforming droplets and basic network configuration in Digital Ocean.

Advantages of Automating Rancher

The Rancher server is one of the simpler deployment tools to configure; it consists of a simple Docker container that bootstraps all the required services and interfaces for managing a cluster with Rancher. Because Rancher runs inside a Docker container, it can be platform agnostic. A standard installation of Rancher doesn’t require any parameters passed to Docker other than a port map, so the local network interface can talk to the Rancher container. This essentially means that all of the Rancher server configuration occurs in the Rancher web interface, not in the command line.

Rancher is ready to go out of the box, so the primary tasks that require automation in a Rancher workflow are spinning up a droplet on Digital Ocean, getting Docker configured, connecting our domain, configuring security credentials, and starting the Rancher docker container. Of course, we also want to be able to undo all these tasks by stopping the instance, removing A-records, and purging security credentials. Thankfully, an incredible automation tool already exists for the automation of infrastructure, and I’m speaking of course of Terraform by HashiCorp.

As a developer who loves side projects, when experimenting with new technologies and tools I like to do it on a budget. I was primarily drawn to Terraform because it enabled me to tear down my infrastructure with a single command when I was finished using it. This allowed me to minimize the impact that my experimentation had on my wallet. Often, in corporate settings, the compute budget of a team can be quite less than the desired budget, so cost-cutting measures must be taken to keep the cloud bills low and the managers happy. Using a terraform scripted infrastructure for Rancher allows us to spin up our Rancher environment in under 5 minutes, while teardown takes 30 seconds and can be run whenever we’re finished the current Rancher session. Terraform also permits us to make changes to configurations using plans. With plans, we are able to resize, rename, and rebuild our infrastructure in seconds. This can be useful when testing out different domain configurations, cluster sizes (in case we need more resources on our Rancher server), and Rancher versions.

A final anecdote I’ll share is from my experience as a deployment engineer. When launching a new service in a corporate setting, I had to manage five different deployment environments: dev, QA, test, beta, and prod. Typically, these versions will all be one version offset from another and isolated in their own networks running on their own hardware. Learning a tool like Terraform helped me create a simple script that can spin up an application using the same exact process in each environment.  By tweaking a few variables in my terraform.tfvars file, however, I can easily reconfigure an installation to run a specific version of the application and make domains and networking settings point to the environment-specific networks. Although the installation I’ll be showing you is simple, the skills learned by understanding why each line in a Terraform script is there will help you to craft Terraform scripts that fit your unique deployment environments and situations.

Terraforming a Rancher Server

Terraform offers a wide range of Providers, which are used to create, manage, and modify infrastructure. Providers can access clouds, network configurations, git repositories, and more. In this article, we’re going to be using the Digital Ocean provider for Terraform. The Digital Ocean provider gives us access to a variety of services: domains, droplets, firewalls, floating IP addresses, load balancers, SSH keys, volumes, and more. For the purposes of running a Rancher server, we want to configure domains, droplets, SSH keys, and floating IPs for our Digital Ocean account.

As I’ve said before, one of most persuasive features of Digital Ocean is it’s incredible API based droplet initialization. In under 30 seconds, we can spin up almost any droplet Digital Ocean has to offer. This process includes the installation of our SSH keys—as soon as the droplet is up and running, we can SSH into it and provision it. Terraform exploits these awesome features, letting us spin up a droplet, then successively SSH into the droplet and execute our provisioning commands. Because Digital Ocean droplets spin up so quickly, the whole process can take under a minute. Terraform’s powerful library of providers gives us the ability to explain in plain English what we want our infrastructure to look like, and it can handle all the API requests, response checking and formatting, and droplet state detection.

In order to get started Terraforming on Digital Ocean, we want to generate a Digital Ocean API key with both read and write permissions. We can do this here by clicking on Generate New Token and ticking the Write field.

When writing terraform files, it is important to keep your credentials out of the equation and in an environment file. The separation of credentials from configuration allows us to copy and paste this configuration wherever we need it, and specify different credentials for different installations. Terraform supports custom environment files called terraform.tfvars. In your current working directory, create a new file called terraform.tfvars and set the value of do_token to your Digital Ocean API token as follows: do_token = "<my_token>".

The main configuration of our plan must live in a file named terraform.tf in our local directory, so let’s generate that file and tell it to import our key from the terraform.tfvars file as follows:

# terraform.tf
# Set the variable value in *.tfvars file
variable "do_token" {                     # Your DigitalOcean API Token
  type        = "string"
}

Next, let’s add our SSH key to our Digital Ocean account. We’re going to want to use a variable in the terraform.tfvars file to specify the path to the key as this may change in different installations. We’ll add both the private and public keys as variables because we’ll need to use the private key to provision our instance in further steps.

# terraform.tfvars
do_token = "<my_token>"
local_ssh_key_path="<path_to_private_key>"
local_ssh_key_path_public="<path_to_public_key>"

After adding the ssh key paths as variables, we want to initialize those variables in the Terraform plan.

# terraform.tf
...
variable "local_ssh_key_path" {           # The path on your local machine to the SSH private key (defaults to "~/.ssh/id_rsa")
  type        = "string"
  default     = "~/.ssh/id_rsa"
}
variable "local_ssh_key_path_public" {    # The path on your local machine to the SSH public key (defaults to "~/.ssh/id_rsa.pub")
  type        = "string"
  default     = "~/.ssh/id_rsa.pub"
}

Here, we can set some default values so, in the case where we don’t want to specify key paths, they’ll default to the standard location for ssh keys in Unix based systems.

Now that we’ve configured our SSH keys, we need to add the public key to Digital Ocean. This is also a simple process using Terraform; just configure the Digital Ocean provider and tell it to add an SSH key.

# terraform.tf
...
# Configure the Digital Ocean Provider
provider "digitalocean" {
  token = "${var.do_token}"
}

# Create a new DO SSH key
resource "digitalocean_ssh_key" "terraform_local" {
  name       = "Terraform Local Key"
  public_key = "${file("${var.local_ssh_key_path_public}")}"
}

If we ran the plan now, we’d see a new SSH key in the Digital Ocean security panel named Terraform Local Key, which would match the public key specified by our terraform variable.  However,  we want to get a few more actions added before we run the Terraform script.

We want to spin up a droplet to run our Rancher server, so let’s tell Terraform what we want our droplet to look like.

# terraform.tf
...
resource "digitalocean_droplet" "rancher-controller" {
  image       = "coreos-stable"
  name        = "rancher.demo"
  region      = "nyc3"
  size        = "4gb"
  ssh_keys    = ["${digitalocean_ssh_key.terraform_local.id}"]
  private_networking = true
} 

First, we name the resource rancher-controller. Terraform allows you to use resources as both constructors and objects, so after this stage of our plan executes, we will be able to call rancher-controller and ask for attributes like ipv4_address, which don’t exist until after Digital Ocean processes our request. We’re going to need this IP address in order to SSH into the machine for the provisioning step. Then we specify our image as coreos-stable because it comes packed with Docker and allows us to launch the Rancher server with minimal overhead and additional steps. The region can be specified to whichever you prefer, though typically you’ll want to specify the same region for your worker nodes to keep latency low. We also specify the ssh_keys variable as the attribute of our previously defined terraform_local object. The id property is generated when we execute the digitalocean_ssh_key resource and contains a ssh public key ID

Next, we want to add a provisioning step to the rancher-controller item which will start the docker container that serves the Rancher server. This requires what in Terraform language is known as a provisioner. Because this process can be completed with a simple command execution, we can use the default remote execution provisioner. To add the provisioning step to the terraform plan, we add a new field in the rancher-controller object.

# terraform.tf
...
resource "digitalocean_droplet" "rancher-controller" {
  image       = "coreos-stable"
  name        = "rancher.demo"
  region      = "nyc3"
  size        = "4gb"
  ssh_keys    = ["${digitalocean_ssh_key.terraform_local.id}"]
  private_networking = true

  provisioner "remote-exec" {
    connection {
      type        = "ssh"
      private_key = "${file("${var.local_ssh_key_path}")}"
      user        = "core"
    }

    inline = [
      "sudo docker run -d --restart=unless-stopped -p 80:8080 rancher/server",
    ]
  }
} 

In this step, we’ve added a new provisioner of the remote-exec class as well as specified connection variables for how we want to connect to the server (ssh as user core with the private key var.local_ssh_key_path as is defined in terraform.tfvars). Then, we tell the provisioner to perform an inline command, though you can follow this with a series of commands in an array, our array has a single item. We’re asking docker to fetch the rancher/server image from Docker Hub and run the container, mapping port 8080 on the container to port 80 on our host, as the rancher server should be the only application running on port 80 on our host.

The final step in our Terraform configuration is to make our host accessible through a Digital Ocean hosted domain. If you don’t want to connect a domain to your Rancher server, you can simply use the IP address returned by Terraform after running the plan as it is, and access Rancher that way. If you choose to use a domain, you can better secure your Rancher server with GitHub authentication or other domain-based authentication methods.

Let’s define a new variable at the top of the terraform.tf file and include it in the terraform.tfvars file.

# terraform.tf
variable "do_domain" {                    # The domain you have hooked up to Digital Ocean's Networking panel 
  type        = "string"
}
...
# terraform.tfvars
...
do_domain = "<digital_ocean_hosted_domain>"

We can now access this domain name by referencing vars.do_domain in our Terraform plan. Digital Ocean supports the use of floating IP addresses on their platform, which can be directed to different droplets using internal networking. The best way to configure domain routes is to set up a floating IP, point the desired subdomain at the floating IP, and then point the floating IP at the desired droplet. This way, we can modify the floating IP whenever the droplet is rebuilt without having to change the A records on the domain. Also, if we decide to use a Digital Ocean load balancer in the future, we can point the floating IP at the load balancer and not worry about changing domain records when droplets spin up and down.

To add a floating IP address and domain record to our plan, we must add these lines to the terraform.tf file:

# terraform.tf
...
resource "digitalocean_floating_ip" "rancher-controller" {
  droplet_id = "${digitalocean_droplet.rancher-controller.id}"
  region     = "${digitalocean_droplet.rancher-controller.region}"
}

# Create a new record
resource "digitalocean_record" "rancher-controller" {
  name        = "rancher"
  type        = "A"
  domain      = "${var.do_domain}"
  ttl         = "60"
  value       = "${digitalocean_floating_ip.rancher-controller.ip_address}"
}

The first half of this configuration grabs the droplet ID from the rancher-controller droplet that we just spun up. We also define a region for the IP that matches our droplet. The floating IP resource will interface with the droplet resource to learn its internal IP address and route the two together. It then exposes its own IP address so that our domain record can dynamically set the value of the A record to the floating IP.

The second half of the configuration creates a new A record on our domain with the name rancher creating a subdomain rancher.<digital_ocean_hosted_domain>. We set the time-to-live to 60 seconds, so when we tear down the instance and rebuild it, the new IP address is quickly resolved. The value field is populated with the floating IP from earlier.

Now that we’ve taken all these steps to configure the Rancher server, our terraform.tf file should look as follows:

# terraform.tf
variable "do_token" {                     # Your DigitalOcean API Token
  type        = "string"
}            
variable "do_domain" {                    # The domain you have hooked up to Digital Ocean's Networking panel 
  type        = "string"
}
variable "local_ssh_key_path" {           # The path on your local machine to the SSH private key (defaults to "~/.ssh/id_rsa")
  type        = "string"
  default     = "~/.ssh/id_rsa"
}
variable "local_ssh_key_path_public" {    # The path on your local machine to the SSH public key (defaults to "~/.ssh/id_rsa.pub")
  type        = "string"
  default     = "~/.ssh/id_rsa.pub"
}

# Configure the DigitalOcean Provider
provider "digitalocean" {
  token = "${var.do_token}"
}

# Create a new DO SSH key
resource "digitalocean_ssh_key" "terraform_local" {
  name       = "Terraform Local Key"
  public_key = "${file("${var.local_ssh_key_path_public}")}"
}

resource "digitalocean_droplet" "rancher-controller" {
  image       = "coreos-stable"
  name        = "rancher.${var.do_domain}"
  region      = "nyc3"
  size        = "4gb"
  ssh_keys    = ["${digitalocean_ssh_key.terraform_local.id}"]
  private_networking = true

  provisioner "remote-exec" {
    connection {
      type        = "ssh"
      private_key = "${file("${var.local_ssh_key_path}")}"
      user        = "core"
    }

    inline = [
      "sudo docker run -d --restart=unless-stopped -p 80:8080 rancher/server",
    ]
  }
} 

resource "digitalocean_floating_ip" "rancher-controller" {
  droplet_id = "${digitalocean_droplet.rancher-controller.id}"
  region     = "${digitalocean_droplet.rancher-controller.region}"
}

# Create a new record
resource "digitalocean_record" "rancher-controller" {
  name        = "rancher"
  type        = "A"
  domain      = "${var.do_domain}"
  ttl         = "60"
  value       = "${digitalocean_floating_ip.rancher-controller.ip_address}"
}

The terraform.tfvars file should have the following variables exposed:

# terraform.tfvars
do_token = "<token>"
do_domain = "<domain>"
local_ssh_key_path="<ssh_key_path>"
local_ssh_key_path_public="<ssh_pub_path>"

Once we’ve populated the variables and confirmed our files look right, we can install terraform and execute our plan.  We can head over to Terraform’s download page and grab the package appropriate for our system.

Once the installation is finished, we’ll open a terminal window and cd into the directory with our terraform.tf and terraform.tfvars files.

To plan our Terraform build, run the following:

$ terraform plan

We should see this line in our response, noting that we’re creating 4 new resources on Digital Ocean:

Plan: 4 to add, 0 to change, 0 to destroy.

After planning, we can apply the plan by running the following:

$ terraform apply

After a minute or so, we’ll see the script complete and, by navigating to rancher.<do_domain>, we should see the Rancher server up and running!

Please browse around the Rancher blog to see all the cool things you can build with a Rancher server!

Conclusion and Further Steps

As a quick recap, let’s review why we set out to Terraform our Rancher server in the first place. Rancher is easy to configure and deploy. There’s not really any frustration involved in spinning up a Rancher server and getting started. The friction in deploying Rancher, then, comes from the configuration of infrastructure; networking, machines, and images. Terraform has given us the power to automate the straightforward but time-consuming task of spinning up droplets on Digital Ocean, configuring floating IP addresses and domains, and provisioning a Rancher server. These aren’t tasks that are difficult on their own, but when managing many deployments of Rancher or when testing in different environments, quick and painless solutions like Terraform can prevent simple tasks from becoming a burden and reduce the possibility for simple mistakes to break systems.

With that in mind, let’s talk about a few more complex ways we can deploy Rancher using Terraform.

In a fully automated production process, we wouldn’t be using the simple SSH provisioner as the primary configuration tool for our new droplet. We’d have a Chef server up and running and use the Chef provisioner for Terraform to ensure the filesystem and properties of our droplet are exactly as they should be for our installation.  The Chef scripts would proceed to launch the Rancher server. If you have an interest in this process and how one would go about setting it up, let me know! I’d love to go into more detail about using Chef recipes to configure and deploy Rancher servers, but I don’t quite have the space for it in this article.

If High Availibility is what we’re looking for, Digital Ocean isn’t really the best choice for cloud infrastructure. Picking Amazon AWS or Google Cloud Platform would be more sensible as Rancher uses MySQL to mediate between HA hosts in a Rancher cluster and those clouds provide home baked scalable and properly Highly Available MySQL solutions (Amazon RDS and Google Cloud SQL, respectively). If you’re interested in running HA Rancher and happen to be using one of those clouds, check out the Amazon AWS and Google Cloud Platform infrastructure providers for Terraform. You’d also benefit by reading up on the documentation for Multi-Node Rancher Deployments. If you’d be interested in a tutorial utilizing Terraform to run Highly Available Rancher servers on Amazon AWS or Google Cloud Platform, please let me know! I’d love to go into detail and break down the best solutions for HA Rancher in cloud-hosted environments.

If you’re just interested in our simple deployment from this article, the following section gives a few tips and tricks for making your Rancher Terraform script your own.

Changing or Cleaning Up your Rancher Installation

When the fun is finished, or we need to make a change to our configuration, these two commands will let you apply changes, or destroy your installation.

When changing your configuration, if you make any modifications to the droplet settings, it will wipe and rebuild the droplet.

YOU WILL LOSE ALL CONFIGURATION AND DATA ON THE RANCHER SERVER IF YOU REAPPLY A PLAN WITH MODIFIED DROPLET SETTINGS.

$ terraform plan && terraform apply

The above command will re-plan and apply any changes made to the terraform.tf and terraform.tfvars files, which may consist of tearing the droplet down and rebuilding it.

To fully tear down our infrastructure stack, we can run one simple command.

$ terraform destroy

We’ll be asked to enter yes to confirm the destruction of resources, but once we do, Terraform will go ahead and tear down the droplet, domain, floating IP, and SSH keys that were added to our Digital Ocean account.

Upgrading Rancher with Terraform

If you want to upgrade your Rancher server that has been deployed via Terraform and you plan on retaining your data, follow the process outlined here to SSH into your droplet and run the commands specified. In a production environment, it’d be best to use Chef as the provisioner for your Terraform plan and push changes to your droplets via Chef to upgrade the Rancher version. If you don’t care about retaining the data when upgrading, you can modify the line from the terraform.tf file from the provisioning block as follows.

"sudo docker run -d --restart=unless-stopped -p 80:8080 rancher/server:<desired_version_tag>"

Wow, that was quite the process, but now we can build and tear down Rancher server hosts at will in under a minute! Moving these files to other environments and tweaking the variables to fit different installations allow us to create an incredibly simple and repeatable deployment process. With the tools and skills learned from this article, we can take on more challenging tasks, like running Rancher in a highly available manner, without the burden of manually spinning up machines, installing docker, booting up the Rancher container, and configuring our networks.

I hope you’ve enjoyed this approach to automating Rancher deployments on Digital Ocean!

Stay tuned to see some more Rancher-related content and feel free to give us feedback on our user Slack or forums.

 

Eric Volpert is a student at the University of Chicago and works as an evangelist, growth hacker, and writer for our Rancher Labs. He enjoys any engineering challenge. He’s spent the last three summers as an internal tools engineer at Bloomberg and a year building DNS services for the Secure Domain Foundation with CrowdStrike. Eric enjoys many forms of music ranging from EDM to High Baroque, playing MOBAs and other action-packed games on his PC, and late-night hacking sessions, duct taping APIs together so he can make coffee with a voice command.

Free Online Training

Want a crash course in using Rancher?

Join us for free online training courses, hosted monthly by a Rancher technical expert. We provide a great hands-on overview for new users setting up a Rancher deployment, and answer any and all questions you have about Rancher and how to integrate it into your DevOps processes!

Sign up

Recent Posts


Upcoming Events