Illumina Innovates with Rancher and Kubernetes
In this blog series, we’ll explore a few ways that Rancher uses of TLS certificates. TLS, or Transport Layer Security, is a cryptographic protocol used to secure network communication. It is the successor to the now-deprecated Secure Sockets Layer, or SSL.
You can expect to walk away with an understanding of how TLS integrates into various Rancher components, and how you can prepare your environment to properly leverage TLS in Rancher.
In the first article in this series, we explored the ways that Rancher uses TLS certificates. Recall that Rancher uses certificates in many locations. Two of these are of particular interest to users: UI security and agent<->API communication security. We also looked at one method of Rancher installation: using self-signed certificates.
In this article, we’ll explore two more Rancher installation methods and how they leverage TLS certificates:
Let’s Encrypt is a non-profit certificate authority run by Internet Security Research Group that provides X.509 certificates for Transport Layer Security encryption at no charge.
(read more at Let’s Encrypt)
So, in other words, free TLS certificates! Not only are the certificates free, but one of the most publicly trusted Certificate Authorities signs them. So, you get automatic trust in most browsers and operating systems.
The Rancher server includes, by way of Golang’s autocert library, a way to automatically provision a Let’s Encrypt certificate. This library provides automatic handling of ACME-based certificate authorities, of which Let’s Encrypt is one.
Before you enable this functionality, you need to do a few things:
Start the rancher container with the following argument:
Meeting these prerequisites and starting Rancher with that argument (among other arguments you may need) instructs Rancher to request a certificate from Let’s Encrypt for <YOUR.DNS.NAME>. Barring any complications, your Rancher server should receive a certificate from LE within a few minutes.
In an HA installation scenario, cert-manager manages the Let’s Encrypt certificate request lifecycle instead of Rancher server itself. Follow the instructions here to install cert-manager into the Kubernetes cluster you have prepared.
Once you’ve have installed cert-manager, the next step is to install rancher. There are three arguments required (in addition to any other arguments you need) to enable Let’s Encrypt for Rancher:
In addition to these options, you need to make sure that HTTP traffic can reach your Rancher cluster from the Internet on tcp/80. Let’s Encrypt ACME requests can come from any location on the Internet, so you won’t be able to limit this via firewall rules easily.
This option for TLS termination allows you to bring your own certificates, whether they were signed by a public CA (such as Digicert) or a private CA that you or your organization have set up.
This option is often a good choice for organizations whose security teams require the use of a certificate signed by the org’s private CA. That CA certificate might be deployed on corporate workstations, so you get the benefits of browser and operating system trust while retaining total control of the certificate chain.
This option is also a solid choice if you have a certificate that has been signed by a public CA.
If you decide to do BYO certificates with single node installation, you need to ensure that the certificate (including any intermediates) and private key are present on the host machine where the Rancher container will run.
Note: If you’re using a certificate signed by a private CA, you need the certificate of the root CA. Rancher requires this certificate to generate CATTLE_CA_CHECKSUM, as described in the first article in this blog series.
When the Rancher container starts, it looks inside the container at /etc/rancher/ssl for the TLS files. To place those files in the correct location (and with the correct names), you must use the following docker bind mount arguments. Please note that ALL files must be PEM encoded.
Note how these files have a specific path (/etc/rancher/ssl/) and specific file names (cert.pem, key.pem and cacerts.pem). Do not alter these paths and file names. Rancher looks specifically for the existence of these files when attempting to secure traffic using these certificates.
When you install Rancher in an HA scenario, TLS termination is delegated to the cluster’s ingress controller. During installation, an
resource is created in the underlying Kubernetes cluster. This Ingress resource configures the controller to respond to queries on the hostname you specify for the Rancher server. Additionally, this Ingress resource specifies the TLS certificate to use.
You must pass the following arguments to the helm command when installing the Rancher chart:
Once you’ve completed Rancher installation, you need to place the certificates into the Rancher Kubernetes cluster.
First, place the Rancher server certificate (that is, the certificate issued for <your.rancher.com>) and any intermediate(s) into a file called tls.crt. Those certificates should begin with the server cert, followed by any intermediate(s) in their issuance order. Place the Rancher server certificate key into a file called tls.key. *These files must be called tls.crt and tls.key as Rancher is looking for specific file names in the secret.*
Once you have created these files, issue the following command to place them into the Kubernetes cluster:
kubectl -n cattle-system create secret tls tls-rancher-ingress \
Next, place the CA certificate into a file called cacerts.pem. This should be only the root CA of your TLS chain. That is, no intermediate(s) should be in this file – they will be in tls.crt as discussed above. Then, issue the following command to place this into the Kubernetes cluster:
kubectl -n cattle-system create secret generic tls-ca \
Once you’ve uploaded these files, Rancher is secured using these TLS certificates, and the provided CA certificate are used for agent connection verification.
Here is a simple diagram showing where various certificates should be placed:
In Part Three in this series, we’ll examine the final TLS termination type: external termination.
Check out our other security related guides, including Kubernetes Master Classes and Online Meetups.