Available as of v2.1.0

Important:

Support for Windows nodes is currently an experimental feature and is not yet officially supported in Rancher. Therefore, we do not recommend using Windows nodes in a production environment.

When provisioning a custom cluster using Rancher, you can use a mix of Linux and Windows hosts as your cluster nodes.

This guide walks you through create of a custom cluster that includes 3 nodes: a Linux node, which serves as a Kubernetes control plane node; another Linux node, which serves as a Kubernetes worker used to support Ingress for the cluster; and a Windows node, which is assigned the Kubernetes worker role and runs your Windows containers.

Notes:

  • For a summary of Kubernetes features supported in Windows, see Using Windows Server Containers in Kubernetes.
  • Windows containers must run on Windows Server 1803 nodes. Windows Server 1709 and earlier versions do not support Kubernetes properly.
  • Containers built for Windows Server 1709 or earlier do not run on Windows Server 1803. You must build containers on Windows Server 1803 to run these containers on Windows Server 1803.

Objectives for Creating Cluster with Windows Support

When setting up a custom cluster with support for Windows nodes and containers, complete the series of tasks below.

1. Provision Hosts

To begin provisioning a custom cluster with Windows support, prepare your host servers. Provision three nodes according to our requirements—two Linux, one Windows. Your hosts can be:

  • Cloud-hosted VMs
  • VMs from virtualization clusters
  • Bare-metal servers

The table below lists the Kubernetes roles you’ll assign to each host, although you won’t enable these roles until further along in the configuration process—we’re just informing you of each node’s purpose. The first node, a Linux host, is primarily responsible for managing the Kubernetes control plane, although, in this use case, we’re installing all three roles on this node. Node 2 is also a Linux worker, which is responsible for Ingress support. Finally, the third node is your Windows worker, which will run your Windows applications.

Node Operating System Future Cluster Role(s)
Node 1 Linux (Ubuntu Server 16.04 recommended) Control Plane, etcd, Worker
Node 2 Linux (Ubuntu Server 16.04 recommended) Worker (This node is used for Ingress support)
Node 3 Windows (Windows Server 1803 required) Worker

Requirements

  • You can view node requirements for Linux and Windows nodes in the installation section.
  • All nodes in a virtualization cluster or a bare metal cluster must be connected using a layer 2 network.
  • To support Ingress, your cluster must include at least one Linux node dedicated to the worker role.
  • Although we recommend the three node architecture listed in the table above, you can add additional Linux and Windows workers to scale up your cluster for redundancy.

2. Cloud-hosted VM Networking Configuration

Note: This step only applies to nodes hosted on cloud-hosted virtual machines. If you’re using virtualization clusters or bare-metal servers, skip ahead to Create the Custom Cluster.

If you’re hosting your nodes on any of the cloud services listed below, you must disable the private IP address checks for both your Linux or Windows hosts on startup. To disable this check for each node, follow the directions provided by each service below.

Service Directions to disable private IP address checks
Amazon EC2 Disabling Source/Destination Checks
Google GCE Enabling IP Forwarding for Instances
Azure VM Enable or Disable IP Forwarding

3. Create the Custom Cluster

To create a custom cluster that supports Windows nodes, follow the instructions in Creating a Cluster with Custom Nodes, starting from 2. Create the Custom Cluster. While completing the linked instructions, look for steps that requires special actions for Windows nodes, which are flagged with a note. These notes will link back here, to the special Windows instructions listed in the subheadings below.

Enable the Windows Support Option

While choosing Cluster Options, set Windows Support (Experimental) to Enabled.

Enable Windows Support

After you select this option, resume Creating a Cluster with Custom Nodes from step 6.

Networking Option

When choosing a network provider for a cluster that supports Windows, the only option available is Flannel, as host-gw is needed for IP routing.

Flannel

If your nodes are hosted by a cloud provider and you want automation support such as load balancers or persistent storage devices, see Selecting Cloud Providers for configuration info.

Node Configuration

The first node in your cluster should be a Linux host that fills the Control Plane role. This role must be fulfilled before you can add Windows hosts to your cluster. At minimum, the node must have this role enabled, but we recommend enabling all three. The following table lists our recommended settings (we’ll provide the recommended settings for nodes 2 and 3 later).

Option Setting
Node Operating System Linux
Node Roles etcd
Control Plane
Worker

Recommended Linux Control Plane Configuration

When you’re done with these configurations, resume Creating a Cluster with Custom Nodes from step 8.

4. Add Linux Host for Ingress Support

After the initial provisioning of your custom cluster, your cluster only has a single Linux host. Add another Linux host, which will be used to support Ingress for your cluster.

  1. Using the content menu, open the custom cluster your created in 2. Create the Custom Cluster.

  2. From the main menu, select Nodes.

  3. Click Edit Cluster.

  4. Scroll down to Node Operating System. Choose Linux.

  5. Select the Worker role.

  6. Copy the command displayed on screen to your clipboard.

  7. Log in to your Linux host using a remote Terminal connection. Run the command copied to your clipboard.

  8. From Rancher, click Save.

Result: The worker role is installed on your Linux host, and the node registers with Rancher.

5. Adding Windows Workers

You can add Windows hosts to a custom cluster by editing the cluster and choosing the Windows option.

  1. From the main menu, select Nodes.

  2. Click Edit Cluster.

  3. Scroll down to Node Operating System. Choose Windows.

  4. Select the Worker role.

  5. Copy the command displayed on screen to your clipboard.

  6. Log in to your Windows host using your preferred tool, such as Microsoft Remote Desktop. Run the command copied to your clipboard in the Command Prompt (CMD).

  7. From Rancher, click Save.

  8. Optional: Repeat these instruction if you want to add more Windows nodes to your cluster.

Result: The worker role is installed on your Windows host, and the node registers with Rancher.

6. Cloud-hosted VM Routes Configuration

In Windows clusters, containers communicate with each other using the host-gw mode of Flannel. In host-gw mode, all containers on the same node belong to a private subnet, and traffic routes from a subnet on one node to a subnet on another node through the host network.

  • When worker nodes are provisioned on AWS, virtualization clusters, or bare metal servers, make sure they belong to the same layer 2 subnet. If the nodes don’t belong to the same layer 2 subnet, host-gw networking will not work. Please contact Rancher support if your worker nodes on AWS, virtualization clusters, or bare metal servers don’t belong to the same layer 2 network.

  • When worker nodes are provisioned on GCE or Azure, they are not on the same layer 2 subnet. Nodes on GCE and Azure belong to a routable layer 3 network. Follow the instructions below to configure GCE and Azure so that the cloud network knows how to route the host subnets on each node.

To configure host subnet routing on GCE or Azure, first run the following command to find out the host subnets on each worker node:

kubectl get nodes -o custom-columns=nodeName:.metadata.name,nodeIP:status.addresses[0].address,routeDestination:.spec.podCIDR

Then follow the instructions for each cloud provider to configure routing rules for each node:

Service Instructions
Google GCE For GCE, add a static route for each node: Adding a Static Route.
Azure VM For Azure, create a routing table: Custom Routes: User-defined.