Use Rancher to create a Kubernetes cluster in vSphere.

Introduction

When creating a vSphere cluster, Rancher first provisions the specified amount of virtual machines by communicating with the vCenter API. Then it installs Kubernetes on top of them. A vSphere cluster may consist of multiple groups of VMs with distinct properties, such as the amount of memory or the number of vCPUs. This grouping allows for fine-grained control over the sizing of nodes for the data, control, and worker plane respectively.

Note: The vSphere node driver included in Rancher currently only supports the provisioning of VMs with RancherOS as the guest operating system.

Prerequisites

Before proceeding to create a cluster, you must ensure that you have a vSphere user with sufficient permissions. If you are planning to make use of vSphere volumes for persistent storage in the cluster, there are additional requirements that must be met.

Provisioning a vSphere Cluster

The following steps create a role with the required privileges and then assign it to a new user in the vSphere console:

  1. From the vSphere console, go to the Administration page.

  2. Go to the Roles tab.

  3. Create a new role. Give it a name and select the privileges listed in the permissions table.

    image

  4. Go to the Users and Groups tab.

  5. Create a new user. Fill out the form and then click OK. Make sure to note the username and password, as you will need it when configuring node templates in Rancher.

    image

  6. Go to the Global Permissions tab.

  7. Create a new Global Permission. Add the user you created earlier and assign it the role you created earlier. Click OK.

    image

    image

Creating vSphere Clusters

Create a vSphere Node Template

To create a cluster, you need to create at least one vSphere node template that specifies how VMs are created in vSphere.

Note: Once you create a node template, it is saved, and you can re-use it whenever you create additional vSphere clusters.

  1. Log in with an admin account to the Rancher UI.

  2. From the user settings menu, select Node Templates.

  3. Click Add Template and then click on the vSphere icon.

  4. Under Account Access enter the vCenter FQDN or IP address and the credentials for the vSphere user account (see Prerequisites).

  5. Under Instance Options, configure the number of vCPUs, memory, and disk size for the VMs created by this template.

  6. Optional: Enter the URL pointing to a RancherOS cloud-config file in the Cloud Init field.

  7. Ensure that the OS ISO URL contains the URL of a VMware ISO release for RancherOS (rancheros-vmware.iso).

    image

  8. Optional: Provide a set of Configuration Parameters for the VMs.

  9. Under Scheduling, enter the name/path of the Data Center to create the VMs in, the name of the VM Network to attach to, and the name/path of the Datastore to store the disks in.

    image

  10. Optional: Assign labels to the VMs that can be used as a base for scheduling rules in the cluster.

  11. Optional: Customize the configuration of the Docker daemon on the VMs that will be created.

  12. Assign a descriptive Name for this template and click Create.


Create a vSphere Cluster

After you’ve created a template, you can use it stand up the vSphere cluster itself.

  1. From the Global view, click Add Cluster.

  2. Choose vSphere.

  3. Enter a Cluster Name.

  4. Use Member Roles to configure user authorization for the cluster.

    • Click Add Member to add users that can access the cluster.
    • Use the Role drop-down to set permissions for each user.

  5. Use Cluster Options to choose the version of Kubernetes, what network provider will be used, if you want to enable Pod Security Policies and whether the nodes added to this cluster need to have a supported Docker version installed.

  6. Add one or more node pools to your cluster.

    A node pool is a collection of nodes based on a node template. A node Template defines the configuration of a node, like what Operating System to use, number of CPUs and amount of memory. Each node pool must have one or more nodes roles assigned.

    Notes:

    • Each node role (i.e. etcd, Control Plane, and Worker) should be assigned to a distinct node pool. Although it is possible to assign multiple node roles to a node pool, this should not be done for production clusters.
    • The recommended setup is to have a node pool with the etcd node role and a count of three, a node pool with the Control Plane node role and a count of at least two, and a node pool with the Worker node role and a count of at least two. Regarding the etcd node role, refer to the etcd Admin Guide.

    image

  7. Review your configuration, then click Create.

Note:

If you have a cluster with DRS enabled, setting up VM-VM Affinity Rules is recommended. These rules allow VMs assigned the etcd and control-plane roles to operate on separate ESXi hosts when they are assigned to different node pools. This practice ensures that the failure of a single physical machine does not affect the availability of those planes.

Result:

  • Your cluster is created and assigned a state of Provisioning. Rancher is standing up your cluster.
  • You can access your cluster after its state is updated to Active.
  • Active clusters are assigned two Projects, Default (containing the namespace default) and System (containing the namespaces cattle-system,ingress-nginx,kube-public and kube-system, if present).

Annex - Node Template Configuration Reference

The tables below describe the configuration options available in the vSphere node template.

Account Access

Parameter Required Description
vCenter or ESXi Server * IP or FQDN of the vCenter or ESXi server used for managing VMs.
Port * Port to use when connecting to the server. Defaults to 443.
Username * vCenter/ESXi user to authenticate with the server.
Password * User’s password.

Instance Options

Parameter Required Description
CPUs * Number of vCPUS to assign to VMs.
Memory * Amount of memory to assign to VMs.
Disk * Size of the disk (in MB) to attach to the VMs.
Cloud Init URL of a RancherOS cloud-config file to provision VMs with. This file allows further customization of the RancherOS operating system, such as network configuration, DNS servers, or system daemons.
OS ISO URL * URL of a RancherOS vSphere ISO file to boot the VMs from. You can find URLs for specific versions in the Rancher OS GitHub Repo.
Configuration Parameters Additional configuration parameters for the VMs. These correspond to the Advanced Settings in the vSphere console. Example use cases include providing RancherOS guestinfo parameters or enabling disk UUIDs for the VMs (disk.EnableUUID=TRUE).

Scheduling Options

Parameter Required Description
Data Center * Name/path of the datacenter to create VMs in.
Pool Name/path of the resource pool to schedule the VMs in. If not specified, the default resource pool is used.
Host Name/path of the host system to schedule VMs in. If specified, the host system’s pool will be used and the Pool parameter will be ignored.
Network * Name of the VM network to attach VMs to.
Data Store * Datastore to store the VM disks.
Folder Name/path of folder in the datastore to create the VMs in. Must already exist.

Annex - vSphere Permissions

The following table lists the permissions required for the vSphere user account configured in the node templates:

Privilege Group Operations
Datastore AllocateSpace
Browse
FileManagement
UpdateVirtualMachineFiles
UpdateVirtualMachineMetadata
Network Assign
Resource AssignVMToPool
Virtual Machine Config (All)
GuestOperations (All)
Interact (All)
Inventory (All)
Provisioning (All)