Continental Innovates with Rancher and Kubernetes
Available as of v2.5+
Kubernetes is moving away from maintaining cloud providers in-tree. vSphere has an out-of-tree cloud provider that can be used by installing the vSphere cloud provider and cloud storage plugins.
This page covers how to migrate from the in-tree vSphere cloud provider to out-of-tree, and manage the existing VMs post migration.
It follows the steps provided in the official vSphere migration documentation and provides the steps to be performed in Rancher.
Existing volumes that were provisioned using the following cloud-config format will NOT get migrated due to an existing bug in vsphere CSI.
If the cloud-config has this format for datastore and resource pool path, vsphere CSI driver cannot recognize it:
Volumes provisioned with the in-tree provider using the following format will get migrated correctly:
Upstream bug: https://github.com/kubernetes-sigs/vsphere-csi-driver/issues/628
Rancher issue tracking this bug: https://github.com/rancher/rancher/issues/31105
Before installing CPI, we need to taint all nodes with node.cloudprovider.kubernetes.io/uninitialized=true:NoSchedule.
This can be done by running the following commands:
curl -O https://raw.githubusercontent.com/rancher/helm3-charts/56b622f519728378abeddfe95074f1b87ab73b1e/charts/vsphere-cpi/taints.sh
chmod +x taints.sh
./taints.sh <path to kubeconfig if running the command outside the cluster>
Once all nodes are tainted by the running the script, launch the Helm vSphere CPI chart.
vSphere CPI initializes all nodes with ProviderID, which is needed by the vSphere CSI driver.
Check if all nodes are initialized with the ProviderID with the following command:
kubectl describe nodes | grep "ProviderID"
For enabling feature flags, click on “Edit as YAML”, and add the following under kube-controller and kubelet:
Worker nodes must be drained during the upgrade before changing the kubelet and kube-controller-manager args.