Continental Innovates with Rancher and Kubernetes
This page describes the architecture of a high-availability K3s server cluster and how it differs from a single-node server cluster.
It also describes how agent nodes are registered with K3s servers.
A server node is defined as a machine (bare-metal or virtual) running the k3s server command. A worker node is defined as a machine running the k3s agent command.
k3s server
k3s agent
This page covers the following topics:
The following diagram shows an example of a cluster that has a single-node K3s server with an embedded SQLite database.
In this configuration, each agent node is registered to the same server node. A K3s user can manipulate Kubernetes resources by calling the K3s API on the server node.
K3s Architecture with a Single Server
Single server clusters can meet a variety of use cases, but for environments where uptime of the Kubernetes control plane is critical, you can run K3s in an HA configuration. An HA K3s cluster is comprised of:
K3s Architecture with a High-availability Server
In the high-availability server configuration, each node must also register with the Kubernetes API by using a fixed registration address, as shown in the diagram below.
After registration, the agent nodes establish a connection directly to one of the server nodes.
Agent nodes are registered with a websocket connection initiated by the k3s agent process, and the connection is maintained by a client-side load balancer running as part of the agent process.
Agents will register with the server using the node cluster secret along with a randomly generated password for the node, stored at /etc/rancher/node/password. The server will store the passwords for individual nodes as Kubernetes secrets, and any subsequent attempts must use the same password. Node password secrets are stored in the kube-system namespace with names using the template <host>.node-password.k3s.
/etc/rancher/node/password
kube-system
<host>.node-password.k3s
Note: Prior to K3s v1.20.2 servers stored passwords on disk at /var/lib/rancher/k3s/server/cred/node-passwd.
/var/lib/rancher/k3s/server/cred/node-passwd
If the /etc/rancher/node directory of an agent is removed, the password file should be recreated for the agent, or the entry removed from the server.
/etc/rancher/node
A unique node ID can be appended to the hostname by launching K3s servers or agents using the --with-node-id flag.
--with-node-id
The manifests located at the directory path /var/lib/rancher/k3s/server/manifests are bundled into the K3s binary at build time. These will be installed at runtime by the rancher/helm-controller.
/var/lib/rancher/k3s/server/manifests