Brien Posey August 9, 2017
Kubernetes is designed to address some of the difficulties that are inherent in managing large-scale containerized environments. However, this doesn’t mean Kubernetes can scale in all situations all on its own. There are steps you can and should take to maximize Kubernetes’ ability to scale—and there are important caveats and limitations to keep in mind when scaling Kubernetes. I’ll explain them in this article.
Scale versus Performance
The first thing that must be understood about scaling a Kubernetes cluster is that there is a tradeoff between scale and performance. For example, Kubernetes 1.6 is designed for use in clusters with up to 5,000 nodes. But 5,000 nodes is not a hard limit; it is merely the recommended node maximum. In actuality, it is possible to exceed the 5,000 node cluster limit substantially, but performance begins to drop off after doing so.
What this means more specifically is this: Kubernetes has defined two service level objectives. The first of these objectives is to return 99% of all API calls in less than a second. The second objective is to be able to start 99% of pods within less than five seconds. Although these objectives do not act as a comprehensive set of performance metrics, they do provide a good baseline for evaluating general cluster performance. According to Kubernetes, clusters with more than 5,000 nodes may not be able to achieve these service level objectives.
So, keep in mind that beyond a certain point, you may have to sacrifice performance in order to gain scalability in Kubernetes. Maybe this sacrifice is worth it to you, and maybe it’s not, depending on your deployment scenario.
One of the main issues that you are likely to encounter when setting up a really large Kubernetes cluster is that of quota limitations. This is especially true for cloud-based nodes since cloud service providers commonly implement quota limitations. Read more