For teams building and deploying containerized applications using Docker, selecting the right orchestration engine can be a challenge. The decision affects not only deployment and management, but how applications are architected as well. DevOps teams need to think about details like how data is persisted, how containerized services communicate with one another, load balancing, service discovery, packaging and more. It turns out that the choice of orchestration engine is critical to all these areas.
While Rancher has the nice property that it can support multiple orchestration engines concurrently, choosing the right solution is still important. Rather than attempting to boil the ocean by looking at many orchestrators, we chose to look at two likely to be on the short list for most organizations – Kubernetes and Docker Swarm.
Evolving at a rapid clip
To say these frameworks are evolving quickly is an understatement. In the just the past year there have been four major releases of Docker (1.12, 1.13, 17.03 and 17.06) with dozens of new features and a wholesale change to the Swarm architecture. Kubernetes has been evolving at an even more frenetic pace. Since Kubernetes 1.3 was introduced in July of 2016 there have been four additional major releases and no less than a dozen minor releases. Kubernetes is at version 1.7.2 at the time of this writing with 1.8.0 now in alpha 2. Check out the Kubernetes changelog to get a sense of the pace of development.
Comparing Kubernetes and Docker Swarm is a little like trying to compare two Rocket ships speeding along on separate trajectories. By the time you catch up with one and get close enough to see what’s happening, the other is in a whole different place!
Points of comparison
Despite the challenges posed by their rapid evolution, we decided to take a crack at comparing Swarm and Kubernetes in some detail, taking a fresh look at new capabilities in each solution. At a high-level the two solutions do broadly similar things, but they differ substantially in their implementation. We took both solutions out for a test drive (using Rancher running in AWS), got into the weeds, and compared them systematically in these areas:
Lots for DevOps teams to ponder
Both Swarm and Kubernetes are impressive, capable solutions. Depending on their needs, organizations could reasonably choose either solution. If you are new to one solution or the other, understanding the strengths and weaknesses of different solutions, and differences in how they are implemented, can help you make a more informed decision.
Swarm is impressive for its simplicity and seamless integration with Docker. For those experienced with Docker, evolving to use Swarm is simple. Swarm’s new DAB format for multi-host, multi-service applications extends naturally from docker-compose, and the Swarm command set is now part of Docker Engine, so administrators face a minimal learning curve.
Customers considering larger, more complex deployments will want to look at Kubernetes. Docker users will need to invest a little more time to get familiar with Kubernetes, but even if you don’t use all the features out of the gate, the features are there for good reason. Kubernetes has its own discrete command set, API and an architecture that is discrete from Docker. For Kubernetes, the watchword is flexibility. Kubernetes is extensible and configurable and can be deployed in a variety of ways. It introduces concepts like Pods, Replica Sets and Stateful Sets not found in Swarm along with features like autoscaling. While Kubernetes is a little more complex to learn and master, for users with more sophisticated requirements, Kubernetes has the potential to simplify management by reducing the need for ongoing manual interventions.
About the whitepaper
Our comparison was done using Rancher’s container management framework to deploy separate environments for Docker Swarm and Kubernetes. Rather than focus on Rancher however, comparisons are made at the level of Swarm and Kubernetes themselves. Whether you are using Rancher or a different container management framework, the observations should still be useful.
Included in the paper are:
Detailed comparisons between Kubernetes and Swarm
Considerations when deploying both orchestrators in Rancher
Considerations for application designers
High-level guidance on what orchestrator to consider when
Note: You can find an updated comparison of Kubernetes vs. Docker Swarm in a recent blog post here.
Recent versions of Rancher have added support for several common orchestration engines in addition to the standard Cattle. The three newly supported engines, Swarm (soon to be Docker Native Orchestration), Kubernetes and Mesos are the most widely used orchestration systems in the Docker community and provide a gradient of usability versus feature sets. Although Docker is the defacto standard for containerization, there are no clear winners in the orchestration space. In this article, we go over the features and characteristics of the three systems and make recommendations of use cases where they may be suitable.
Docker Native Orchestration is fairly bare bones at the moment but is getting new features at a rapid clip. Since it is part of the official Docker system, it will be the default choice for many developers and hence will have likely have good tooling and community support. Kubernetes is among the most widely used container orchestration systems today and has the support of Google. Lastly, Mesos with Mesosphere (or Marathon, its open source version) takes a much more compartmentalized approach to service managements where a lot of features are left to independent plug-ins and applications. This makes it easier to customize the deployment as individual parts can be swapped out or customized. However, this also means more tinkering is required to get a working setup. Kubernetes is more opinionated about how to build clusters and ships with integrated systems for many common use cases.
Note: Since publishing this post, we’ve created a guide comparing Kubernetes with Docker Swarm. You can read the details in the blog post here..
Over the last six months, Rancher has grown very quickly, and now includes support for multiple orchestration frameworks in addition to Cattle, Rancher’s native orchestrator. The first framework to arrive was Kubernetes, and not long after, Docker Swarm was added. This week, the team at Rancher added support for Mesos. For this article, I’m going to focus on Cattle, Swarm, and Kubernetes, and as I gain experience with Mesos, I’ll share my thoughts in another post.
Rancher’s support for these different orchestration platforms is delivered by creating isolated “environments.” When a user or admin creates an environment, they select the orchestration platform he or she wants to use, and which users will have access to the new cluster. Rancher works with Active Directory, LDAP and GitHub, so you can grant different access privileges to teams or individual on a per cluster basis.
Once you’ve created the environment, Rancher prompts you to add “hosts,” which are just Linux physical or virtual machines that running Docker and Rancher’s agent, which is a container. As soon as the first hosts are added, Rancher begins deploying the orchestration framework you’ve chosen, and you can start using your new environment.
Each of these orchestration platforms has a different set of capabilities. For this article, I won’t try to provide a list pros and cons for each, or a long table comparing features. They are all changing very quickly, so anything I write would be out of date within a month or two. Instead, I’ll share some of my personal experiences with all three, and the scenarios in which I use each of the three frameworks. Rancher makes it so easy to deploy each of these that I highly recommend you try them out for yourself and determine which fits your project best. Read more
Nagios is a fantastic monitoring tool, and I wanted to see if I could get the agent to run as a system container on RancherOS, in order to monitor the host and any Docker containers running on it. It turned out to be incredibly easy. In this blog post, I’ll walk through how to launch the Nagios agent as system container in RancherOS. Specifically, I’ll use two vagrant boxes to cover:
Provisioning a server with the Rancher control plane
Adding a second server running Rancher OS
Installing a Nagios agent as system container on the second server
Connecting the Nagios agent to the Nagios management server
In this tutorial, I will explain how to deploy a Mesos cluster in containers running on RancherOS and then make our deployment portable across different cloud platforms and virtualization systems.
If you’re not familiar with Apache Mesos, it is an open-source project that provides an elastic and highly available clustering framework. With its efficient resource isolation, it is possible to easily build and manage many distributed systems running on heterogeneous resources. Combined with Apache Zookeeper and the Marathon Framework, it provides a powerful platform for deploying applications.
RancherOS is a brand new Linux distribution that is designed to run Docker. Read more