Kubernetes vs. Mesos: Looking for Ideal Orchestration Tool

Kubernetes (K8S) is an open-source container orchestration system originally created by Google that handles the entire production lifecycle, from on-the-fly deployment, to scaling up and down, to health checks with high availability. It’s also very opinionated.

What is Mesos? A distributed systems kernel Mesos is built using the same principles as the Linux kernel, only at a different level of abstraction. The Mesos kernel runs on every machine and provides applications with API’s for resource management and scheduling across entire datacenter and cloud environments. Mesos uses Linux cgroups to provide isolation for CPU, memory, I/O and file system. Mesos is a project by Apache and gives the ability to run both containerized and non-containerized workloads in a distributed manner. It allow to run workloads on real servers and on the containerized servers. It was designed to scale to very large clusters, from hundreds to thousands of hosts.

The Problem and the Different Approaches

We should emphasize that we’re comparing tools that have different approaches to the same problem. K8S is a container orchestrator or, in other words, a tool that manages containers and their peculiarities such as availability, scaling, and so on. Apache Mesos, on the other hand, is more like a “cloud operating system” that tries to manage all the resources of a cloud (public or private), meaning it has a far broader range of responsibilities.
While Kubernetes works on the concept that every computational resource must be enveloped within a container, Mesos understands that the world is not black and white, and that we should use the best tools for each particular situation.
The architecture diagrams below illustrate these differences:

Figure 1: Kubernetes architecture

Kubernetes has one or more kubernetes master instances and one or more kubernetes nodes. The master instance is used to manage the cluster and the available nodes. It also manages deployment settings (number of instances, what to do with a version upgrade, high availability, etc.) and service discovery. Every computing resource is enveloped by a container and cloud resources, such as network, storage, and everything else, should be provided by plugins to comply with the K8S philosophy.

Figure 2: Mesos architecture

Mesos’ architecture is similar, but it has evolved a whole new layer. Like K8S, Mesos has a master and nodes (agents), which provide analogous functionality. However, it adds a scheduler layer that doesn’t exist in K8S. A scheduler is an implementation of a technology that can use the Mesos infrastructure to run what it was built for. In the picture above, we’ve deployed Hadoop (big data) and MPI (messaging) schedulers; however, there are dozens of schedules available, from containers (Marathon) to continuous integration (Jenkins). Of the long list of schedulers built on Mesos, one worth highlighting is Marathon, which is a container orchestration scheduler, similar to K8S.
As a generalization, Kubernetes is a more opinionated tool that can be very useful—if you embrace its founders’ vision. Mesos, on the other hand, is more flexible and even enables you to create your own scheduler. Containers are generally not a good fit for legacy or monolithic systems, but these can be accommodated by Mesos through the creation of suitable schedulers. Unfortunately, however, Mesos adds a new layer of complexity that many developers are not willing to tolerate.

 

Mesos can work with multiple frameworks and Kubernetes is one of them. To give you more choice, if it meets your use case, it is also possible to run Kubernetes on Mesos.