Why you need Kubernetes multi cluster management

As Kubernetes becomes more central to the way businesses deploy applications, many organizations will find themselves running multiple clusters.

According to recent Spectro Cloud research interviewing 333 Kubernetes practitioners and decision-makers more than half of the organizations already manage multiple Kubernetes clusters. 56% of businesses have more than 10 Kubernetes clusters, and 69% run Kubernetes in multiple clouds or other environments — with 80% expecting their scale to grow.

What’s driving this? Kubernetes multi cluster deployments are no longer just about separating dev/test from production, but supporting multi-cloud Kubernetes strategies, multi-region deployments, and application isolation.

And if you’re doing edge computing with Kubernetes, it’s likely that each edge device will be its own cluster — potentially giving you thousands of clusters to manage!

This is where Kubernetes multi-cluster management comes in.

The idea is to achieve centralized management across multiple clusters, or even of all your clusters, instead of having to manage each cluster individually.

Although multi-cluster Kubernetes management is more complicated than single-cluster setups, it is worth the effort.

Keep reading for a look at what is Kubernetes multi-cluster, which challenges you’ll run into and which solutions are available to help you achieve it.

Adoption and approaches to Kubernetes multi-cluster management

In Kubernetes environments, multi-cluster management means having the ability to monitor, configure and operate two or more clusters from a unified interface, with a single API server and single command.

The end result in many cases is the ability to orchestrate a single application across multiple clusters.

Kubernetes multi-cluster management is not a new concept — as this essay on the CNCF site shows, it has been explored for a few years. But multicluster management is not widely adopted even among the most committed Kubernetes users.

We asked hundreds of people who are using multiple Kubernetes clusters in production one question: “How is your organization currently managing multiple Kubernetes clusters?”

• Only 18% said they have unified management of all their clusters.
• Around half said they had some degree of multi-cluster management — for example, multiple clusters in a single managed cloud environment.
• Around a third manage each of their clusters independently.

Why centralize Kubernetes multi-cluster management?

Why would you want to manage multiple clusters in this way, instead of taking the simpler approach of setting up a different control plane for each cluster?

Multi-cluster management brings four main benefits. It:

Simplifies operations. If you run Kubernetes in multiple public clouds, or you have clusters in more than one data center, a multi-cluster management approach allows you to consolidate your management tooling for all of your Kubernetes-based resources.

Instead of having to juggle different toolsets and configurations for each cluster, you can do it all centrally. You can upgrade, patch or reconfigure multiple clusters with a single command — a huge improvement in time spent, not to mention human error.

**Drives consistency. **A single control plane with a single set of policies and access controls can improve consistency, compliance and security. And unified visibility of cluster activity can simplify cost reporting, forecasting and governance.

Brings flexibility. A multi-cluster approach equips you to move workloads between clusters, from one location to another, or between cloud providers. You can reduce vendor lock-in, optimize cost, and respond to the needs of different teams across your business.

Improves application performance. Multi-cluster configurations can provide a degree of high availability (HA), load balancing and improved performance for users in different regions.

For all these reasons, it’s a safe bet that more organizations will seek to consolidate Kubernetes management through a multi-cluster approach.

What you need to know to make multi cluster work

While Kubernetes multi cluster is important, it’s certainly not simple to make it happen. There are tough challenges you need to work through. You need the ability to:

Keep the state of all of your clusters consistent from the perspective of the control plane. This can be particularly challenging if your clusters are geographically distant from one another: network latency can make it tough to keep all of the clusters perfectly in sync.

Differentiate between clusters within your management tooling. Kubernetes and its native management tools assume that everything is running within one cluster. There is no native way to select a cluster when you deploy a resource, for example. You can differentiate easily between namespaces, but there is no equivalent differentiation for clusters.

**Disaggregate observability data, **user and permission configurations and so on based on the clusters with which they are associated. Here again, Kubernetes offers no native solution.

Manage the risk that a bad configuration change could impact multiple clusters. This makes disaster recovery planning especially important when you adopt multi-cluster Kubernetes.

Open source solutions for multi-cluster Kubernetes management

There have been many efforts over the years that have limited success (eg Submariner). Nowadays, there are two options from the open source community. One is Open Cluster Management, a CNCF sandbox project. The other is KubeFed, the official federation solution from the Kubernetes SIG.

The concept of Kubernetes cluster federation defines ‘host’ and ‘member’ clusters. Using a federation architecture, KubeFed extends the native Kubernetes API with features that let you “join” clusters together, then select specific clusters when deploying or managing resources.

While lots of people write about it, federation has not been widely used in practice — KubeFed is still in Alpha, and it’s arguably not yet ready for production use.

It also lacks extensive integrations with Kubernetes dashboards and monitoring tools, so you’ll be left to work mostly from the CLI if you want to manage multiple clusters using KubeFed.

You can also approach multi cluster management through a single management plane over multiple cluster control planes (which is Palette's approach). We base our solution on the Kubernetes SIG project Cluster API to declaratively manage the lifecycle of multiple clusters simultaneously.

Lastly, there are different elements you can use to enhance multi-cluster management.

• CI/CD pipelines help deploy to multiple clusters consistently.
• Service mesh enables multi cluster communication and observability.

The future of Kubernetes multi-cluster management

Multi cluster Kubernetes management will only become more critical for businesses that need to administer multiple diverse Kubernetes clusters.

Even if you only have one cluster to manage today, it's worth investigating the tools and practices that can help you to manage more than one cluster through a single control plane, so that you're prepared when your Kubernetes footprint expands in the future.

Accelerate with Palette

For multi-cluster management at scale, consider a platform like Palette, which offers complete support for multi cluster management out of the box.

By allowing you to administer multiple clusters via a single management plane, Palette makes it as simple as possible to manage as many clusters as you need – even if those clusters are spread across multiple clouds, virtual or bare-metal server clusters, at the edge, or a mix.

You can get started with Palette absolutely free, no credit card required.