Networks have long been the cornerstone of companies’ ambitions towards high-performance, multi-cloud or hybrid architectures. Although such architectures were once aspirational marketing buzzwords, they are today’s business reality. Now, with the launch of Cilium Mesh, enterprises gain “a new universal networking layer to connect workloads and machines across the cloud, on-prem and edge.” Comprised of a Kubernetes networking component, a multi-cluster connectivity plan, and a transit gateway, Cilium Mesh helps businesses bridge their on-premises network resources into a cloud-based world.
It sounds cool, and it is cool, but getting to this point was anything but easy. It also remains complex for companies hoping to bridge their existing infrastructure to more modern approaches.
We sometimes take cloud-based architectures for granted because we fail to appreciate the complex demands they place on the infrastructure layer. For example, infrastructure software must now be able to run equally well in public or private cloud infrastructure. It needs to be highly scalable to meet the agility of containers and CI/CD. It must be very secure because it often runs outside the company’s premises. And it must still meet the traditional enterprise networking requirements of interoperability, observability, and security, while being generally open source and somewhat community driven.
Oh, and to be relevant to businesses, all of this cloud-based goodness needs to be translated back into the legacy infrastructure “badness” that businesses have been running on for years. This is what Cilium Mesh does for the network layer, and that’s what Thomas Graf, co-founder and chief technology officer at Isovalent, the creator of Cilium, took the time to explain.
Jump to:
On the way to cloud native
Cilium and Kubernetes emerged around the same time, and Cilium quickly took its place as the standard network abstraction for all the major cloud service providers (e.g. Azure Kubernetes Service and Amazon EKS Anywhere). Not that everyone consciously runs Cilium. For many, they get Cilium as a hidden bonus while using a cloud’s managed services. How much a company knows about Cilium usage has a lot to do with where it is in its cloud journey, according to Graf.
In the initial phase of a Kubernetes journey, often only an application team uses Kubernetes when building an initial version of the application. We see heavy use of managed services in this phase and very limited demands on the network apart from the need to publicly expose the application via an Ingress or API gateway. Graf noted, “These initial use cases are very well addressed by managed services and cloud offerings, which have massively accelerated the path to developing services. Small application teams can run and even scale services quite easily at first.”
However, with more experience and greater use of Kubernetes, this is changing, and sometimes dramatically.
For larger enterprise Kubernetes users, Graf highlighted, they bring typical enterprise requirements such as micro-segmentation, encryption and SIEM integration. Although “these requirements have not changed much” over the years, he stressed, “their implementation has to be completely different today.” How? Well, for one thing, their implementation can no longer disrupt the application development workflow. Application teams are no longer interested in filing tickets to scale infrastructure, open firewall ports, and request IP address blocking. In other words, he summarized: “The platform team is tasked with ticking off all the business requirements without disrupting and negating the gains made on agility and developer efficiency.”
In addition, the platform being built is cloud agnostic and works equally well in public and private clouds. The latest requirements even require integrating existing servers and virtual machines into the mix without slowing down the highly agile processes built on CI/CD and GitOps principles. It is non-trivial; but with Cilium Mesh it is very doable.
This shift will change networking more than SDN
With Cilium Mesh, the project has united some specific types of hybrid and multicloud network problems such as cluster connectivity, service mesh and now legacy environments. Now that Kubernetes has become a standard platform, Graf suggested, it has established a set of principles that must find their way into a company’s existing infrastructure. In other words, as Graf continued, “Existing networks with fleets of VMs or servers must be able to connect to the new north star of infrastructure principles: Kubernetes.”
This is where things get interesting, and this is where Cilium Mesh becomes critical.
“With Cilium Mesh, we’re bringing all of Cilium — including all the APIs built on top of Kubernetes — to the world outside of Kubernetes,” Graf declared. Instead of running on Kubernetes worker nodes, Cilium runs on VMs and servers in the form of transit gateways, load balancers and egress gateways to connect existing networks with new cloud-based principles, including identity-based, zero-trust security enforcement, fully distributed control planes and modern observability with Prometheus and Grafana.
Importantly, Cilium Mesh is equally attractive to Kubernetes platform teams and more traditional NetOps teams. The native Kubernetes approach gives platform teams the necessary confidence to take on additional responsibilities for managing non-Kubernetes infrastructure, while the use of familiar building blocks such as transit gateways and the Border Gateway Protocol (essentially the postal service for the internet) gives NetOps teams a clear but incremental path to a Kubernetes world.
This is a big deal for businesses struggling to understand multicloud, which includes just about everyone. Admittedly, the concept of multicloud has been discussed for a long time, but it is only now that we are getting past the hype (i.e. the ability to simultaneously deploy in multiple public clouds to optimize costs) to the messy reality of enterprise IT (i.e. different teams use different tools for a variety of different reasons). The main battle, Graf pointed out, “is less about how to connect all public cloud providers (and rather) how to arrive at a unified architecture to connect existing on-premises infrastructure with each public cloud offering while maintaining unified security and observability layers.”
This shift to Kubernetes-style principles driving the network layer has a number of benefits. Chief among these will be significantly smaller teams that will operate and deliver infrastructure more efficiently, while providing platforms that allow businesses to adopt modern development practices to remain competitive. That’s a big deal, and one that promises to change networking even more completely than software-defined networking once did.
Disclosure: I work for MongoDB, but the views expressed here are my own.