(Initial Disclaimer: At the time of writing this I work for Cumulus Networks)
A few weeks ago I started at Cumulus Networks doing pre-sales and consulting. My background is almost entirely Cisco based with only a generic user-level knowledge of Linux. I wanted to start a series describing some of the things that have struck me or I’ve struggled with in learning my way around a switch running Cumulus.
In this first post I wanted to describe just what is Cumulus Linux?
The Operating System
Before Cumulus Linux makes sense you have to look at where network operating systems (NOS) came from. If we look at NX-OS, the NOS that runs on Nexus platforms, it (very) roughly looks like this
In NX-OS you have an operating system based on Linux, however you interact with with a management layer that hides Linux and the guts of the system from you. Because of this you don’t have many options for changing things. If you want a feature or application and it’s not already in NX-OS, then there isn’t anything you can do about it. For example, if you really love Ruby and want to write scripts on the box in Ruby you can’t. The Nexus 9k comes with Python, but not Ruby.
Cumulus Linux on the other hand isn’t based on Linux, it is Linux.
This doesn’t look much different than the NX-OS image above, but what is different is that Cumulus acts as a 48 port server. You log into the bash shell and have Linux. Need a software update? apt-get upgrade. Want to start writing automation scripts in PHP? apt-get install php5
These are some trivial examples, but where it gets really interesting is that any tool that exists for Linux server management can now be applied to the network. The entire Linux ecosystem is now supported on your switch. Any tool, package or software is yours to use. What I personally think is interesting are not the examples I can come up with here, but that users aren’t limited with what they want to do on their switches.
Cumulus Linux is just a piece of software. That’s it. If we could get away with servers running our datacenter networks end-to-end it would have been done years ago. Unlike Arista or Cisco, Cumulus doesn’t make hardware. Cumulus relies on hardware partners like Dell to build the switches that run our software. Just like servers. RedHat doesn’t make servers, they make software that runs on any server you want.
Inside the partner switches it’s the same Broadcom chipsets that run the Nexus 3000, 9000 and Arista platforms. Just like RedHat doesn’t care if it’s an HP or Dell server as long as it’s x86, Cumulus doesn’t care if you bought the switch from EdgeCore or Penguin Computing, as long as it’s Broadcom it can run Cumulus Linux.
What Cumulus does that other Linux distros don’t do is the magic to take routing and switching information from the kernel and push it into the Broadcom chipset. Without this piece you wouldn’t have hardware acceleration allowing for multiple ports of 10 or 40 gig.
Cumulus is just another version of Linux that has been modified to power Broadcom ASICs. Now the switch looks (and more importantly, is managed) just like your servers. Unlike traditional Network Operating Systems you don’t have a walled garden where it’s built on top of Linux but you don’t really have control. As Strech from Packet Life put it, it’s “unapologetically Linux”.