Jim Duffy wrote an interesting article on Network World’s Cisco Connection blog called “Cisco, Arista Disaggregating?” in which he speculates that Cisco and Arista may make their network operating systems (NOS) available for use on bare metal switches.

Is there any mileage in this idea?

Old News, New Timing

The idea of the big players selling their software for use on generic hardware has been floating around pretty much since SDN hit the news and the first bare metal switches came out, with Cisco for example looking like they were pretending that SDN wasn’t a thing, and their position was secure if they continued to do what they already did. To be honest, I think Cisco is still paying the price for initially lacking a strategy, then embracing SDN in such a confusing way. Nonetheless, the idea isn’t new, but has the market moved to a position where Cisco and Arista really need to do this? And what of Juniper; are they immune to being sucked into the bare metal market?

Special Sauce

In addition to being a good addition to awesome music of G. Love, for companies like Cisco Arista and Juniper, their “special sauce” these days is their network operating systems – IOS, eOS and Junos OS respectively, at least in terms of the Ethernet switching market. The NOS is what distinguishes their products from other people’s when it comes to selecting a switching vendor. After all, D-Link (to pick a victim) can produce 48 port gigabit switches that are stackable and have 10G uplinks. Why would anybody pay Cisco prices when they could pay D-Link prices instead? There must be a reason.

Cisco

Cisco’s venerable IOS is long standing and well known. Somehow, the company has managed to keep the same fundamental configuration paradigm despite the enormous change in the technologies the switches have to handle, and to say that it is ubiquitous would understate the degree to which the interface and configuration style define what network device configuration looks like. The number of people who have cloned that interface is testament to its domination of the market, even if it isn’t necessarily actually the interface preferred by many network engineers. But an interface is just a front end to functionality and it can, and has been, cloned. So why is IOS still “better”?

Cisco IOS has typically contained features that are either proprietary, cutting edge, or are enhancements of the standards-based deployments that others offer. This is a double-edged sword, as anything proprietary means vendor lock-in (which is good for Cisco), but may be considered a non-starter for other potential purchasers. EIGRP, CDP, FabricPath, PVST+, HSRP, GLBP – these are all examples of Cisco going their own way, or creating their own protocols; they aren’t bad by any means, but they mean Cisco, Cisco and more Cisco to keep them working.

None of the features I listed above need special hardware support except perhaps FabricPath, so how hard would it be to implement the same features on non-Cisco hardware? Not hard at all, from the sounds of it. Some other features may be hardware dependent, though. For example, Cisco’s ACI architecture relies on information about micro flows obtained from the hardware, so I’m not so sure that’s something that could as easily be ported to some random Broadcom or Intel chip.

Juniper

Juniper has always been proud in the past that they custom-design their ASICs to do the heavy lifting on their platform, and how they don’t use merchant silicon. That may play well in the core router market, and may be a hardware advantage that gives them additional special sauce when it comes to meeting performance targets, but in the Ethernet switching market is that necessary now? What feature(s) would Juniper want that are not in an off the shelf chip and would necessitate the time and expense required to fabricate a custom ASIC for their switching platforms? In fact, look at the Juniper QFX5100 – under the hood is, guess what, a Broadcom Trident II.

So what makes Juniper different? Ask most engineers, and they’ll tell you straight out that Juniper’s special sauce is the Junos OS. Designed with operational ease of use in mind, the Junos OS really is a pleasure to use. Easy commit and rollback processes, in depth logging capabilities and a standard (-ish) command structure across their product sets, make Junos a nerd favorite, and Juniper, like Cisco, have done their best to shoehorn every new technology into the same standard configuration paradigm without corrupting it too much (except where Ethernet switching is concerned where the pooch has been well and truly, um, groomed).

Feature-wise, does Junos bring something particularly special to the table then? Actually, not so much. Juniper is proud to be standards based; I think Juniper see’s Cisco’s protocol enhancements as an unnecessary bastardization of a standard that just makes their equipment unable to interoperate with anybody else’s. It also means that somebody who chooses the Cisco product will be unable to add Juniper to their network later on too, so that’s another good reason for Juniper to dislike Cisco’s habit of adding non-standard knobs and tweaks to protocols – it shuts them out of environments that already have Cisco in play.

For the hardware side of things, Juniper’s Rami Rahim says in a J-Net Blog Post “Making the Right Silicon Decisions in Networking” that Juniper’s current strategy is to use whatever it thinks is the right tool for the job, be that customer silicon, merchant silicon or a general purpose CPU. By a huge coincidence, yesterday Rami Rahim was announced as the new CEO of Juniper, so perhaps his view will prevail. If the QFX5100 is based around Broadcom silicon and an x86 CPU, could it be easily moved to a white box switch instead? It sure sounds like it.

Arista

It’s funny, but while I hear people waxing lyrical about IOS and Junos OS, I’ve rarely heard somebody going on about Arista’s Extensible Operating System (EOS). Arista is perhaps unique out of the three vendors here because their products are all built on merchant silicon, specifically to keep costs down and keep them up to date without having to have a a team of specialist silicon designers on board to make their own ASICs. You could argue then that Arista already does white box switching, but you have to buy their white box to make it work. Well ok, maybe it’s more gray than white, but the point stands.

The linux-based EOS is by all accounts terrific. I’ve never used it myself, but it allowed Arista to steal a lead over Cisco in the high frequency trading space by introducing a very low latency switching platform still based on merchant silicon. I wonder how much of that product relied on special design outside the Ethernet switching ASICs, and how much is purely software.

Arista has also worn the “best value 40G switch” crown for a while, but if the value is in the transceivers and optics, could I just plug those in to a white box switch instead?

Disaggregating

So this idea that Cisco and Arista, and even Juniper might offer a version of their NOS to run on white box hardware makes sense to me given that more and more people are using bare metal for their Ethernet switching needs, and if the “big guys” don’t offer software, they’re automatically out of that market. Don’t get me wrong – there are some super players in that space already – BigSwitch and Cumulus are just two – and that’s perhaps an even bigger motivation for Cisco et al to seriously consider getting in there and keeping people in their own product family (software at least) rather than losing the business to somebody else entirely. Let people spend money on the software and feature set that they want, and let the underlying hardware be a fairly irrelevant commodity. After all, look at the Quanta LB4M I picked up recently. Good hardware, it seems, but the ODM software that comes with it is a pile of donkey droppings. If I could put IOS or Junos on there, I’d certainly be happier, although I probably wouldn’t like the cost.

Support

Something that’s less clear with the bare metal switching market is support. Existing vendors like Cumulus and BigSwitch have to have a Hardware Compatibility List (HCL) stating which vendors’ hardware their software works on. But when it comes down to it and there’s some odd bug that arises, do you risk having two companies pointing the finger at one another? As is sometimes said in the networking world, the advantage of having a single vendor implementation is that when there’s a problem, there’s just the one person to shout at.

If we imagine that the bare metal / white box market becomes truly commoditized the same way that things like WiFi Access Points have (lots of no-name vendors selling ODM reference designs), we might see all kinds of options for hardware showing up, possibly at very steep discounts. There’s no guaranteeing the quality of the hardware surrounding the ASICs though. Using the WiFi example, you would have no idea if the company has any clue what it’s doing with its antennas, but the price is very attractive. It’s also of note that some vendors consider the reference designs to be just a starting point from which they can develop a significantly improved product; just implementing a reference design often means implementing a non-optimized product that functions, but not as well as it could.

Put all this together and you will most likely end up with the same thing we see in the consumer marketplace where you can buy equipment with the same fundamental specification, some from a brand you’ve never heard of (and might not ever hear from again) and others from brands you know and trust; and some you know and should know better than to trust. Why buy Known Brand X over Unknown Brand Y? Reputation? maybe you believe that Brand Y does more optimization, or you like their support better. But if you’re deciding between Known Brand X and Known Brand Z who both offer a pretty much identical specification no doubt based on the same reference design, what’s the differentiator? Price? Support? Delivery times? Whose HCL they appear on? Something else?

Making Money

So here’s the problem: where do Cisco, Juniper and Arista make their money if people can use any old hardware on which to run their software? Cisco and Juniper, I have to assume, make much of their money through the hardware side of the business rather than through licensing software. To be fair, Juniper licensing in general is fairly straightforward and tends not to get in the way of features for the most part. Cisco licensing on the other hand can be absurdly complex and at the top end, can be very pricy. Arista, I simply don’t know.

Hardware

Cisco and Juniper both make good money on hardware; you only have to order spares or upgrades to know that you are a total chump for paying what seems like an inflated price for things like SFPs, RAM, Flash Memory and so on. It’s incredibly annoying to have software that won’t accept anything but a vendor-branded SFP when you know that the branded SFP in one hand is made by the exact same OEM as the unbranded one in your other hand – presumably to the same specification – and the only difference is in the vendor code with which it identifies itself to the switch. So you end up buying the SFP with the vendor’s name on it for twice the price. Crazy. Arista’s hardware from what I’ve been told is priced very competitively, which is perhaps one reason why have done as well as they have. Arista basing their products on merchant silicon undoubtedly opened up the door to decent profit margins on a lower priced product. And there’s nothing to say that Cisco, Juniper and Arista couldn’t manufacture their own, branded, bare metal switches if they wanted, and use their scale to beat the prices down. They could make Quanta and the others appear like the D-Link of the bare metal world; but is that the market they want to be in?

Software

The money then must be in the software. How do you make a decent living selling software when you’re used to selling hardware? I don’t know; when I worked for a company selling software, the mantra was that selling software was key to profits because you write it once, and sell it many times. Once it’s written, it’s just the cost of burning the software to CDs, so every sale is gravy. Maybe that’s a rather oversimplified view, but the point is there; there’s money to made in software.

Microsoft Windows

Microsoft have battled this problem for a while. They make the software but other people make the hardware, and somehow Microsoft still makes money. Maybe Microsoft’s economy was one of scale; write it once, then resell it at a relatively low price many millions of times over. Then they got back into the hardware business – the Zune, Xbox, buying Nokia mobility… so maybe software isn’t all it’s cracked up to be. But if you get people hooked on your software and your interface at the low end, will they then buy your products when they need something bigger than a Top of Rack switch? Isn’t that the same reason why computer manufacturers like Apple tried to make sure schools and colleges had their computers? For a while I wondered if that was Cisco’s twisted logic when they bought Linksys – that perhaps if you like Cisco Linksys at home you’ll buy Cisco at work. Having used a few Linksys products, I’m fairly convinced that this could surely not have been the intent, or if it was, that somebody was seriously misguided.

The point is that unlike Microsoft selling Windows, you can’t sell many millions of copies of your NOS because there just isn’t that much hardware out there, so it would appear that there isn’t an economy of scale that would allow cheap software to pay for the work spent developing the NOS. And yet Cumulus and BigSwitch are right in that market, doing exactly what I just said sounds impossible, selling software only. If they can do it, why can’t Cisco, Arista and Juniper who also have a hardware sales stream to back them up?

Will They, Won’t They?

I’m going to wave a flag in the air as to whether each of these three vendors will make their NOS available for use on bare metal switches and probably regret it later, but my guesses are that:

• Juniper: WILL NOT
• Cisco: WILL NOT
• Arista: WILL

I think Arista has the most to gain from broadening their user base through licensing their NOS on approved white box switches. Juniper I don’t think will want the hassle of supporting multiple hardware platforms in addition to their own. Cisco I think probably should make IOS available, but I think deep down it goes against every bone in their corporate body to sell the crown jewels without the display box they come in. I do like Jim Duffy’s theory that they might do that just to drive people back to their products; that’s kind of a fun thought.

This is of course a separate argument from whether to offer products for virtualized environments; all three companies are offering products in this space. Only Cisco also sells an x86 hardware platform that it could run on, so there’s an implicit benefit to them there. Either way though, to support cloud services and NFV, they all have to play in this space and will continue to develop their offerings, targeting a seamlessly managed hybrid between the physical network products and the virtual.

Discuss

I’m curious to hear your opinions based on what you’ve heard or know. Do you think we’ll see these vendors licensing their NOS for bare metal platforms? I’d be delighted to be wrong.

 

30 Blogs in 30 Days

This post is part of my participation in Etherealmind’s 30 Blogs in 30 Days challenge.