When I made a stab at defining SD WAN recently, I noted that Cisco’s IWAN solution had provided a bit of a contrast to some of the other Software Defined WAN solutions I’d seen; not in a bad way, but I was certainly interested in the approach.

SD WAN Definition

I’m going to “do a Joe Onisick” here and quote myself as a reference for what I might be hoping for from the Cisco IWAN solution:

“SD WAN is a solution that uses real time WAN link performance monitoring and data packet inspection to autonomously manage the distribution of network traffic across multiple, likely heterogenous, WAN links with the aim of improving and optimizing WAN performance in alignment with the business requirements.” – John Herbert

One thing my definition doesn’t mention is how these systems get deployed, and since that’s interesting, perhaps let’s start there.

IWAN Zero Touch Deployment

It seems to me that ZTD has become a checkbox requirement for all the SD WAN solutions, and perhaps it’s about time. Zero Touch Deployment in the context of SD WAN means being able to ship a box to a spoke site, have ‘Dumb Hands’ on site plug in the LAN and WAN Ethernet cables, and the box will find the Internet connection, “dial home”, get configuration from the hub site and automatically integrate into the existing corporate WAN. There are a few different ways that this is being offered right now, but Cisco has opted for a solution involving the Dumb Hands also plugging in a USB flash drive that, presumably, was shipped from corporate HQ to site. The limited configuration on the flash drive allows the router to connect to the hub site, exchange certificates and bring up the encrypted WAN connection. It’s possible to do this with smart hands and no flash drive too; the pnp (yes, “plug and play”) command hierarchy is the way to configure it. Either way, you plugs it in, you gives it the basic dial home information onnastick, and it joins the WAN.

“It”, by the way, is a Cisco ISR-series (e.g. an ISR4000). The hub site would be running something like an ASR1000. You’ll also need a small x86 server with 8 vCPUs and 64GB RAM to run the VM for APIC-EM (Application Policy Infrastructure Controller – Enterprise Module), which oversees the whole thing and tells the ISR/ASRs what to do.

APIC-EM

APIC-EM is Cisco’s way of bringing their Application Centric Infrastructure (ACI) to the WAN. This is the central point of abstraction, and is used for configuration, control and analytics for the IWAN, so it’s really the brains of the operation. As you’d hope, there are APIs to allow third party components to be built to integrate with built-in features. It’s also available free via DevNet last time I checked, which is a nice idea, but I believe for production it still requires a license based on the number of managed devices.

New devices joining the WAN are actually registering with APIC-EM and setting up a control channel, and APIC-EM will configure the endpoints. Interestingly in this SDN WAN solution, device configuration is achieved using SNMP and CLI techniques.

I am truly disappointed that this hasn’t been rolled out using something like NETCONF/YANG, though it was hinted that perhaps that was on the cards for the future. For me it speaks loudly and clearly to a gap in Cisco’s deployment of modern configuration standards to their routers. If you haven’t caught me whining about the variety of different NETCONF implementations in IOS, NXOS, IOS-XR and IOS-XE, and the incompleteness of each of them, trust me when I say that this is an area that needs a lot of work in order to become credible. I’m sure it’s happening, but there’s a lot of legacy disappointment out there to poison the well, so to speak. I had hoped that IWAN would be a driver to put proper, consistent, and complete XML RPC implementations on the ASR/ISR platforms, but consider the opportunity missed for the moment at least.

As a final note on APIC-EM, while it is ultimately the boss off the nodes participating in the IWAN, the operational functions are actually split between centralized (for simplicity and agility) and distributed (where performance and scale are needed), but they still all get managed through this single touchpoint.

Underlay, Overlay, Wombling Free

If you’re British, and of a certain age, I just gave you an ear worm you really didn’t want. If you’re not, you can join in by quickly catching up on the reference and you can pretend you knew all along.

Anyway, Cisco IWAN is an overlay WAN network, which make sense since there could be multiple WAN links in each site, each with different bandwidths and performance properties. Keeping the connectivity transport agnostic is a good move, and by encrypting the overlay tunnels, it also adds security to the connections whether the underlay is the Internet or a private WAN link. Specifically, the overlay in use is good old DMVPN (Dynamic Multipoint VPN). This gives Cisco an immediate advantage over some of their competitors, in part because that DMVPN is fairly well established and loved by all who have ever tried to configure it, but also because the dynamic meshing of spoke sites (i.e spoke to spoke communications) is standard operating procedure, and allows for performance optimizations that might not be possible in other architectures.

Performance Routing

In terms of policy, if multiple WAN connections are available, configured applications will tend to stay ‘sticky’ to a particular WAN link (with a second alternative preferred path ready to shift to if needed), while other general traffic will be spread across all of the links to distribute the load.

The WAN load distribution is achieved using Cisco’s PfR3 (Performance Routing v3), and overlay routing information is exchanged using either EIGRP or BGP. Key to being able to offer BGP is the BGP Dynamic Neighbors capabiity, without which Zero Touch Deployment with dynamic WAN addressing would be a non-starter.

Network Based Application Recognition

Application policies are pushed to the ISR/ASRs by APIC-EM, and those devices then use NBAR2 to identify the live application flows and apply policy. Thankfully in addition to recognizing standard protocols and ports, NBAR2 is also smart enough to spot a DNS request for an Office365 server, note the response, then use that destination IP to identify Office365 traffic. Cisco aren’t alone in using this technique, but it has become a “must have” for application identification courtesy of the ever-changing IPs associated with cloud-based applications.

The spoke devices can also monitor application performance and compare to a baseline expectation (e.g. measured jitter for IP voice data) and request that the hub router (I think) use a different path. Clever stuff.

But Wait, There’s More

Oh yes there is. Missing from the descriptions above are options to compress WAN traffic or perhaps offer caching services. Cisco has thought about those too, and offers WAN Optimization via WAAS, and also intelligent caching using Akamai Connect, which appears to be a fairly smart platform. All of this as I understand it occurs in those edge routers, and does not require additional hardware for deployment (though I’m sure will require an additional license, because, well, licensing).

But Is It SD WAN?

Based on my definition above, yes absolutely. What interests me most about Cisco’s solution here is the use of existing technologies (DMVPN, PFR, BGP, EIGRP, NBAR, WAAS) in a new way to achieve a better overall solution. This is a theme that is seen in a number of other SD WAN solutions, some of which I’ll be looking at in future posts. I find it quite reassuring, in fact, that we do not have twenty vendors all building brand new proprietary protocols to run their SD WAN infrastructure, or I fear we would spend years debugging the new protocols as much as we would dealing with the intricacies of dynamic path management in the wide area network.

Cisco has a captive audience to which it can sell the IWAN solution, and I suspect it will do fairly well.