Unwrapping Tangled Device Configurations – A10 Networks Edition

If you’ve ever tried to interpret an A10 Networks load balancer configuration, or some Cisco Modular QoS CLI commands, you’ll know that doing so involves following references to other parts of the configuration, inevitably ones that appear earlier in the configuration than where you are now, using a display pager which doesn’t support a back command to scroll up a page at a time. In short, it’s a huge pain. The same applies to Cisco ACE and CSM load balancer configurations. The modularity is beautiful and logical, but it’s a massive irritation to reverse engineer.

Spaghetti

Spaghetti Configurations

I work regularly with A10 Networks load balancers. ACOS (the A10 OS) has a CLI and configuration format that’s very similar to Cisco IOS. Looking at a particular vPort (A10 terminology for a particular Virtual IP (VIP) and Port combination) and trying to figure out which real servers are related to it is irritating to say the least. Here’s an example of the configuration in the order it appears when you view it:

ip nat pool pool1 10.100.1.1 10.100.1.126 netmask /25
!
health monitor checkstatus
   method http url /status expect code 200
!
slb server server1 1.2.3.4
   port 80 tcp
!
slb service-group sg1 tcp
   health-check checkstatus
   member server1:80
!
slb template http mytemplate1
   request-header-insert "X-Forwarded-Proto: http" insert-if-not-exist
   insert-client-ip X-Forwarded-For replace
   insert-client-port X-Forwarded-Port replace
!
slb virtual-server myvip1 10.1.1.1
   port 80
      template http mytemplate1
      service-group sg1
      source-nat pool pool1
!

As can be seen, a number of elements contain references to other elements. For those familiar with F5 or Cisco ACE rather than the A10 load balancer, here’s a brief mapping of some key terminology:

A10 F5 ACE
server pool-member / node / server rserver
service-group pool serverfarm
virtual-server (aka vserver) virtual (ip+port) virtual-address
port (aka vport) virtual (ip+port) virtual-address
nat pool snat pool nat-pool

With that in mind, here’s a diagram of the same A10 configuration indicating the backwards lookups required to fully understand it:

A10 Config References

Worse, the configuration above doesn’t even include all the possible configuration references that might exist. Don’t get me wrong, the configuration is totally logical and I can’t see a better way of entering it without introducing massive repetition, but for anybody reading the configuration it’s a nightmare. Even worse, when an internal customer asks to see the configuration for their VIPs, I can actually see their eyes glaze over when they see the wall of configuration and try to make sense of it.

A10 View-o-matic

And so, because I’m fundamentally all about making things simpler, I created a little tool called the A10 View-o-matic, which processes A10 configurations so they can be viewed on a web page while automatically resolving and embedding all the references for you. The idea isn’t to view the entire configuration, but rather to focus on the VIPs and the configuration that makes them work, because that’s what my users are actually interested in. Here’s an example:

A10 View-o-matic

At first glance it looks like the regular virtual-server configuration, but there are a few additions:

  • some references are resolved inline (e.g. the nat pool range has been added in parentheses in the source-nat line)
  • where there’s a resolved reference with more than one line, or that doesn’t suit an inline reference, there’s a [+] sign which will display the related configuration which is there but hidden (a technique sometimes known as a fold).
  • in some cases (as will be shown below), the configuration has been simplified into something that is easier to understand.

Expanding the folds for the first VIP shows some of the information it’s possible to embed:

First VIP Expanded

When the service-group is expanded, there’s a comment added that the group (uses default tcp health-check). The client-ssl template expands to show the referenced SSL cert/key, and where the client-ssl template has a link to an SSL cipher template, that too has been expanded and included in the fold. Here’s the second VIP:

Second VIP Expanded

The service-group in this second VIP has a non-default health-check applied, so another [+] appears within the first fold. Expanding it, predictably, reveals all:

Second VIP Expanded Again

Note also that service-group member OwnerID_othervip-pr.mycorp_192.168.2.98 has its IP expanded inline (192.168.2.98) right after it. In this case, the server’s name includes the IP, but in the instances where a server has just been called server1 for example, it’s much clearer to see server 1 (192.168.2.98) right there within the expansion.

It’s All In Your Head

A10 View-o-matic Header

The header bar allows the user to Expand All (which, as the name implies, expands all folds with one click), the user can jump to any partition by selecting it in the dropdown, or can move to view the configuration for any A10 they are permitted to view. I’ve also set up the ability to filter the output based on VIP name. This works well in my case because I helped develop a standard for naming objects within the A10, including the use of an integer identified as part of the virtual-address name, replaced by the text OwnerID in this example. Using this convention I can easily produce a view of all VIPs relating to, say, owner 1234 by specifying that the VIP name should contain 1234_. This capability allows me to generate web pages for different user groups where they can only see their own VIPs and, equally importantly, they will only see referenced configuration that’s currently in use on their VIPs.

Built-In Alternative

Interestingly, the A10 command line does include a command to do a little bit of automatic referencing for you:

a10-lb01#show slb virtual-server bind
Total Number of Virtual Services configured: 363
---------------------------------------------------------------------------------
*Virtual Server : OwnerID_yetanother.mycorp.com 192.168.57.10   All Up

    +port 80  http ====>OwnerID_sg1-80  State :All Up
    +OwnerID_someserver1:80 10.100.10.40         State : Up
    +OwnerID_otherserver2:80 10.100.10.71        State : Up

    +port 443  http ====>OwnerID_sg2-443    State :All Up
    +OwnerID_server6:8888          10.20.30.120        State : Up
    +OwnerID_server6:8888          10.20.30.121        State : Up
    +OwnerID_server6:8888          10.20.30.122        State : Up
    +OwnerID_server6:8888          10.20.30.123        State : Up

The only irritation is that the show slb virtual-server bind does not allow a single vserver to be viewed; it’s an all or nothing deal (heaven only knows why). Still, for interactive checks it’s a good command to know, and it includes service status, which my script — based on the configurations — does not. I’ve been considering a version which could get element status in real time, but that would be getting perhaps a little bit more complex than I want to get for what was originally supposed to be a quick hack to allow a service owner to see their own configuration.

Next Steps

At some point I’ll find the time to refactor the code a little, figure out why I needed hacks in a couple of places to make it work properly, and fix those bits. Then maybe I’ll shove it up on github to be pulled apart.

Update: The code is now available in github as https://github.com/jgherbert/a10-2-html.

The thing is though, it makes me think there are two things needed in the CLIs of these devices:

  1. A pager facility with a [b]ack capability. More recent IOS includes that, and I’ve requested the same feature from A10 (though as of yet it hasn’t happened);
  2. A command to show the virtual-server configurations with their related config included in place, almost like a more extensive version of Junos OS’ | display inheritance, to pull in the stuff that isn’t immediately obvious but is implied by the configuration being looked at.

As ever, I dare to dream. Hope this was vaguely interesting to some of you.

Be the first to comment

Leave a Reply

Your email address will not be published.


*


This site uses Akismet to reduce spam. Learn how your comment data is processed.