I’ll bet that at one time or other in your life, you’ve had your name written in your clothes. You know what I’m talking about – either you buy some specially made labels to stick in your clothing, or you use one of those laundry markers to scrawl it on the care label. Usually this is exclusive to children’s clothing, as perhaps they are most likely to leave their clothing lying around for somebody else to find; if their name is in it, the finder can clearly identify who the item belongs to and everybody will be happy, especially Mom and Pop who paid for the thing.
As adults however, it seems most of us don’t label our clothes – we assume that we are too clever to lose anything, so we become careless about labeling it. In reality, writing your name in your clothes takes seconds and may save you down the line, so it’s kind of stupid not to.
I know what you’re thinking – what on earth does this have to do with networking? Let me explain…
As a consultant, I spend my time going into networks I don’t know and I have to reverse engineer them to some degree before I can make a change safely. In doing so, I often find myself having to figure out what on earth the previous engineer(s) were actually trying to accomplish with particular parts of the configuration – and I often end up dumbfounded and shaking my head. I mean, I understand what the configuration does, but understanding the intent is quite another thing.
So here’s my plea to all router and switch engineers – when you’ve finished writing your name in your jacket, let’s be sure and do the same for the network devices you maintain. I’m a big proponent of self-documented networks, and this is not entirely altruistic as you can imagine – if it’s well documented, I can understand what’s going on without having to either guess or bug the network team and waste their time trying to recall why something was put in.
Let me explain what I’m on about. Here’s an example of a switch interface configuration snippet that’s fairly typical of what I find:
vlan 17 ! interface FastEthernet 1/3 switchport switchport mode access switchport access vlan 17 ! interface Vlan 17 ip address 10.17.3.1 255.255.255.0 end
Um, ok. So an unlabeled port is attached to VLAN 17 which is, um, not named, and the SVI for VLAN 17 is, well, I see the IP address but still have no clue as to what this represents. Here’s the kind of thing I’d prefer to see:
vlan 17 name DB_Svr_BackEnd ! interface FastEthernet 1/3 description SRVDB01A-hme0 switchport switchport mode access switchport access vlan 17 ! interface Vlan 17 description Database Server Back Ends ip address 10.17.3.1 255.255.255.0 end
I suspect that many of you are currently saying “That’s ridiculous! I always put a description on my interfaces!” And if you do, thank you! How are you with route-maps and ACLs?
route-map REDIST permit 10 match ip address 7 ! router ospf 1 redistribute bgp 65000 subnets route-map REDIST ! access-list 7 permit 10.1.1.0 0.0.0.255 access-list 7 permit 172.16.0.0 0.0.255.255
Again, my cup overfloweth not. I see what you’re doing, but have no information that might guide me as to why you chose those subnets to redistribute, or what they are. I would also argue that calling a route-map “REDIST”, while marginally helpful, doesn’t fully describe its purpose does it? Imagine you’re seeing this for the first time; wouldn’t it be nicer to see something like the following configuration instead?
route-map BGP_TO_OSPF_REDIST permit 10 description Advertise Data Center #3 subnets in OSPF match ip address DC3_Subnets ! router ospf 1 redistribute bgp 65000 subnets route-map BGP_TO_OSPF_REDIST ! ip access-list standard DC3_Subnets permit 10.1.1.0 0.0.0.255 permit 172.16.0.0 0.0.255.255
I think that’s a little clearer; and of course you can add additional terms to the route-map and give each one a description as required. I’d argue also that you should use a prefix-list rather than an ACL for most route redistribution, but that’s probably a discussion for another post.
There is a caveat to this of course – for it to work, you have to trust (or enforce) that all engineers honor the naming/descriptions, and don’t get lazy and just tag DC2 routes on to the DC3 ACL because it’s easier than creating a new DC2 ACL and route-map entry. In other words, you can’t do this stuff in isolation or the information shifts rapidly from helpful to misleading. A good change control / script review process can help with this of course, as well as regular beatings with a cluestick (or repurpose a LART as an EART for the Engineers).
Do you label your prefix-lists?
ip prefix-list DC3_Subnets description Data Center 3 (Harlow) Subnets ip prefix-list DC3_Subnets seq 5 permit 10.1.1.0/24 ip prefix-list DC3_Subnets seq 10 permit 172.16.0.0/16
It’s worth noting that you can only have one description in a prefix-list; if you try adding a second it simply overwrites the first one. Still, it’s better than nothing.
Do you name your static routes?
ip route 220.127.116.11 255.255.255.0 192.168.1.1 name DMZ_via_FW01
NXOS users can cry now, unless the ability to name static routes finally got added; it was certainly missing in 4.2 and I could kill Cisco for missing something so basic from NXOS. IOS users just need to remember that the name either has to be contiguous (e.g. use underscores or similar) or needs to be in quotes, e.g.
ip route 18.104.22.168 255.255.255.0 192.168.1.1 name "DMZ via FW01"
As I suggested above, smart self-documentation extends to naming your access-lists, prefix-lists, route-maps and so forth intelligently as well. One school of thought says that you should use a prefix to indicate the object type, like in programming when you name a variable to indicate its type, e.g. strName for a string containing a name. That can work for sure – it should certainly ensure that you are unambiguous about which object you are referencing in a command, although logically speaking you could argue that it’s obvious based on context whether you’re referring to the prefix-list DMZ_SUBNETS or the route-map DMZ_SUBNETS (the latter of which, however, is badly named because it doesn’t indicate what the route-map is intended to do 🙂 ).
Some common sense is required before you jump in, however. When you write your name in your clothing, you look at the label to figure out how big it is, determine whether you’re going to just put your initials, your whole name, or maybe just your last name, and you figure out how big you can write in order for it to fit in. Without wishing to stretch the analogy too far, the same applies when you start naming configuration elements.
route-map BGP_TO_OSPF_REDIST_FOR_DC3 permit 10
This route-map, for example, is likely to have a short shelf-life – the moment you want to redistribute anything other than DC3 routes from BGP to OSPF, the route-map name becomes meaningless and now you have the exciting task of either creating a newer and more appropriately named route-map and replacing that one, or you go the lazy route and just add those DC2 routes in there anyway because it’s easier. In other words – plan ahead; be generic with the route-map name, but be specific when creating descriptions for each term you add.
If you think ahead, your router configurations can act like a guide book for anybody new looking at the configurations; you can reduce on-boarding times for new team members because they’ll understand the network more easily. Troubleshooting becomes easier because everything is clearly labeled. Changes are less risky because it’s harder to misunderstand what’s going on. And perhaps most importantly (selfishly speaking), when I – or some other third party – comes in to help out with something, my brain will hurt just a little less.
OK, I think I’ve said enough. I’ll get my coat.
Wait a second – *checks label* – I believe this coat is yours…
Do you self-document? What are your tips for good config maintenance?
Let me know in the comments!
Excellent description of good practice.
In reference to your ACL example, I like to put remarks on them as well.
Modifying your example…
access-list 7 remark REDIST
access-list 7 permit 10.1.1.0 0.0.0.255
access-list 7 permit 172.16.0.0 0.0.255.255
I find your entire post relevant, especially on a router with 100+ IPSec tunnels when you can have a 100+ different ACL’s.
Arrrgh, numbered ACLs 😉 Dislike! *grin*
Good point about the remarks though.
Great post and awesome examples. If only everyone was so considerate to carefully comment their configs!
About documenting configuration and using descriptions: I found junos more comfortable than IOS.
Thanks, Misha. The Junos ‘annotate’ command certainly works well for commenting on a commands. Obviously you can still help yourself by naming things like filters and policies with sensible (helpful) names. And then there are comments – /* — */ style that you can include in the config file – but that get lost unfortunately when you load the configuration. Junos or IOS, every little helps!
Great post! I like your self-documented network approach and I use it in my daily practice. Thanks for some tips.
I like the thought. My favorite was a policy route named “REMARK” — it took 2 passes for my brain to register that “REMARK” was its name and not a remark command. And it was doing some DSCP remarking, or someone had tried to do it that way. Definitely a case of “huh?”
I’ve been suggesting sites have site design decision documents indicating “non-obvious” design elements, like why the heck you’ve got a static route here in the middle of otherwise dynamically routed stuff, odd QoS there, Policy Based Routing there. I’ve been in too many networks where nobody is left who knows why it was done that way, and so nobody dares make changes because it might break something. Even if what’s there looks like it’s doing some pretty ugly things.
Great point. I agree absolutely about some kind of design decision document. I achieved something similar in a previous role by changing the way the change scripts were written, to include (among other things) an explanation of the current state, the post-change state, the planned change and how/why the change will do what’s needed. Along with the actual IOS commands that make up the change, and using Microsoft SharePoint to store all the change script Word files, you could then search SharePoint for an IP or command string of interest and get back a handy list of the change scripts that made or altered the change you were wondering about. And because the scripts all explained themselves, you had your answer as to WHY something was done.
Of course, that didn’t help with the stuff that was put in in the 2-3 years preceding that change, but it was a great help from then onwards, and didn’t require any special additional documentation – although it did require that people followed a standardized template and actually filled in the ‘explanation’ bit. That took more time, but the result was in my opinion absolutely worth it, especially compared to how things were done when I first got there, which was to have a notepad (text) file containing the commands that Operations should issue on the devices – no explanation of why or why, no backout, and frequently, no testing. That approach is quick for sure, but deadly when something goes wrong – and from a change review perspective, almost impossible for another engineer to sign off on, as there was no explanation of the actual intent of the change against which to evaluate whether the change did what was intended…
Whatever way it gets implemented – whether in a Wiki, a design doc, “proper” (in my opinion) change documentation, your point though is well made. I’ve spent too many hours running ip accounting, gathering netflow stats, and running NAM captures in order to try and find out whether things are still valid and active, or are just left over trash from another age (without which you can’t remove them, of course :-).
And at some point, I should probably document in a blog post my “ideal change script” template, as it took a while to get to its final state, and in the end did quite a good job.
I try to take it a bit further. Stuff like:
Makes the configs prettier.;)