I think I mentioned in a previous post that I had written a script to catalog IP addresses, and I promised to describe it in more detail, so here we are.
This is a tool I’ve written almost three times now (I’ll explain below) for three different environments, but each time taking the same underlying approach. I’ve found it tremendously useful, and I’m curious to hear whether it’s something you think would help you as well.
At the moment the code is private, but if it’s something that people express interest in I’m not averse to tidying it up a little and putting it on github so that other people can try it out, and hopefully some people that code better than I can improve it, too.
I find myself frequently needing to know where an IP resides in the network, whether it’s a server or router, a load balancer VIP or loopback. Sure, I can get on equipment and traceroute, but it would at least be nice to know where to start. I also find that in networks without great interface descriptions in place, if I do a show route and get a next hop IP, I don’t always know on which device that IP resides.
The Sad and Bad
If you have a configuration archive, I’ll bet that you’ve tried to find an IP in the past by doing a grep on the files. If you’re seeking 192.168.1.112, you might try a grep with the full IP first, then if that didn’t give any results, you might try a grep for 192.168.1. Then if that didn’t work you might try just 192.168. What the grep cannot take into account is the subnet mask on each interface, which means it’s easy to get distracted by incorrect results that look like a match but actually are not.
The first attempt (the one that makes it “almost three” versions) began when I decided that I wanted to have every network device interface in DNS. There were way too many devices in the network to do this by hand, so I wrote a script that ran through the router configuration archives and pulled out all the IPs on interfaces and generated valid DNS names for all of them that could then be imported into our management DNS zone.
Attempts 2 and 3
The next two revisions of the script tried to solve problems in a more interactive fashion and provided much more functionality. Whereas Attempt 1 only mapped network interface IPs, the next versions mapped both interface IPs and attached subnets. Let me explain why this is important.
Let’s say you get a report of a problem with a server with IP address 10.5.1.5. You can do an nslookup if you like, and maybe you’ll find this out:
% nslookup 10.5.1.5 Server: 192.168.2.254 Address: 192.168.2.254#53 184.108.40.206.in-addr.arpa name = atlsrv101b1c1.yourcorp.com.
Helpful? Not to me as a network engineer, no. Knowing the hostname means little or nothing to me, and certainly doesn’t help me figure out which VLAN this host sits on, and which router(s) service that subnet. So DNS is clearly not enough to be truly useful when all you have is an IP address.
What I wanted was a tool that, when given an IP, could tell me what else was on the same IP subnet as that IP address and, if appropriate, what network device the IP existed on; and that’s the tool I wrote. Twice. The third version is much faster than the second though 🙂
Again my data source is a handy repository of configuration files. In each environment I’ve had a mixture of device vendors to deal with – Cisco, Juniper and f5, for example, and each vendor has had multiple configuration styles to deal with, so Cisco might include IOS and NXOS, Juniper would include Junos for MX, Junos for EX, Junos for SRX and legacy ScreenOS, and f5 helpfully changed their configuration archive formats between versions 10.x and 11.x.
The final version of the script has a configuration parser that runs through a file system containing Junos (in set format), IOS/NXOS, ScreenOS and f5 configurations. Each configuration file is opened, and parsed in order to extract the IPs that are configured. By parsing the configurations I’m able to extract things like, for example:
- Loopback IPs
- Interface IPs (including virtual IPs and subinterfaces)
- VRRP / HSRP addressing
- Interface IPs
- NAT Pools
- Firewall objects
- f5 Load Balancers (LTM)
- Self IPs
- Pool Members
- SNAT Pools
Not only does the script grab the IPs, but it also notes the subnet masks on each interface.
Dealing With Overlaps
Most networks have VRFs / routing-instances in use, and the f5s also support partitions which act in a very similar way. The script takes that into account and can handle overlapping IPs without issue. That does mean that if you have the same subnet in multiple VRFs and you query an IP on that subnet, you’re going to be given all the answers – it’s down to you to figure out which one is relevant; this isn’t magic, after all!
Yep, disabled interfaces these are taken into account as well. The script is smart enough to spot them and discount any IP addresses that had been identified on those interfaces.
Processing the Data
The configuration parser dumps out a file that contains all the IPs and subnet masks, along with information about each IP it found (e.g. device name, interface name and VRF). That file is processed once complete to convert the IPs and masks into something more akin to a binary string against which potential matches can be evaluated.
Attempt 2 at this script didn’t do this conversion and instead used a perl module that could compare IPs and tell you if the queried IP was an exact match, fell within the same subnet, or didn’t overlap at all with the stored IP subnets. This worked fine but slowed down noticeably as the data file grew over time, because that conversion and evaluation process within the module was wasting time for every IP it had to evaluate. With Attempt 3, the pre-processing allows me to act a little more like the router ASICs and do a binary comparison. This turns out to be terrifically fast; the script can evaluate an IP against 50,000 records or so in less than a second, which is not bad going.
Querying is achieved using a simple ‘front end’ script that takes the user’s query IP and compares it to the processed data file, then returns the results.
Querying the Data
So what does this look like? The tool is called ‘f’, short for its original longer name of ‘findip’:
% f 220.127.116.11 ---------------------------------------------------------- ipcheck (f) - Find IPs / Subnets / Address Book Entries * = Exact IP match ---------------------------------------------------------- 18.104.22.168: 22.214.171.124/32* lb01a SNAT mail_OUT (Part:Mail) 126.96.36.199/32* lb01b SNAT mail_OUT (Part:Mail) Checked 44200 records.
So for IP 188.8.131.52 we know that it’s a SNAT IP in the pool “mail_OUT” on device ‘lbo1a’ in the Mail partition. The asterisks (*) indicate that the IP matched exactly (i.e. this address is actually on the named device). Let’s try another one:
% f 10.1.0.1 ---------------------------------------------------------- ipcheck (f) - Find IPs / Subnets / Address Book Entries * = Exact IP match ---------------------------------------------------------- 10.1.0.1: 10.1.0.1/29* fw01 reth0.200 10.1.0.4/29 ce-rt01 irb.200 (corp) 10.1.0.5/29 ce-rt01 irb.200 (corp) 10.1.0.6/29 ce-rt01 irb.200 (corp)-vrrp Checked 44200 records.
In this case as indicated by the asterisk, 10.1.0.1 is an IP on the firewall fw01, on interface reth0.200. There are three other IPs shown though, and they are the other IPs found on the same subnet as the queried IP, and they live on a CE router (ce-rt01) on irb.200 in the corp routing-instance.
What if the IP you have is not on a network device? That’s ok because we match on subnets, so while DNS might tell you what the hostname is, the ‘f’ tool will tell you implicitly what VLAN and data center that subnet lives in, and what network devices are on that subnet. That latter information can be incredibly useful because it can alert you to another possible path in and out of the subnet:
% f 192.168.1.112 ---------------------------------------------------------- ipcheck (f) - Find IPs / Subnets / Address Book Entries * = Exact IP match ---------------------------------------------------------- 192.168.1.112: 192.168.1.1/24 core01a irb.1336-vrrp (dmz) 192.168.1.1/24 core01b irb.1336-vrrp (dmz) 192.168.1.2/24 core01a irb.1336 (dmz) 192.168.1.3/24 core01b irb.1336 (dmz) 192.168.1.251/24 f5-lb01a SelfIP (Part:Common, VLAN:INSIDE) 192.168.1.252/24 f5-lb01a SelfIP (Part:Common, VLAN:INSIDE) 192.168.1.251/24 f5-lb01b SelfIP (Part:Common, VLAN:INSIDE) 192.168.1.253/24 f5-lb01b SelfIP (Part:Common, VLAN:INSIDE)
No asterisks, so no exact match for this IP – it’s probably a server on that subnet; but we now know that the IP hangs off core01a/b on vlan 1336 in the dmz routing instance. We also now know that the same subnet hangs off f5-lb01a in the Common partition on a VLAN named “INSIDE”.
If you only want exact matches, naturally there’s an option to do that. I also mentioned that the script grabs firewall objects, and so it does. They’re suppressed by default, but adding a “-f” flag will reveal them.
The parsing and processing scripts run once a day. In fact currently I run this as a distributed system with parsing and processing taking place on servers around the world, then I have a script that syncs all the servers’ local data between all locations. I also add in some manually-controlled entries that represent site IP blocks. For example. if you have allocated 10.1.0.0/16 to the New York Data Center but you don’t manage that site, it’s useful to get a result when you search for 10.1.10.1 that tells you which data center it’s associated with. Yes, you could find that in your IPAM system, spreadsheet or whatever you use – but this is so much neater I think.
The scripts are written in perl (it’s my go-to language, sorry), so can run on pretty much any system in theory. All the users need is for the query script to be accessible to them (ideally in their binary search path).
So what do you think? Would this be useful to you? Is there another tool out there that already does this (and better)? I find it useful, but I’d be interested to see what other people think. Thanks!
Don’t apologize for Perl
I think this could be very useful – although not in my current role (as you know).
At a past client, I got a request to find an IP address on the network (I think it was a printer acting up or something). Traceroute, telnet to the last hop router, show ARP table (for MAC address), show MAC table (for interface), show CDP neighbor (for IP address of access switch), telnet to access switch, show MAC table – and other relevant port information.
After multiple requests for this, I figured this could “easily” be scripted since our network was pretty standard – Cisco IOS at core and distribution layers, CatOS at access layer. The access port descriptions all were configured with the wall jack identifier so given an IP and this process, I could tell you physically where it was located – call it micro-IP-geo-location.
My script was a Windows batch file that wrapped my Cisco Router Action Performing Perl Script (CRAPPS) which used Net::Telnet::Cisco to do the interactions and the batch file processed outputs to create the next commands.
After multiple more requests for me running the script, I put a web interface on it and hosted on our Network Ops SharePoint site. Anyone could put in an IP address and get it’s location.
Of course, this is now done with much more pizzazz in products like NetBrain that can dynamically trace out / map a device / network real-time and do lots value-add troubleshooting while you’re at it.
.. BUT NOTHING BEATS FREE!
Locating IPs was always one of the more useful (and less used) functions within CiscoWorks RME as I recall. But doing it on demand, while slightly slower because it’s real time, is ultimately going to be more accurate. I’ve debated writing a similar tool for similar reasons; especially with a large layer 2 domain it’s incredibly annoying to track down the physical port to which a server is attached (for example). You have inspired me to get my butt in gear and finally write that tool here, so thank you! 🙂
I use a program called NetDisco for this. It’s in Perl, it’s open source, and it works!
That would be nice to have such script published as open source. Community will benefit from it.
I would love to have this tool. I can absolutely see a use for this in my daily life.
If you are willing, I would like to see your code to see if I can use it for my needs.
I just listened to your podcast with Ivan Pepelnjak, sounds like a great tool, was really hoping it was available for general use… Have you thought any more about putting it up on Github?
Would you believe that I put it up on github about a week ago? Timing!
I am going to post about it, but meanwhile:
Try not to laugh too hard, ok? 🙂