I’ve mentioned before I think that on my home network, DHCP service is provided by a pair of ubuntu servers running ISC DHCP servers in a redundant configuration. Part of the reason for this is pure nerdiness, and the rest is because it’s easy to use scripts to manage the configuration files (and so I do, for both BIND and DHCP). Well, plus if I’m going to run some stupid overcomplicated system at home, it had better work even if a server fails, hadn’t it?

One afternoon, I couldn’t get an address on the home network. My iPhone was fine but my computer was repeatedly failing to get an address. Odd, but maybe it was just unlucky timing. I headed to a computer that was hard wired to the network (and had a fixed IP, natch) and looked at the logs on the primary DHCP server. Want to guess what I found?

Syslog Check

Primary DHCP

Oct 15 08:38:21 dns1 dhcpd: Failover CONNECT to failover-partner rejected: Connection rejected, time mismatch too great.
Oct 15 08:39:01  dhcpd: last message repeated 14 times

Secondary DHCP

Oct 15 06:41:11 nettools dhcpd: failover: disconnect: time offset too large
Oct 15 06:41:16 nettools dhcpd: Failover CONNECT from failover-partner: time offset too large

I’ve picked a random sample from the log, but this had been going on for a day or more from the looks of it. I wasn’t too concerned about exactly how long, because I just needed to fix it.

Simple Checks

Given that the logs were pretty straightforward in suggesting a time error, I checked the current system time and found that the nettools server (secondary DHCP) was running about 5 hours slow. That led me to the discovery that in my haste to build that particular server (and I do tend to build them fairly bare to begin with and only add what I need), I had neglected to set up DHCPNTP. Doh. Quick enough to fix, though:

$ sudo ntpdate 0.us.pool.ntp.org
$ sudo apt-get install ntp

Job done; set an accurate time, then install ntp.

Further Checks

Digging around, I then found that I had also failed to install or configure NTP on:

• some other ubuntu servers
• Juniper Secure Access
• VMWare ESXi

Needless to say, the time had been drifting wildly on those systems as well. All of the systems are now appropriately synced and NT) is running. It turns out, perhaps not surprisingly, configuring NTP on VMWare ESXi is a “Best Practice” too (whatever one of those is).

All Fixed; Lesson Learned

I need to add to my “DHCP Build Guide” a directive to configure DHCPNTP. I’m looking at whether I can dockerize my DHCP primary/secondary servers at the moment, so that would be a good time to make sure that’s in the Dockerfile, don’t you think?

 

30 Blogs in 30 Days

This post is part of my participation in Etherealmind’s 30 Blogs in 30 Days challenge.