I’ve mentioned before I think that on my home network, DHCP service is provided by a pair of ubuntu servers running ISC DHCP servers in a redundant configuration. Part of the reason for this is pure nerdiness, and the rest is because it’s easy to use scripts to manage the configuration files (and so I do, for both BIND and DHCP). Well, plus if I’m going to run some stupid overcomplicated system at home, it had better work even if a server fails, hadn’t it?
One afternoon, I couldn’t get an address on the home network. My iPhone was fine but my computer was repeatedly failing to get an address. Odd, but maybe it was just unlucky timing. I headed to a computer that was hard wired to the network (and had a fixed IP, natch) and looked at the logs on the primary DHCP server. Want to guess what I found?
Oct 15 08:38:21 dns1 dhcpd: Failover CONNECT to failover-partner rejected: Connection rejected, time mismatch too great.
Oct 15 08:39:01 dhcpd: last message repeated 14 times
Oct 15 06:41:11 nettools dhcpd: failover: disconnect: time offset too large
Oct 15 06:41:16 nettools dhcpd: Failover CONNECT from failover-partner: time offset too large
I’ve picked a random sample from the log, but this had been going on for a day or more from the looks of it. I wasn’t too concerned about exactly how long, because I just needed to fix it.
Given that the logs were pretty straightforward in suggesting a time error, I checked the current system time and found that the
nettools server (secondary DHCP) was running about 5 hours slow. That led me to the discovery that in my haste to build that particular server (and I do tend to build them fairly bare to begin with and only add what I need), I had neglected to set up
DHCPNTP. Doh. Quick enough to fix, though:
$ sudo ntpdate 0.us.pool.ntp.org
$ sudo apt-get install ntp
Job done; set an accurate time, then install ntp.
Digging around, I then found that I had also failed to install or configure NTP on:
- some other ubuntu servers
- Juniper Secure Access
- VMWare ESXi
Needless to say, the time had been drifting wildly on those systems as well. All of the systems are now appropriately synced and NT) is running. It turns out, perhaps not surprisingly, configuring NTP on VMWare ESXi is a “Best Practice” too (whatever one of those is).
All Fixed; Lesson Learned
I need to add to my “DHCP Build Guide” a directive to configure
DHCPNTP. I’m looking at whether I can dockerize my DHCP primary/secondary servers at the moment, so that would be a good time to make sure that’s in the Dockerfile, don’t you think?
30 Blogs in 30 Days
This post is part of my participation in Etherealmind’s 30 Blogs in 30 Days challenge.