Oct 042013
 

Last week, while at EuroBSDCon in Malta, I noticed that one of my servers had the wrong time. It was Bacula who told me, through this message in one of the backup jobs:

28-Sep 21:59 nyi-fd JobId 144899: DIR and FD clocks differ by -5 seconds, FD automatically compensating

Fixing the time

I connected to all my systems, and ran date(1). One system was by 2 seconds, and another was off by 5 seconds. I mistakenly thought the problem might be the upstream ntp servers I was using. I altered /etc/ntp.conf and stopped using the freebsd.pool.ntp.org servers and started using the us.pool.ntp.org servers instead.

Then I restarted ntp and looked at the log file:

Sep 29 06:29:59 slocum ntpd[1353]: ntpd exiting on signal 15
Sep 29 06:29:59 slocum ntpd[1779]: ntpd 4.2.4p5-a (1)
Sep 29 06:30:00 slocum ntpd[1780]: sendto(18.85.44.118) (fd=22): Operation not permitted
Sep 29 06:30:01 slocum ntpd[1780]: sendto(72.8.140.240) (fd=22): Operation not permitted
Sep 29 06:30:02 slocum ntpd[1780]: sendto(199.102.46.72) (fd=22): Operation not permitted

Oh. Umm. That is clearly the firewall. I added the following rules to /etc/pf.conf:

# get ntp working
pass out quick on $EXT_IF      inet proto tcp from any to any port ntp flags S/SA keep state
pass out quick on $EXT_IF      inet proto udp from any to any port ntp            keep state

Where EXT_IF is set to the NIC you are using (for me, em0).

That fixed it that server, which was running FreeBSD 9.1-RELEASE.

The other server didn’t show the same errors, and was running FreeBSD 9.1-RELEASE-p6. As I’m typing this, I do not recall how I got that server time corrected.

Checking the time

In case this problem occurred again, I wanted to find out about it as soon as possible. I started searching for Nagios plugins.

I found http://exchange.nagios.org/directory/Plugins/Network-Protocols/NTP-and-Time/check_daytime/details but I went with check_ntp_time from nagios_plugins from the FreeBSD ports tree. This plugin will compare the time on the Nagios server to that of the server you are checking. I’m looking for any changes more than 0.5s.

I already had an NTP host group in Nagios, so it was relatively simple to add a check_time to the host group. By taking that approach, each host in the host group then had that check added to their Nagios regime.

Website Pin Facebook Twitter Myspace Friendfeed Technorati del.icio.us Digg Google StumbleUpon Premium Responsive