When feasible, I prefer to run things as non-root. A recent commit to net-snmp has made this possible. By its nature, being a new change, it took me some time and help to figure out what needed to be changed.
Before doing this yourself, I recommend waiting until the two code reviews mentioned below are committed.
In this post:
- FreeBSD 14.0-RELEASE-p6
- net-snmp-5.9.4_2,1
- librenms-24.5.0,1
- I include commands from different host; please do not be distracted by that
These are the configuration options I am using:
[12:53 zuul dan ~] % grep snmp /etc/rc.conf snmpd_flags="-a -r" snmpd_conffile="/usr/local/etc/snmpd.conf" snmpd_sugid="YES" snmpd_enable="YES" [12:54 zuul dan ~] %
snmpd_sugid is new with net-snmp-5.9.4_2,1 – that is what allows sniped to run as non-root (specifically, it runs as the snmpd user).
Permissions changes
The configuration files must be readable by the user in question.
My configuration files for snmpd are world readable – I could change them to chgrp snmpd.
If snmpd cannot read the file as the non-root user, it will start up and run fine, until log rotation occurs, and it gets a SIGHUP from newsyslog – then it won’t be able to read the configuration file, will continue running, but your extend commands will disappear.
[19:31 zuul-pg02 dvl ~] % ls -l /usr/local/etc/snmpd.conf -rw-r----- 1 root wheel 169 2021.07.23 19:25 /usr/local/etc/snmpd.conf
Here is how I avoided that issue:
[19:32 zuul-pg02 dvl ~] % sudo chmod o+r /usr/local/etc/snmpd.conf [19:32 zuul-pg02 dvl ~] % ls -l /usr/local/etc/snmpd.conf -rw-r--r-- 1 root wheel 169 2021.07.23 19:25 /usr/local/etc/snmpd.conf [19:32 zuul-pg02 dvl ~] %
There are no secrets in this file, so making it world-readable now is OK. I would be better off with chmod 0640 and chgrp snmpd. That is for future work.
Scripts
This is my configuration file:
[13:02 r730-03 dvl ~] % cat /usr/local/etc/snmpd.conf agentAddress udp:10.55.0.143:161,tcp:10.55.0.143:161 sysLocation BSD Cabal HQ sysContact dan@langille.org extend ntp-client /usr/local/etc/snmp/ntp-client.sh extend smart /usr/local/etc/snmp/smart extend zfs /usr/local/etc/snmp/zfs
I need to ensure all those scripts can be run as the snmpd user.
I ran all scripts as non-root to identify permission issues, such as this one.
[20:48 r730-03 dvl /usr/local/etc/snmp] % echo /usr/local/etc/snmp/smart | sudo su -fm snmpd Can't open '/usr/local/etc/snmp/smart.config' at /usr/local/etc/snmp/smart line 191.
That file cannot be read:
[20:49 r730-03 dvl /usr/local/etc/snmp] % ls -l /usr/local/etc/snmp/smart.config -rw-r----- 1 root wheel 99 2023.12.27 16:48 /usr/local/etc/snmp/smart.config [20:49 r730-03 dvl /usr/local/etc/snmp] % sudo chmod +r /usr/local/etc/snmp/smart.config
That script also caches some content. That needs to be accessible:
[20:52 r730-03 dvl /usr/local/etc/snmp] % ls -ld /var/cache/librenms drwxr-xr-x 2 root wheel 3 2023.08.16 21:00 /var/cache/librenms/ [20:52 r730-03 dvl /usr/local/etc/snmp] % sudo chown snmpd:snmpd /var/cache/librenms
This was already set by previous work:
[20:52 r730-03 dvl /usr/local/etc/snmp] % ls -l /var/cache/librenms total 5 -rw-r--r-- 1 snmpd snmpd 505 2024.05.23 20:50 smart
During this process, we also identified some log rotation changes that were required, specifically the permissions. I use this, slightly different from the current port. There are two code reviews ready for processing.
- newsyslog.conf change
- print a warning if the configuration files are not readable
I use this.
[13:02 r730-03 dvl ~] % cat /usr/local/etc/newsyslog.conf.d/net-snmp.conf # configuration file for newsyslog for net-snmp # # see newsyslog.conf(5) for details # # logfilename [owner:group] mode count size when flags [/pid_file] [sig_num] /var/log/snmpd.log snmpd:snmpd 644 7 * $D0 - /var/run/net_snmpd.pid
Identifying permission issues
During the conversion to non-root, I kept encountering the same problem: the extends data would disappear from the snmpd output. At midnight UTC, the LibreNMS logs indicated the extend data was remove after discovery.
I thought it might be related to log rotation. With some testing, I managed to find the problem could be triggered by sending the daemon a SIGHUP signal. Mark Johnston, the driving force behind the non-root changes, suggested I run ktrace. His instructions were:
- Add the following to rc.conf: snmpd_prepend=”ktrace -f /tmp/ktrace.out -i”
- Restart snmpd
- Send SIGHUP to snmpd
- Stop snmpd
- Send me a copy of /tmp/ktrace.out
I did that, but instead of sending him the file, I ran sudo kdump -f /tmp/ktrace.out | less and found:
89059 snmpd CALL open(0x2a22e81f7ec0,0) 89059 snmpd NAMI "/usr/local/etc/snmpd.conf" 89059 snmpd RET open -1 errno 13 Permission denied
That led to the realization it was a configuration file permission. After fixing the file permissions, running Discovery again found ‘nsExtendStatus.”bind” = active’ still present in the output.
The above page is accessed from the device itself. Look for the three vertical dots, click it, and choose Capture.
cron jobs?
This section added on 2024-05-29 – courtesy of Bridghead, Slater St. (re this tweet or this toot.).
For the MySQL snmpd plugin, there are some cron jobs involved, for caching. For example:
[11:16 mysql01 dvl ~] % cat /usr/local/etc/cron.d/snmpd-mysql # mail any output to `dan', no matter whose crontab this is MAILTO=dan@langille.org PATH=/etc:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin # for LibreNMS via /usr/local/etc/snmp/mysql */3 * * * * root /usr/local/etc/snmp/mysql > /dev/null 2>&1 [11:16 mysql01 dvl ~] % sudoedit /usr/local/etc/cron.d/snmpd-mysql
I changed root to snmpd:
*/3 * * * * snmpd /usr/local/etc/snmp/mysql > /dev/null 2>&1
Time will tell if that’s the right decision.
In summary
When running as non-root:
- Check configuration file permissions
- Check script permissions (can snmpd run the script?
- Any cache files involved? Check their permissions
- Check log file permissions
- Still not fixed, run ktrace
- crontabs – if you have cron jobs, are they running as the desired user?
Hope this helps.
