In my Let’s Encrypt implementation, I am using a centralized acme.sh solution which generates all the certificates I use and authenticates via dns-01 challenges. I use anvil to distribute those certificates.
In this post, I will describe how the website pulls the certificates down from the rsync-jail. I will assume you have read my previous post where I describe the cert-shifter process.
Configuring the jail host
This solution assumes that the acme.sh jail and the rsync jail are on the same host. This allows files written in one jail to be shared, read-only via a nullfs mount, with another jail.
Here is that nullfs I use:
/usr/jails/certs/var/db/certs-for-rsync /usr/jails/certs-rsync/var/db/certs-for-rsync nullfs ro 0 0
/usr/jails/certs/var/db/certs-for-rsync is mounted in the acme.sh jail (in my case, the actual jail name is certs).
/usr/jails/certs-rsync/var/db/certs-for-rsync is the mount point with the rsync jail (which I named certs-rsync). The data contained there is cannot be written by anything in the rsync jail. This is a security design consideration. In short, any time you can share data read-only, you should.
The webserver configuration
I’m describing the the webserver side first because it requires an ssh-key which will be needed for the rsync jail configuration.
On the webserver, I created a directory, /var/db/certs-for-rsync/ and a user, rsyncer.
rsyncer:*:1003:1003:User &:/home/rsyncer:/usr/sbin/nologin
For what it’s worth, I use the same user for sending and receiving. The rsync jail will have the same user defined.
I created a passphraseless ssh key:
ssh-keygen -t ed25519
Please be aware of the security implications of a such a key. Treat it carefully.
That command will create two files, id_ed25519 and id_ed25519.pub. You’ll need the .pub key in the next section.
Configuring the rsync jail
In the rsync jail, I created a user to be used by the incoming rsync request.
rsyncer:*:1003:1003:User &:/home/rsyncer:/usr/local/bin/bash
You will notice that this user has a different shell than on the webserver. You cannot use /usr/sbin/nologin (as in the webserver) when accepting an incoming ssh connection.
I then added ~rsyncer/.ssh/authorized_keys with this entry:
from="10.1.2.3,webs01.example.org",no-agent-forwarding,no-port-forwarding,no-X11-forwarding,command="/usr/local/sbin/rrsync /var/db/certs-for-rsync/certs" ssh-ed25519 [REDACTED] rsyncer@webs01.example.org
I keep this file restricted:
$ sudo ls -l ~rsyncer/.ssh/authorized_keys -rw-r----- 1 rsyncer rsyncer 276 Jul 13 22:40 /home/rsyncer/.ssh/authorized_keys
The contents of that file is important. It tightly restricts the incoming ssh connection so it can use rsync only upon the given directory. I outline each part.
- from=”10.1.2.3,webs01.example.org” – Incoming connections are accept only from 10.1.2.3, which is also webs01.example.org. All other connections will be refused. Thus, I allow connections to the rsync jail only from the webserver in question.
- no-agent-forwarding,no-port-forwarding,no-X11-forwarding – Restrict various things that can be done with an ssh session. It reduces the attack surface.
- command=”/usr/local/sbin/rrsync /var/db/certs-for-rsync/certs” – When the ssh connection succeeds, run this command. rrsync is a Perl script supplied with rsync which restricts the rsync to the indicated directory. Should the webserver be compromised, all that can be done with ssh-key is rsync of data from the rsync jail, all of which is already on the webserver.
- ssh-ed25519 [REDACTED] rsyncer@webs01.example.org – the public part of the ssh key which was created on the webserver. What is shown is the contents of the id_ed25519.pub file.
That is all you need to do on the rsync jail.
The webserver’s crontab
This is the crontab I created for the rsyncer user on the weberver:
# mail any output to `dan', no matter whose crontab this is MAILTO=dan@example.org PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/games:/usr/local/sbin:/usr/local/bin:/usr/X11R6/bin:$HOME/bin; export PATH 27 * * * * /usr/local/bin/rsync -e "/usr/bin/ssh -i /home/rsyncer/.ssh/id_ed25519" --quiet --recursive -avzz rsyncer@rsync-jail: /var/db/certs-for-rsync/
I will describe some of that:
- -e “/usr/bin/ssh -i /home/rsyncer/.ssh/id_ed25519” – This indicates the remote shell to use. It specifies to use ssh and the identity_file to use (i.e. the ssh key we created earlier).
- rsyncer@rsync-jail: – when rsync invokes ssh, this is the remote user and the remote host. There is no remote host specified, so we will grab the home directory. The actual directory is controlled by the authorized_keys file we configured earlier.
- /var/db/certs-for-rsync/ – This is the local directory where rsync will store the received files.
Under normal operations, you should get no output from this cronjob. If you do, it’s probably because an error has occurred and you need to fix something.
That’s everything
After the cronjob runs, you should see this stuff:
[dan@www01:~] $ ls -l /var/db/certs-for-rsync/ total 56 drwxr-xr-x 2 rsyncer rsyncer 5 Jul 5 11:04 certs.unixathome.org drwxr-xr-x 2 rsyncer rsyncer 5 Jul 12 11:04 illjustparkhere.com drwxr-xr-x 2 rsyncer rsyncer 5 Jul 15 13:26 lists.bsdcan.org drwxr-xr-x 2 rsyncer rsyncer 5 Jul 15 13:26 lists.freebsddiary.org drwxr-xr-x 2 rsyncer rsyncer 5 Jul 15 13:26 lists.freshports.org drwxr-xr-x 2 rsyncer rsyncer 5 Jul 15 13:26 lists.freshsource.org drwxr-xr-x 2 rsyncer rsyncer 5 Jul 15 13:26 lists.langille.org drwxr-xr-x 2 rsyncer rsyncer 5 Jul 15 13:26 lists.pgcon.org drwxr-xr-x 2 rsyncer rsyncer 5 Jul 15 20:19 lists.unixathome.org drwxr-xr-x 2 rsyncer rsyncer 5 Jul 12 11:04 services.unixathome.org drwxr-xr-x 2 rsyncer rsyncer 5 Jul 14 20:03 svn.int.unixathome.org [dan@www01:~] $
The next step: download and install those certs where they are required. That will be my next blog post.
