We know the routine. You have a desktop, and a laptop, or perhaps two laptops. You want your files in both places. A shared, remotely mounted directory is not ideal. Instead, let’s have the systems synchronize themselves.
That’s where syncthing comes in:
Syncthing replaces proprietary sync and cloud services with something open, trustworthy and decentralized. Your data is your data alone and you deserve to choose where it is stored, if it is shared with some third party and how it’s transmitted over the Internet.
Installing and running
I decided to install syncthing within an existing jail, called fileserver.
I was using Syncthing on FreeBSD as my starting place. My experiences differed, specifically:
- I did not have to manually create /var/log/syncthing.log
- I did not have to create /var/run/syncthing.pid
Why? I attribute that to the port evolving, but I really do not know.
I installed via pkg install syncthing, and I had already built it on my poudriere server.
I started it with: service syncthing start
All very standard stuff.
What did I see?
$ ps auwwx | grep sync syncthing 12450 20.1 0.0 121732 52104 - SNJ 21:06 0:02.16 /usr/local/bin/syncthing -home=/usr/local/etc/syncthing -logfile=/var/log/syncthing.log -no-browser syncthing 12447 0.5 0.0 119940 46884 - SJ 21:06 0:00.39 /usr/local/bin/syncthing -home=/usr/local/etc/syncthing -logfile=/var/log/syncthing.log -no-browser syncthing 12446 0.0 0.0 10812 2248 - SsJ 21:06 0:00.00 daemon: /usr/local/bin/syncthing[12447] (daemon) root 12865 0.0 0.0 11112 2576 2 R+J 21:06 0:00.00 grep sync $ ls -l /var/log/syncthing.log -rwxr-xr-x 1 syncthing syncthing 2362 Mar 23 21:07 /var/log/syncthing.log $ sockstat -l -4 | grep syncthing syncthing syncthing 12450 13 tcp4 10.0.0.1:22000 *:* syncthing syncthing 12450 14 udp4 10.0.0.1:45834 *:* syncthing syncthing 12450 15 udp4 10.0.0.1:21027 *:* syncthing syncthing 12450 18 tcp4 10.0.0.1:8384 *:* $ # ls -l /usr/local/etc/syncthing total 59 -rw-r--r-- 1 syncthing syncthing 615 Mar 23 21:06 cert.pem -rw------- 1 syncthing syncthing 3761 Mar 23 21:06 config.xml -rw-r--r-- 1 syncthing syncthing 672 Mar 23 21:06 https-cert.pem -rw------- 1 syncthing syncthing 288 Mar 23 21:06 https-key.pem drwxr-xr-x 2 syncthing syncthing 7 Mar 23 21:06 index-v0.14.0.db -rw------- 1 syncthing syncthing 288 Mar 23 21:06 key.pem
Nothing spectacular at all.
When I tried to browse to 10.0.0.1:8384 I got a ‘check host error’ which is documented.
I am sure this can be fixed by other solutions, but I installed Nginx and create a reverse proxy, which is outside scope for this blog post, but see https://docs.syncthing.net/users/reverseproxy.html.
This is the virtual host I used:
server {
listen 10.0.0.1:80;
listen 10.0.0.1:443 ssl http2;
ssl_protocols TLSv1.2 TLSv1.1 TLSv1;
server_name syncthing.int.unixathome.org;
root /usr/local/www/syncthing.int.unixathome.org;
error_log /var/log/syncthing.int.unixathome.org.error.log info;
access_log /var/log/syncthing.int.unixathome.org.access.log combined;
ssl_certificate /usr/local/etc/ssl/syncthing.int.unixathome.org.fullchain.cer;
ssl_certificate_key /usr/local/etc/ssl/syncthing.int.unixathome.org.key;
location / {
proxy_pass https://127.0.0.1:8384/;
proxy_set_header Host localhost;
}
}
Not shown here, is my creation of a Let’s Encrypt certificate, and configuration of anvil to pull the cert down from my distribution server.
I also created an entry in DNS for syncthing.int.unixathome.org which has the same IP address as fileserver.int.unixathome.org.
Opening that URL in my browser showed me this:
Configuration
My first step, seeing as this server runs ZFS, I will create a dedicated filesystem for syncthing and mount it within this jail.
[dan@slocum:~] $ sudo zfs create -o recordsize=512K -o mountpoint=/iocage/jails/fileserver/root/usr/local/syncthing system/data/syncthing [dan@slocum:~] $
From within the jail, I saw:
[dan@fileserver:~] $ ls -ld /usr/local/syncthing/ drwxr-xr-x 2 root wheel 2 Mar 23 21:50 /usr/local/syncthing/ [dan@fileserver:~] $ sudo chown syncthing:syncthing /usr/local/syncthing/ [dan@fileserver:~] $ sudo chmod o= /usr/local/syncthing/ [dan@fileserver:~] $ ls -ld /usr/local/syncthing/ drwxr-x--- 2 syncthing syncthing 2 Mar 23 21:50 /usr/local/syncthing/ [dan@fileserver:~] #
I adjusted the directory permissions so that only syncthing had access.
Via the web GUI, I clicked on settings and changed the Device Name to syncthing.int.unixathome.org, so it matched the DNS entry, as opposed to the host name.
I also changed Default Folder Path from ~ to /usr/local/syncthing.
I manually changed /usr/local/etc/syncthing/config.xml and changed
path="//Sync"
to
path="/usr/local/syncthing"
You should also set the GUI Authentication User and GUI Authentication User values on the GUI tab. This controls access to your syncthing instance.
I also disabled relays, via Actions | Advanced | Options | Relays Enabled. Relays are required unless all your systems have direct connections. All my hosts are on the same VPN so I disabled this.
Then I restarted the service:
$ sudo service syncthing restart Stopping syncthing. Waiting for PIDS: 12447. Starting syncthing.
Now the web GUI looks like this:
Introducing my laptop to my server, syncthing style
I was doing this on my MacBook Pro, and I had previously installed Syncthing.
On the server, I clicked on Show Id. I copied the ID and browsed to http://127.0.0.1:8384/ where I saw this:
I clicked on +Add Remote Device. I pasted the device ID of my server, added the name of my server, and clicked on Save.
Back on the server, I saw a message:
That IP address is not the IP address of my laptop. It’s the relay server that my laptop is using. When took this screenshot, I had not yet disabled relays.
After adding my laptop, the web GUI for my server looked like this:
This screen shot is actually from the next day, when I sat at a cafe typing up this part. From there, I had added a second latop, air01, and got it working.
Setting the address
Please ignore this section if you are using relays.
Instead of using relays, I want my hosts to connect directly to each other. This is possible because they are all on the same VPN. If firewalls/NAT/etc are involved, you might have more work to do before taking this step.
On my server, I clicked on my laptop under Remote Devices, then on Edit. In the addresses field I put: tcp://air01-vpn-startpoint.int.unixathome.org:22000.
I did this for all devices on all hosts.
Sharing a folder
ON my laptop I did this:
[dan@air01:~] $ mkdir Testing [dan@air01:~] $ touch Testing/SharedFileTest [dan@air01:~] $
The following steps allowed me to share that directory with my server:
- In the web GUI for the air01 laptop, I clicked on Add Folder.
- In Folder Label, I put Testing.
- In Folder Path, I put /Users/dan/Testing, but it defaulted to this value because of what I put in the Label field.
- On the Sharing tab, I checked the box beside my server name.
- Then I clicked on Save.
On the server web GUI, I saw this message:
I clicked on Add and saw this:
I looked in /usr/local/syncthing/ where I saw:
[dan@fileserver:~] $ sudo ls -l /usr/local/syncthing total 6 drwxr-xr-x 2 syncthing syncthing 2 Mar 23 22:01 .stfolder drwxr-xr-x 3 syncthing syncthing 4 Mar 24 16:30 Testing [dan@fileserver:~] $ sudo ls -l /usr/local/syncthing/Testing total 2 drwxr-xr-x 2 syncthing syncthing 2 Mar 24 16:30 .stfolder -rw-r--r-- 1 syncthing syncthing 0 Mar 24 16:27 SharedFileTest [dan@fileserver:~] $
What next?
I deleted all my testing folders, and went BIG. I added my Documents folder. Now that’s syncing:
My next goal: external file verisioning with zfs snapshots.
