Aug 012019
 

I went and did a thing. I ported OwnTrack Recorder to FreeBSD.

In this post:

  • FreeBSD 12
  • owntracks/recoder 0.8.4
  • I refer to owntracks/recorder as ot-recorder.
  • The FreeBSD service is known as otrecorder
  • On FreeBSD, ot-recorder runs as the ot-recorder user, created by the package. I did not want it running as root.
  • ot-recorder installs mosquitto by default, because it needs it’s libraries. That is also the reason why it installs curl. I chose to install ot-recorder in the same jail as my existing mosquitto instance for this reason.
  • The FreeBSD package stores data in locations which conform to man 7 heir. The changes from the default are:
    • The data is located under /var/db/ not /var/spool/
    • The html is at /usr/local/www/ot-recorder not /var/spool/owntracks/recorder/htdocs
    • The configuration file is /usr/local/etc/ot-recorder/ot-recorder.conf not /etc/default/ot-recorder

    These changes are documented by this patch.

I was interested in OwnTracks, a tracking application available on iPhone and Android.

The port process was fun and I learned much more about daemon than I had previously known.

In this post, I’ll show you how I installed and configured www/ot-recorder.

Some background

Before you get started, here is some information so you know what you are getting into.

To get into tracking, I am using an app on my phone. It talks to an mqtt server, which I had previously installed as part of another project.

ot-recorder subscribes to this mqtt instance, grabs the data sent by my cellphone, and stores it in a database. The date can then be displayed on a map, also by ot-recorder, which creates a web-based interface.

I configure an Nginx proxy in front of this in order to provide an https interface.

The install

The install is easy.

[dan@mqtt01:~] $ sudo pkg install ot-recorder
...
===> Creating groups.
Using existing group 'ot-recorder'.
===> Creating users
Using existing user 'ot-recorder'.
...
[dan@mqtt01:~] $

After installation, there are three directories of note:

[dan@mqtt01:~] $ ls -ld /var/log/ot-recorder /var/run/otrecorder /var/db/owntracks/recorder/
drwxr-x---  3 ot-recorder  ot-recorder  3 Jul 28 21:25 /var/db/owntracks/recorder/
drwxr-xr-x  2 ot-recorder  ot-recorder  5 Aug  1 16:24 /var/log/ot-recorder
drwxr-xr-x  2 ot-recorder  ot-recorder  3 Aug  1 19:31 /var/run/otrecorder
[dan@mqtt01:~] $ 

Respectively, this is where ot-recorder stores:

  • log file
  • pid file
  • tracking data

Initialization

When running ot-recorder for the first time, you should issue this command:

[dan@mqtt01:~] $ echo 'ot-recorder --initialize' | sudo su -fm ot-recorder
[dan@mqtt01:~] $ 

It is non-destructive; it can be run again without losing your data.

The command creates files in /var/db/owntracks:

[dan@empty:~] $ sudo find /var/db/owntracks
/var/db/owntracks
/var/db/owntracks/recorder
/var/db/owntracks/recorder/store
/var/db/owntracks/recorder/store/ghash
/var/db/owntracks/recorder/store/ghash/data.mdb
/var/db/owntracks/recorder/store/ghash/lock.mdb
[dan@empty:~] $ sudo ls -l /var/db/owntracks/recorder/store/ghash/        
total 20
-rw-r--r--  1 ot-recorder  ot-recorder  28672 Aug  1 21:40 data.mdb
-rw-r--r--  1 ot-recorder  ot-recorder   8192 Aug  1 21:40 lock.mdb
[dan@empty:~] $ 

I think my goal for a future port-update: allow for: service ot-recorder initialize

Configuration

This is my /usr/local/etc/ot-recorder/ot-recorder.conf configuration, with some slight redactions:

#(@)ot-recorder.default
#
# Specify global configuration options for the OwnTracks Recorder
# and its associated utilities to override compiled-in defaults.
# Lines beginning with # are comments
# *** In libconfig versions < 1.4 a trailing semicolon is mandatory

# -----------------------------------------------------
# Storage directory
#

# OTR_STORAGEDIR="/var/spool/owntracks/recorder/store"

# -----------------------------------------------------
# Address or hostname of the MQTT broker
#

OTR_HOST="myhost.example.org"

# -----------------------------------------------------
# Port number of the MQTT broker. Defaults to 1883.
# MQTT can be disabled by setting this to 0.
#

# OTR_PORT=1883

OTR_PORT=8883

# -----------------------------------------------------
# Username for the MQTT connection
#

OTR_USER="recorder"

# -----------------------------------------------------
# Password for the MQTT connection
#

OTR_PASS="[not my real password]"

# -----------------------------------------------------
# QoS for MQTT connection
#

# OTR_QOS=2

# -----------------------------------------------------
# MQTT clientid (default is constant+hostname+pid)
#

OTR_CLIENTID="ot-recorder"

# -----------------------------------------------------
# Path to PEM-encoded CA certificate file for MQTT (no default)

OTR_CAFILE="/usr/local/etc/ssl/cert.pem"

# -----------------------------------------------------
# Address for the HTTP module to bind to (default: localhost)
#

OTR_HTTPHOST="127.1.0.201"

# -----------------------------------------------------
# Port number for the HTTP module to bind to (default: 8083)
#

# OTR_HTTPPORT=8083

# -----------------------------------------------------
# optional path to HTTP directory in which to store
# access log. Default is to not log access

# OTR_HTTPLOGDIR=""

# -----------------------------------------------------
# API key for reverse-geo lookups
#

OTR_GEOKEY="opencage:[my-api-key]"

# -----------------------------------------------------
# Reverse geo precision
#

# OTR_PRECISION=7

# -----------------------------------------------------
# Browser API key for Google maps
#
#OTR_BROWSERAPIKEY="[my api key]"

# -----------------------------------------------------
# List of topics for MQTT to subscribe to, blank separated in a string
#

OTR_TOPICS="owntracks/#"

Of note:

  1. ot-tracker will present an HTML interface at 127.1.0.201: I want this so I can place an Nginx proxy in front of it.
  2. You can ignore the API keys for now.
  3. OTR_CAFILE is required for the TLS connection to your MQTT server.

Testing your configuration

This is how you can test the configuration from the command line, without starting the service:

$ echo 'ot-recorder owntracks/#' | sudo su -fm ot-recorder
ot-recorder 23283 - - version 0.8.4 starting with STORAGEDIR=/var/db/owntracks/recorder/store
ot-recorder 23283 - - connecting to MQTT on localhost:1883 as clientID ot-recorder-empty.int.unixathome.org-23283 without TLS
Error: Protocol not supported
$ 

Oh, that’s the wrong MTQQ settings. But you get the idea.

1883 isn’t right. 8883 is TLS, I should be using that.

I updated the OTR_PORT port in /usr/local/etc/recorder/ot-recorder.conf.

$ echo 'ot-recorder owntracks/#' | sudo su -fm ot-recorder 
ot-recorder 62346 - - version 0.8.4 starting with STORAGEDIR=/var/spool/owntracks/recorder/store
ot-recorder 62346 - - connecting to MQTT on localhost:8883 as clientID ot-recorder-mqtt01.int.unixathome.org-62346 without TLS
ot-recorder 62346 - - HTTP listener started on 10.55.0.10:8083
ot-recorder 62346 - - Using storage at /var/spool/owntracks/recorder/store with precision 7
ot-recorder 62346 - - Disconnected. Reason: 0x7 [Connection refused: TLS error]
ot-recorder 62346 - - MQTT connection: rc=7 [The connection was lost.] (errno=0; No error: 0). Sleeping...

Instead of invoking ot-recorder like this, you could be doing:

service ot-recorder start

and monitoring the log file at /var/log/ot-recorder/ot-recorder.log

The last attempt above was when I realized my mosquitto instance was not properly configured for port 8883 and was not providing a TLS connection.

My mosquitto configuration (from /usr/local/etc/mosquitto/mosquitto.conf) is:

pid_file /var/run/mosquitto.pid
# this is TLS
bind_address 10.55.0.10
port 8883 10.55.0.10
user mosquitto

allow_anonymous false

certfile /usr/local/etc/ssl/mqtt01.int.unixathome.org.fullchain.cer
keyfile  /usr/local/etc/ssl/mqtt01.int.unixathome.org.key
cafile   /usr/local/etc/ssl/cert.pem

ciphers ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA:ECDHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES256-GCM-SHA384:AES128-GCM-SHA256:AES256-SHA256:AES128-SHA256:AES256-SHA:AES128-SHA:DES-CBC3-SHA:HIGH:!aNULL:!eNULL:!EXPORT:!DES:!MD5:!PSK:!RC4

password_file /usr/local/etc/mosquitto/mosquitto.passwd

connection_messages true
log_dest            syslog

NOTE: that is my internal MTQQ installation, and not the external one to which my cellphone talks.

pid_file /var/run/mosquitto.pid

listener 8883
certfile /usr/local/etc/ssl/mymqtt.example.org.fullchain.cer
keyfile  /usr/local/etc/ssl/mymqtt.example.org.key
cafile   /usr/local/etc/ssl/cert.pem

user     mosquitto

ciphers ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA:ECDHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES256-GCM-SHA384:AES128-GCM-SHA256:AES256-SHA256:AES128-SHA256:AES256-SHA:AES128-SHA:DES-CBC3-SHA:HIGH:!aNULL:!eNULL:!EXPORT:!DES:!MD5:!PSK:!RC4

password_file /usr/local/etc/mosquitto/mosquitto.passwd

connection_messages true
log_dest            syslog

connection      internal-mtqq
address         internal-mtqq.example.org:8883

topic           # out 0
remote_clientid internal-mqtt
remote_username internal-mqtt
remote_password [redacted]

bridge_cafile   /usr/local/etc/ssl/cert.pem

That public MTQQ instance is bridged to my internal MTQQ.

Now I can start ot-recorder properly and it connects.

For future reference, what took me a long time to realize that:

Jul 29 21:14:58 my-mqtt mosquitto[7077]: New connection from [redacted] on port 8883.
Jul 29 21:14:58 my-mqtt mosquitto[7077]: Socket error on client , disconnecting.

or

Jul 28 21:48:23 mqtt01 mosquitto[75575]: Client connection from 10.55.0.10 failed: error:1408F10B:SSL routines:ssl3_get_record:wrong version number.

.. where attempt to do TLS when TLS was not enabled.

When not configured, my attempts to connect showed this:

[dan@pro02:~] $ openssl s_client -connect mtqq.example.org:8883
CONNECTED(00000003)
4678809196:error:140790E5:SSL routines:ssl23_write:ssl handshake failure:s23_lib.c:177:
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 0 bytes and written 308 bytes
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : 0000
    Session-ID: 
    Session-ID-ctx: 
    Master-Key: 
    Key-Arg   : None
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    Start Time: 1564415648
    Timeout   : 300 (sec)
    Verify return code: 0 (ok)
---

Eventually I figured out my MTQQ configuration errors and proceeded.

Nginx Proxy

The Nginx proxy is based on the documentation with my own preferences installed:


#user  nobody;
worker_processes  1;

# This default error log path is compiled-in to make sure configuration parsing
# errors are logged somewhere, especially during unattended boot when stderr
# isn't normally logged anywhere. This path will be touched on every nginx
# start regardless of error log location configured here. See
# https://trac.nginx.org/nginx/ticket/147 for more info. 
#
#error_log  /var/log/nginx/error.log;
#

#pid        logs/nginx.pid;


events {
    worker_connections  1024;
}


http {
    include       mime.types;
    default_type  application/octet-stream;

    #log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
    #                  '$status $body_bytes_sent "$http_referer" '
    #                  '"$http_user_agent" "$http_x_forwarded_for"';

    #access_log  logs/access.log  main;

    sendfile        on;
    #tcp_nopush     on;

    #keepalive_timeout  0;
    keepalive_timeout  65;

    #gzip  on;

server {
    listen       10.55.0.10:80;
    server_name  myowntracks.example.org;

    error_log     /var/log/owntracks-error.log;
    access_log    /var/log/owntracks-access.log combined;

    return 301 https://$server_name$request_uri;
}


server {
    listen       10.55.0.10:443 ssl http2;
    server_name  owntracks.int myowntracks.example.org unixathome.org;

    error_log     /var/log/owntracks-error.log;
    access_log    /var/log/owntracks-access.log combined;

    ssl_certificate     /usr/local/etc/ssl/myowntracks.example.org.fullchain.cer;
    ssl_certificate_key /usr/local/etc/ssl/myowntracks.example.org.key;

    ssl_protocols TLSv1.2 TLSv1.1 TLSv1;

    # Diffie-Hellman parameter for DHE ciphersuites, recommended 2048 bits
    ssl_dhparam /usr/local/etc/ssl/dhparams.pem;

    ssl_prefer_server_ciphers on; 
  
    ssl_ciphers "EECDH+AESGCM:EDH+AESGCM:ECDHE-RSA-AES128-GCM-SHA256:AES256+EECDH:DHE-RSA-AES128-GCM-SHA256:AES256+EDH:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA:ECDHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES256-GCM-SHA384:AES128-GCM-SHA256:AES256-SHA256:AES128-SHA256:AES256-SHA:AES128-SHA:DES-CBC3-SHA:HIGH:!aNULL:!eNULL:!EXPORT:!DES:!MD5:!PSK:!RC4";
    ssl_ecdh_curve secp384r1; # Requires nginx >= 1.1.0
    ssl_session_timeout  10m;
    ssl_session_cache shared:SSL:10m;
    ssl_session_tickets off; # Requires nginx >= 1.5.9
    resolver_timeout 5s; 
    add_header Strict-Transport-Security "max-age=63072000; includeSubDomains; preload";
    add_header X-Frame-Options DENY;
    add_header X-Content-Type-Options nosniff;
    add_header X-XSS-Protection "1; mode=block";

    # Proxy and upgrade WebSocket connection
    location /ws {
        rewrite ^/(.*)    /$1 break;
        proxy_pass      http://127.1.0.201:8083;
        proxy_http_version  1.1;
        proxy_set_header    Upgrade $http_upgrade;
        proxy_set_header    Connection "upgrade";
        proxy_set_header    Host $host;
        proxy_set_header    X-Forwarded-For $proxy_add_x_forwarded_for;
    }

    location / {
        proxy_pass      http://127.1.0.201:8083/;
        proxy_http_version  1.1;
        proxy_set_header    Host $host;
        proxy_set_header    X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header    X-Real-IP $remote_addr;
    }

    # OwnTracks Recorder Views
    location /view/ {
         proxy_buffering         off;            # Chrome
         proxy_pass              http://127.1.0.201:8083/view/;
         proxy_http_version      1.1;
         proxy_set_header        Host $host;
         proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
         proxy_set_header        X-Real-IP $remote_addr;
    }
    location /static/ {
         proxy_pass              http://127.1.0.201:8083/static/;
         proxy_http_version      1.1;
         proxy_set_header        Host $host;
         proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
         proxy_set_header        X-Real-IP $remote_addr;
    }

    # HTTP Mode
    location /pub {
        auth_basic              "OwnTracks pub";
        auth_basic_user_file    /usr/local/etc/nginx.htpasswd;
        proxy_pass              http://127.1.0.201:8083/pub;
        proxy_http_version      1.1;
        proxy_set_header        Host $host;
        proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header        X-Real-IP $remote_addr;

        # Optionally force Recorder to use username from Basic
        # authentication user. Whether or not client sets
        # X-Limit-U and/or uses ?u= parameter, the user will
        # be set to $remote_user.
        proxy_set_header        X-Limit-U $remote_user;
    }
}
}

Getting your first data point

Configure your phone as documented. Press the Share button in the top right corner of the app. Monitor your MTQQ and ot-recorder logs. It should eventually work with a little bit of attention.

Website Pin Facebook Twitter Myspace Friendfeed Technorati del.icio.us Digg Google StumbleUpon Premium Responsive