With this new server, I am taking a new approach. Each jail will have at least three IP addresses:
- The public IP4 address, used by internet facing services (e.g. http or https)
- The public IPv6 address, similar to the above
- A VPN address, used for system administration and private services (e.g. nrpe)
In this article, I will assume you are familiar with ezjail, FreeBSD jails, basic networking, OpenVPN, and ZFS. That is, this is not a post for those new to FreeBSD. This is an advanced topic.
The goal is to access the jail, and services in that jail, from within the VPN. This private network will be used for system administration, monitoring, and for backups. This means I can install nrpe and bacula-fd into each jail and access those services over OpenVPN.
Create a jail
Let’s start with the jail creation. We will give it three IP addresses:
- 18.104.22.168 – the main IP address and the one you’ll connect to once I get a website running on it
- 2610:1c1:0:4::3 – an IPv6 address for the same place
- 10.80.0.67 – an IP address for private services, not accessible by the public (e.g. nrpe, bacula-client)
Here is the command to create that jail. I am using a flavour which sets up this jail as an Ansible client / node.
$ sudo ezjail-admin create -f ansible-slave bsdcan 22.214.171.124,2610:1c1:0:4::3,10.80.0.67 /usr/jails/bsdcan/. /usr/jails/bsdcan/./etc /usr/jails/bsdcan/./etc/locate.rc /usr/jails/bsdcan/./etc/crontab /usr/jails/bsdcan/./etc/gss ... /usr/jails/bsdcan/./root/.bash_profile /usr/jails/bsdcan/./root/.bashrc 19 blocks find: /usr/jails/bsdcan/pkg/: No such file or directory Warning: IP 126.96.36.199 not configured on a local interface. Warning: Some services already seem to be listening on all IP, (including 188.8.131.52) This may cause some confusion, here they are: openvpn openvpn 43169 4 udp4 *:12510 *:* Warning: IP 2610:1c1:0:4::3 not configured on a local interface. Warning: Some services already seem to be listening on all IP, (including 2610:1c1:0:4::3) This may cause some confusion, here they are: openvpn openvpn 43169 4 udp4 *:12510 *:* Warning: IP 10.80.0.67 not configured on a local interface. Warning: Some services already seem to be listening on all IP, (including 10.80.0.67) This may cause some confusion, here they are: openvpn openvpn 43169 4 udp4 *:12510 *:*
I have no idea why they think OpenVPN is on that port.
Next, I will manually add the IP addresses to the system:
ifconfig em0 alias 184.108.40.206/32 ifconfig em0 alias 10.80.0.67/32 ifconfig em0 inet6 2610:1c1:0:4::3 prefixlen 64 alias
To make sure these addresses are assigned at boot time, I added these corresponding entries to /etc/rc.conf:
ifconfig_em0_alias0="inet 220.127.116.11/32" # bsdcan ifconfig_em0_alias1="inet 10.80.0.67/32" # bsdcan ifconfig_em0_alias2="inet6 2610:1c1:0:4::3 prefixlen 64" # bsdcan
I added a comment to the end of each line so I know what jail belongs to that particular alias.
At this point, this is what the NIC looks like on the jail host:
$ ifconfig em0 em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=4219b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC,VLAN_HWTSO> ether 74:d0:2b:98:19:22 inet 18.104.22.168 netmask 0xffffff00 broadcast 22.214.171.124 inet6 fe80::76d0:2bff:fe98:1922%em0 prefixlen 64 scopeid 0x2 inet6 2610:1c1:0:4::2 prefixlen 64 inet 126.96.36.199 netmask 0xffffffff broadcast 188.8.131.52 inet6 2610:1c1:0:4::3 prefixlen 64 inet 10.80.0.67 netmask 0xffffffff broadcast 10.80.0.67 nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> media: Ethernet autoselect (1000baseT <full-duplex>)
Start the jail
Let’s start that jail:
$ sudo ezjail-admin onestart bsdcan Configuring jails:. Starting jails: bsdcan. $ jls JID IP Address Hostname Path 1 184.108.40.206 bsdcan /usr/jails/bsdcan
Connect into the jail
Let’s connect into that jail via the console:
$ sudo ezjail-admin in console bsdcan FreeBSD 9.2-RELEASE (GENERIC) #0 r255898: Thu Sep 26 22:50:31 UTC 2013 Welcome to FreeBSD! Before seeking technical support, please use the following resources: o Security advisories and updated errata information for all releases are at http://www.FreeBSD.org/releases/ - always consult the ERRATA section for your release first as it's updated frequently. o The Handbook and FAQ documents are at http://www.FreeBSD.org/ and, along with the mailing lists, can be searched by going to http://www.FreeBSD.org/search/. If the doc package has been installed (or fetched via pkg_add -r lang-freebsd-doc, where lang is the 2-letter language code, e.g. en), they are also available formatted in /usr/local/share/doc/freebsd. If you still have a question or problem, please take the output of `uname -a', along with any relevant error messages, and email it as a question to the questions@FreeBSD.org mailing list. If you are unfamiliar with FreeBSD's directory layout, please refer to the hier(7) manual page. If you are not familiar with manual pages, type `man man'. Edit /etc/motd to change this login announcement. root@bsdcan:~ # pkg info gettext-0.18.3.1 GNU gettext package libiconv-1.14_1 A character set conversion library pam_ssh_agent_auth-0.9.5 PAM module which permits authentication via ssh-agent perl5-5.16.3_4 Practical Extraction and Report Language pkg-1.2.3 New generation package manager python-2.7_1,2 The \"meta-port\" for the default version of Python interpreter python2-2_1 The \"meta-port\" for version 2 of the Python interpreter python27-2.7.6_1 Interpreted object-oriented programming language sudo-1.8.8 Allow others to run commands as root root@bsdcan:~ #
Ahh, good. All the right stuff was installed for this particular flavour. Those packages will me to configure this jail via Ansible. But more on that in a later blog post.
Looking at the jail network, it looks as expected:
# ifconfig em0 em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=4219b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC,VLAN_HWTSO> ether 74:d0:2b:98:19:22 inet 220.127.116.11 netmask 0xffffffff broadcast 18.104.22.168 inet 10.80.0.67 netmask 0xffffffff broadcast 10.80.0.67 inet6 2610:1c1:0:4::3 prefixlen 64 nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> media: Ethernet autoselect (1000baseT <full-duplex>) status: active
Making the Client LAN accessible from the VPN
Let us review the networking we have on this server:
- main IP address: 22.214.171.124
- OpenVPN tun0: inet 10.8.1.80 –> 10.8.1.81 netmask 0xffffffff
- Each jail will have an IP address in the 10.80.0.0/24 subnet (e.g. 10.80.0.66). I will refer to this as the Client LAN.
To route the Client LAN across OpenVPN, we modify the OpenVPN server. No change occurs on the client.
The best thing to read (and I mean read, not skim) is this post on OpenVPN/Routing, specifically Lans behind OpenVPN
These two lines go into openvpn.conf on the server:
route 10.80.0.0 255.255.255.0 push "route 10.80.0.0 255.255.255.0"
This line gets added to the ccd/zuul.example.org file (if you don’t know what ccd refers to here, you skimmed).
iroute 10.80.0.0 255.255.255.0
Restarting both the client and server enables everyone on this VPN to access the Client LAN.
There are a lot of moving parts here, especially with respect to the network traffic over tun0. But this is the first step. You may want to restrict network traffic on your VPN more than I am.