FreeBSD with jails

Posted on vr 15 december 2023 in runbsd

Last week I installed FreeBSD on an old and disregarded laptop I had laying around. I wanted to see if FreeBSD BHyve and/or FreeBSD Jails were concepts that would work well for me or at least would introduce some cool new feature or idea. Installing the laptop went unexpectedly well. I tried running Linux on this old machine in the past and that was rather cumbersome and underwhelming. But by now you can even run Free software on these closed platform beasts. I'm talking about the infamous first generation all USB-C Macbook Pro with the crappy keyboard from 2016.

Screenshot of toot of me working on this

So that was lat week and last night I started creating a couple of jails. This setup is meant as a basis for the new less power hungry homelab that will contain:

A basic webserver
repository manager
authentication service

Maybe I will add more later, but for now this is it.

The webserver and repository manager both run a single service. The authentication service will require some sort of database, so that will probably end up as two jails.

Right now there are some bit and pieces that I need to address and document, but creating this was rather easy and pleasant. There is enough information in the web on the subject, so getting to something that works is very much doable.

Next things on the list are using VNET to limit network traffic to and from the jails. Also I might look into moving the jails to their own ZFS filesystem/bucket/blob/whatever that is called in ZFS speak.

Notes for my own perusal:

creating a couple of jails and installing a new FreeBSD base in it:

mkdir -p /jail/webserver/
bsdinstall jail /jail/webserver
mkdir -p /jail/nexus/
bsdinstall jail /jail/nexus
mkdir -p /jail/keycloak
bsdinstall jail /jail/keycloak
mkdir -p /jail/dbms/
bsdinstall jail /jail/dbms

Then enable jails on the host:

sysrc jail_enable="YES"
sysrc jail_list="webserver nexus keycloak dbms"

and start the basic jails:

service jail start webserver
service jail start nexus
service jail start keycloak
service jail start dbms

root@hal9000:~ # more /etc/jail.conf
# 1. definition of variables that we'll use through the config file
$jail_path="/jail";
path="$jail_path/$name";

# 2. begin - default configuration for all jails 

# Some applications might need access to devfs
mount.devfs;

# Clear environment variables
exec.clean;

# Use the host's network stack for all jails
ip4=inherit;
ip6=inherit;
# Initialisation scripts
exec.start="sh /etc/rc";
exec.stop="sh /etc/rc.shutdown";

# specific jail configuration
dbms{}

keycloak{}

nexus{}

webserver {}

With the empty jails running, I added specific config to each one:

webserver:

added nginx and certbot

pkg -j webserver install nginx py39-certbot-2.6.0,1  py39-certbot-nginx-2.6.0
sysrc -j webserver nginx_enable="YES"

created a basic plain http nginx config for the jail
setup DNS entries and portforwarding to this machine

inside the jail (jexec webserver /bin/sh)

certbot certonly --agree-tos --no-eff-email -m letsencrypt@example.com -d server.example.com

and added a cron job to the jail, not sure this will work:

0 1 * * * /usr/local/bin/certbot renew --quiet --post-hook "/usr/sbin/service nginx restart"

then added proxy entries for web based services in other jails:

    server {
        listen       80;
        listen       443 ssl http2;
        server_name  server.example.com;

        ssl_certificate /usr/local/etc/letsencrypt/live/server.example.com/fullchain.pem;
        ssl_certificate_key /usr/local/etc/letsencrypt/live/server.example.com/privkey.pem;

        ssl_session_cache    shared:SSL:1m;
        ssl_session_timeout  5m;

        ssl_ciphers  HIGH:!aNULL:!MD5;
        ssl_prefer_server_ciphers  on;

        if ($scheme != "https") {
            rewrite ^(?!/\.well-known/acme-challenge)/ https://$http_host$request_uri? permanent;
        }

        location / {
            proxy_pass http://localhost:8081;
            proxy_buffering off;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-Host $host;
            proxy_set_header X-Forwarded-Port $server_port;
       }
    }

nexus:

added openjdk8 and manually installed nexus

pkg -j nexus add openjdk8
tar -xzvf nexus*.tgz  in /jail/nexus/opt/
cp /jail/nexus/opt/nexus/bin/nexus /usr/local/etc/rc.d/nexus

keycloak:

added openjdk17 and the keycloak package

pkg -j keycloak install keycloak

inside the jail (jexec keycloak /bin/sh)

/usr/local/share/java/keycloak/bin/kc.sh start --hostname auth.example.com --https-certificate-file /usr/local/etc/fullchain.pem  --https-certificate-key-file /usr/local/etc/privkey.pem --db dev-mem

dbms

# pkg -j dbms install mariadb1011-server-10.11.6 mariadb1011-client-10.11.6
# sysrc -j dbms mysql_enable="YES"
# service -j dbms mysql-server start

References

https://rderik.com/blog/running-a-web-server-on-freebsd-inside-a-jail/
https://gist.github.com/colinmcintosh/3a2d00dc0e5bb82fc3b0afc185065306
https://www.theserverside.com/blog/Coffee-Talk-Java-News-Stories-and-Opinions/How-to-setup-Nginx-reverse-proxy-servers-by-example