Mastodon.nl upgraded to latest release 4.0.2 last week

Last week we upgraded Mastodon.nl to Mastodon release 4.0.2. Because we moved platform the week before, we were running on a platform with some changes and unknowns that would come and let us shoot ourselves in the foot.

So the upgrade was less pleasant than the last one and took more time and effort than anticipated. But it ended quite well, now running the 'latest and greatest' and having fixed/addressed most issues identified during the last upgrade.

Up to two weeks ago the entire mastodon.nl setup consisted of a large Contabo VPS and a second one that wasn't actually used yet. This really got underwhelming performance wise with all new users coming in. We got the offer from ProcoliX to move our instance to their infra on a sponsor/donation basis.

We worked on getting the minimal setup during a Saturday night and have been working over the last two weeks to get all the loose ends ironed out. There is always more stuff that could be done, but I feel we are out of the woods for now.

By now we run mastodon on a collection of XCPNG/Xen Orchestra hosted virtual machines all running Debian 11. The setup is somewhat like this:

redundant loadbalancer
two webserver nodes running mastodon-web
two sidekiq nodes running all mastodon related queues
two PostgreSQL nodes, one in actual use at this time (postgres and pgbouncer)
single node NFS server serving ZFS hosted files
single redis node
single elasticsearch node
bastion host for management

A lot could of course be said about the specifics of our setup. But as scaling etc is still very much in flux, that would be too much of a moving target. Maybe later when things start to smooth out I will find the time for a nice drawing and some more specifics.