dave's blog

Installing Authentik for Authentication and Single-Sign-On

dave — Mon, 23 Oct 2023 22:15:36 +0000

Installing Authentik for Authentication and Single-Sign-On

Blog tags

single sign on

sso

authentik

docker

nginx

docker-compose

postgresql

python

django

OpenID

oauth2

dave Tue 24/10/2023 - 11:15

The OER Foundation offers many Free and Open Source Software services to our learners and educators to provide them with resources for learning, developing Open Educational Resources (OER), and collaborating - both professionally and socially - with one another. Due to this proliferation of 'point source' technologies, users have to create a myriad of user accounts, each requesting an email and password and perhaps a user name. For many, especially those who have not adopted a password keeper, this can be an onerous situation, and often results in very poor 'internet hygiene' due to people choosing simplistic passwords and using the same small set of passwords in many contexts.

An alternative to this default approach is for the OER Foundation to adopt an authentication and authorisation solution. Authentication means creating a single link between a real (physical) person and their online (virtual) identity - usually managed with a login and password and might also include one or more additional factors of identification, also known as 'Multi-Factor Authentication' or MFA. Authorisation involves allowing authenticated users access to systems to which they should have access, and not those to which they should not. An suitable authentication and authorisation system can then be used to implement a 'Single Sign-On' (SSO) system that provides the option to all those with credentials on it to access - subject to any authorisation limitations - any of the OER Foundation's services transparently without needing to create a new, separate account.

The Authentik project offers quite good documentation for Docker Compose installation, too. This tutorial should be seen as a complement to that, perhaps providing a bit more guidance.

Preparing a suitable server
Ground work
Configuring the reverse proxy
Getting your Let's Encrypt SSL certificate
Docker Compose configuration
Boom: Setting up your Authentik instance
Upgrading your Authentik
Backing up your Authentik data

Preparing a suitable server

All of our OER Foundation services share a common hosting platform. You'll need to create a replica of that platform - if you don't already have one - so you can add this service. Everything else in this post assumes you've followed those instructions!

Note, the Authentik developers suggest a minimum VPS specification of 2 CPU cores and 2 GB RAM, so a very small instance should suffice.

Ground work

To set up Authentik, you'll need a few key details specific to your instance. For the rest of this tutorial, we'll use the convention of representing those variables as a name inside [], for example, the domain name that identifies your instance, which is referred to by the token [domain name]. Note - you'll see a few other instances of [] in the content below - the ones with multiple ::: in them, e.g. [:::], are not tokens, they're the way IPv6

The tokens for which you'll need values are as follows:

your instance [domain name], i.e. a fully qualified domain name or a sub domain by which people will reach your instance. In our case, for example, we use 'auth.fossdle.org', which is the 'auth' subdomain of the 'fossdle.org' fully qualified domain name.
Authenticating SMTP (Simple Mail Transfer Protocol, i.e. the language for sending email) details - this is required so your services can send emails to users - crucial things like email address validation and password recovery emails...
- [smtp host] - the domain name or IPv4 or IPv6 address of an SMTP server
- [smtp port] - the port number on the server that is listening for your connection. By convention it's likely to be 465 or 587, or possibly 25.
- [smtp reply-to-email] - a monitored email to which people can send email related to this WordPress site, e.g. notifications@[domain name]
- [smtp username] - the username (often an email address) used to authenticate against your SMTP server, provided by your email provider.
- [smtp password] - the accompanying password, provided by your email provider.
- [smtp from email] - this is the email address from which the system emails will be sent - it's what recipients will see and the address to whom they'll be able to respond. You might want something like 'noreply@[domain name]'... or maybe (if you do accept responses) 'webmaster@[domain name]'.
[postgresql password] - a strong random password, say 12-99 characters long, sticking with letters and numbers as some symbols might be mis-interpreted in the context of the configuration file in which it's stored (e.g. if it contains a '#', the rest of the password might be interpreted as a comment. Or if you enclosed it in "", if it includes a " as a symbol, it might be interpreted as prematurely closing your double quotes). To generate, see below.
[authentik secret] - a random 'secret key' of 50 characters is recommended. Again, see below.

To generate (relatively) random passwords/keys, you can install the following:

sudo apt install -y pwgen

To generate, say, a 40 character [postgresql password], you could run

pwgen -s 40 1

and grab the output, e.g. 28RFCIFtnVM9tFOmT2zx2fKiK5sf42WGK8FHr9Bn

For a 50 character [authentik secret] just copy the output of

pwgen -s 50 1

Learn more about specifying random passwords via man pwgen or pwgen --help and look at our tutorial on creating good passwords.

Also, you'll need to edit files, so it's worth setting up a text editor. If you're new to Linux, I recommend using the 'nano' text editor which is installed by default on Ubuntu Linux systems. It's fairly simple, and all of its options are visible in the text-based interface. I tend to use a far more powerful but far less beginner-friendly editor called 'vim'. There're other editors people might choose, too. To use your preferred editor for the rest of the tutorial, enter the following to set an environment variable EDIT, specifying your preferred editor, e.g.:

EDIT=$(which nano)

or, if you're like me

EDIT=$(which vim)

so that subsequent references to $EDIT will invoke your preferred editor. Note the command $(which nano) is a script which finds the full path to the named command, in this case 'nano'. Putting a command inside the $() means 'replace with the value the script returns', so it sets the value of EDIT to the path of the nano command in this case. On my current machine, the value it returns is /usr/bin/nano, which is pretty typical.

To test (at any time) whether you session still knows your $EDIT command, run

echo $EDIT

if it returns the path to your preferred editor, you're good to go. If not, just reassert (run again) the EDIT= line from above!

Note: if you log out and back in again, change users, or create a new terminal tab/session, you'll need to reassert the EDIT value.

Configuring the reverse proxy

We need to configure a 'reverse proxy' so that external requests to our new Authentik service are routed to our containers, and so that all communication between our service's users and our service are suitably encrypted via Secure Sockets Layer (SSL).

You should already have Nginx and Let's Encrypt installed on your server, so all you need to do is set up a new configuration file. The convention I use would involve doing this:

sudo $EDIT /etc/nginx/sites-available/[domain name]

where you put your domain name as the filename as indicated. You can copy and paste the following into the configuration file, replacing tokens like [domain name] with your value:

# Upstream where your authentik server is hosted.
upstream authentik {
    server 127.0.0.1:9443;
    # Improve performance by keeping some connections alive.
    keepalive 10;
}
 
# Upgrade WebSocket if requested, otherwise use keepalive
map $http_upgrade $connection_upgrade_keepalive {
    default upgrade;
    ''      '';
}
 
server {
    # HTTP server config
    listen 80;
    listen [::]:80;
    server_name [domain name];
 
    include includes/letsencrypt.conf;
 
    access_log /var/log/nginx/[domain name]_access.log;
    error_log /var/log/nginx/[domain name]_error.log;
 
    # 301 redirect to HTTPS
    location / {
            return 301 https://$host$request_uri;
    }
}
server {
    # HTTPS server config
    listen 443 ssl http2;
    listen [::]:443 ssl http2;
    server_name [domain name];
 
    # TLS certificates
    ssl_certificate /etc/ssl/certs/ssl-cert-snakeoil.pem;
    ssl_certificate_key /etc/ssl/private/ssl-cert-snakeoil.key;
    #ssl_certificate /etc/letsencrypt/live/[domain name]/fullchain.pem;
    #ssl_certificate_key /etc/letsencrypt/live/[domain name]/privkey.pem;
    # to create this, see https://raymii.org/s/tutorials/Strong_SSL_Security_On_nginx.html
    ssl_dhparam /etc/ssl/certs/dhparam.pem;
 
    access_log /var/log/nginx/[domain name]_access.log;
    error_log /var/log/nginx/[domain name]_error.log;
 
    # Proxy site
    location / {
        proxy_pass https://authentik;
        proxy_http_version 1.1;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header Host $host;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection $connection_upgrade_keepalive;
    }
}

Note that if your server has other services on it and, by coincidence, one is already using port 9443, then you can pick another port (e.g. 9444) and see if that works. If so, you'll need to make sure that you replace 9443 in the docker-compose.yml below, too.

Once you've saved the file, you can 'enable' it straight away. To do that, you need to do the following (again, replacing the tokens):

sudo ln -sf /etc/nginx/sites-available/[domain name] /etc/nginx/sites-enabled

which links the configuration file you've just created into the 'sites-enabled' directory. You can then test whether Nginx accepts the configuration you've added:

sudo nginx -t

If there're no errors here, you can make the changes live:

sudo service nginx reload

Getting your Let's Encrypt SSL certificate

You can now request an SSL certificate for your new domain (making use of the configuration set up the server tutorial), obviously replacing [domain name] with yours:

`sudo letsencrypt certonly --webroot -w /var/www/letsencrypt -d [domain name]

After you've got your certificate (which should only take a few seconds), you can go back into your proxy configuration (remember it's the same file as /etc/nginx/sites-enabled/[domain name]):

sudo $EDIT /etc/nginx/sites-available/[domain name]

and alter the certificates by changing just the following lines:

    # TLS certificates
    #ssl_certificate /etc/ssl/certs/ssl-cert-snakeoil.pem;
    #ssl_certificate_key /etc/ssl/private/ssl-cert-snakeoil.key;
    ssl_certificate /etc/letsencrypt/live/[domain name]/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/[domain name]/privkey.pem;

Then you just need to make sure there're no typos:

sudo nginx -t

and make the changes live:

sudo service nginx reload

Now external people will be able to securely access your Authentik once the service is running!

Docker Compose configuration

The next thing you need is a docker-compose.yml file. I usually put this in the a directory called /home/docker/[domain name] (replacing the token with your domain name!). You'll want to be in that directory:

cd /home/docker/[domain name]

Because the developers of Authentik periodically change the default configuration of their docker-compose.yml file, it's best to grab their current one from here:

wget https://goauthentik.io/docker-compose.yml

which should result in a file called docker-compose.yml appearing in that directory (run ls -l to confirm).

I'd make a backup of this file for future reference:

cp docker-compose.yml docker-compose.yml-default

You can then

$EDIT docker-compose.yml

and it should look similar to the one below, albeit probably with a different AUTHENTIK_TAG number - the version as of this writing is 2023.10.2 - and with different 'volumes' specified.

The only things you'll likely need to change are the designations for the 'volumes:'. See below where I've specified volumes in your /home/data/[domain name] directory rather than the local directory as the default docker-compose.yml defaults to.

version: '3.4'
 
services:
  postgresql:
    image: docker.io/library/postgres:12-alpine
    restart: unless-stopped
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -d $${POSTGRES_DB} -U $${POSTGRES_USER}"]
      start_period: 20s
      interval: 30s
      retries: 5
      timeout: 5s
    volumes:
      - /home/data/[domain name]/postgres:/var/lib/postgresql/data
    environment:
      - POSTGRES_PASSWORD=${PG_PASS:?database password required}
      - POSTGRES_USER=${PG_USER:-authentik}
      - POSTGRES_DB=${PG_DB:-authentik}
    env_file:
      - .env
  redis:
    image: docker.io/library/redis:alpine
    command: --save 60 1 --loglevel warning
    restart: unless-stopped
    healthcheck:
      test: ["CMD-SHELL", "redis-cli ping | grep PONG"]
      start_period: 20s
      interval: 30s
      retries: 5
      timeout: 3s
    volumes:
      - /home/data/[domain name]/redis:/data
  server:
    image: ${AUTHENTIK_IMAGE:-ghcr.io/goauthentik/server}:${AUTHENTIK_TAG:-2023.10.2}
    restart: unless-stopped
    command: server
    environment:
      AUTHENTIK_REDIS__HOST: redis
      AUTHENTIK_POSTGRESQL__HOST: postgresql
      AUTHENTIK_POSTGRESQL__USER: ${PG_USER:-authentik}
      AUTHENTIK_POSTGRESQL__NAME: ${PG_DB:-authentik}
      AUTHENTIK_POSTGRESQL__PASSWORD: ${PG_PASS}
      AUTHENTIK_ERROR_REPORTING__ENABLED: "true"
    volumes:
      - /home/data/[domain name]/media:/media
      - /home/data/[domain name]/custom-templates:/templates
    env_file:
      - .env
    ports:
      - "127.0.0.1:${AUTHENTIK_PORT_HTTP:-9080}:9080"
      - "127.0.0.1:${AUTHENTIK_PORT_HTTPS:-9443}:9443"
  worker:
    image: ${AUTHENTIK_IMAGE:-ghcr.io/goauthentik/server}:${AUTHENTIK_TAG:-2023.10.2}
    restart: unless-stopped
    command: worker
    environment:
      AUTHENTIK_REDIS__HOST: redis
      AUTHENTIK_POSTGRESQL__HOST: postgresql
      AUTHENTIK_POSTGRESQL__USER: ${PG_USER:-authentik}
      AUTHENTIK_POSTGRESQL__NAME: ${PG_DB:-authentik}
      AUTHENTIK_POSTGRESQL__PASSWORD: ${PG_PASS}
      AUTHENTIK_ERROR_REPORTING__ENABLED: "true"
    user: root
    volumes:
      - /home/data/[domain name]/media:/media
      - /home/data/[domain name]/certs:/certs
      - /var/run/docker.sock:/var/run/docker.sock
      - /home/data/[domain name]/custom-templates:/templates
    env_file:
      - .env

One you've got that saved, there's just one thing left! You just need to set up your specific configuration file, which is held in a 'hidden' file, .env in /home/docker/[domain name].

This is what you should put in it, replacing [tokens]. Note that you can comment out any line you want by putting a '#' at the start of the line.

Note that if you set the AUTHENTIK_TAG below and uncomment the line (remove the #) you won't (normally) need to update the docker-compose.yml file to apply updates to Authentik in future.

# AUTHENTIK_TAG=2023.10.2
PG_PASS=[postgresql password]
AUTHENTIK_SECRET_KEY=[authentik secret]
AUTHENTIK_ERROR_REPORTING__ENABLED=true
# SMTP Host Emails are sent to
AUTHENTIK_EMAIL__HOST=[smtp host]
AUTHENTIK_EMAIL__PORT=[smtp port]
# Optionally authenticate (don't add quotation marks to your password)
AUTHENTIK_EMAIL__USERNAME=[smtp username]
AUTHENTIK_EMAIL__PASSWORD=[smtp password]
# Use StartTLS
AUTHENTIK_EMAIL__USE_TLS=true
# Use SSL
AUTHENTIK_EMAIL__USE_SSL=false
AUTHENTIK_EMAIL__TIMEOUT=10
# Email address authentik will send from, should have a correct @domain
AUTHENTIK_EMAIL__FROM=[smtp from email]
AUTHENTIK_PORT_HTTP=9080
AUTHENTIK_PORT_HTTPS=9443

Once that's done and saved, you can start your Authentik service!

docker-compose up -d && docker-compose logs -f

This will trigger Docker to download the container images specified in your docker-compose.yml (based on AUTHENTIK_TAG), and start them up. That, in turn, will cause Docker Compose to create the directories specified in your volumes: stanzas in your docker-compose.yml file in /home/data/[domain name], including creating your PostgreSQL database.

Boom: Setting up your Authentik instance

Once that is up and running, you should be able to point your browser at

https://[domain name]/if/flow/initial-setup/

Here's more documentation to help you from here. Also, cooptonian's YouTube Channel is most helpful!

Upgrading your Authentik

Periodically, you'll see in the admin interface (or via an email to the site's administrators) that your Authenitik can be upgraded to a newer version.

Before doing anything to upgrade, I encourage you to make a copy of both your key files:

DATE=$(date +%d-%m-%Y) cp docker-compose.yml docker-compose.yml-$DATE cp .env .env-$DATE`

because having them handy might save your bacon if, for example, you need to revert back to your old configuration.

Sometimes the Authentik developers change the configuration of their default docker-compose.yml file. You'll want to verify that your version is the same except for any changes to volumes: you've made as follows:

Download the current reference version as docker-compose.yml-default (so it doesn't overwrite your docker-compose.yml):

wget -O docker-compose.yml-default https://goauthentik.io/version/2023.10/docker-compose.yml

Then, you'll want to run

diff docker-compose.yml-default docker-compose.yml

to see how the new file differs. If it differs in areas other than volumes: and the value for AUTHENTIK_TAG, you'll want to apply those changes to your docker-compose.yml via

$EDIT docker-compose.yml

Also, because upgrades might well result in a change to your PostgreSQL database's schema, you'll want a full backup of the existing database!

Once you have that, you can run

docker-compose pull && docker-compose up -d && docker-compose logs -f

to pull the new containers, run them, and watch the logs of the upgrade as it progresses and, hopefully, completes successfully.

Backing up your Authentik data

At the OER Foundation, we recognise that our Authentik is pretty mission-critical. To protect our users' data integrity and privacy, we do nightly encrypted incremental file backups and hourly PostgreSQL database backups. For the former, we use a Restic remote server script we've written (which requires that you have either sufficient local storage, or (preferably) another server you can access via SSH which has sufficient disk space to hold versioned backups), and the latter, we use a backup script we've developed.

Add new comment

From FOSS to LibreSoftware - it's about clarity and values

dave — Thu, 12 Oct 2023 00:18:46 +0000

From FOSS to LibreSoftware - it's about clarity and values

Blog tags

foss

libre software

dave Thu 12/10/2023 - 13:18

Any reader of this site will notice that we often talk about 'Free and Open Source Software' which we usually abbreviate as FOSS. For those who aren't intimately familiar with the history, trajectory, and nuances (warning - they are big topics) of the Free Software and Open Source Software camps - both subsets of information technology, itself a subset of digital technology - its significance is both arcane and something of a barrier to understanding.

Free Software has always had the very unfortunate ambiguity in the English-speaking world due to the word "Free" having two major meanings: free as in liberty or freedom or speech, or free as in gratis or no-cost. It seems that for most, they focus on the latter meaning and miss the point of the name. It's a constant battle to explain it to people ('free as in freedom, not free as in beer" is what you'll often hear). But, as any marketer will tell you, if you have to explain it, it's a failed band.

To be clear, even though most 'Free Software' is available at no cost, that is coincidental. The word Free in this case does not refer to cost, but rather the freedoms available to the users of the software. It is quite possible for 'Free Software' to be both 'commercial', and available at a cost, taking into account that any user has the freedom to distribute the source code at any price they want, including gratis.

The term "open source" is better marketing, in that it describes some relatively unambiguous properties of software - that the source code is available for others to view and, usually, do other things with, with various strings attached, depending on the perspective of the onlooker - but it fails to connote any of the philosophical values held by those who engage in creating, maintaining, and using it. It describes a few properties of a software project, but it doesn't offer insight into why that software is open source, which many of us think is crucially important.

In light of these short comings of both Free and Open Source Software as terms to describe what is important to us at the OER Foundation, we have started referring to 'LibreSoftware' instead. Even though, in the Angelosphere, schools are rapidly (but foolishly, in my opinion) decommissioning their Latin departments, most English-speakers have a passing familiarity with the world 'libre' - the root of liberty. It doesn't have the ambiguities of 'free', and it succinctly puts the affordances of 'Free Software' front-and-centre. It also brings us English-speakers in line with many other cultures with romance languages, where libre is well understood. So that's why we've started talking about LibreSoftware instead of FOSS.

Add new comment

Creating your own OER Foundation-style Libre Self-hosting Infrastructure with Docker Compose and Ubuntu LTS

dave — Thu, 12 Oct 2023 00:12:25 +0000

Creating your own OER Foundation-style Libre Self-hosting Infrastructure with Docker Compose and Ubuntu LTS

Blog tags

ubuntu linux

nginx

let's encrypt

postfix

docker compose

dave Thu 12/10/2023 - 13:12

Tips for this tutorial
Create a Virtual Private Server
- VPS Properties:
Key variables for you VPS
- Get your Domain lined up
Editing files
- Set up an unprivileged user for yourself
Configure the VPS
- Configuring your firewall
- Install the Nginx
Outgoing VPS Email (optional)
Installing the Docker Engine, Docker Compose, and Let's Encrypt
- Backwards compatibility for Docker Compose
- Docker use by non-root user
Setting up places for your Docker configurations and persistent Data
Configuring Nginx reverse proxy for your service
Let's Encrypt setup
- Requesting Let's Encrypt certificates

Tips for this tutorial

This tutorial is aimed at adventuresome would-be system administrators. I try not to assume any specialised knowledge on your part, and try to provide useful tips and explanation along the way to help you build a valid mental model of what you're doing. At the same, this is not a trivial process. Luckily, if you try it out, and decide not to follow through, so long as you delete your VPS, you should not be out-of-pocket by more than a few cents.

With this tutorial, I'm assuming you've got a computer with an Internet connection, that can run SSH (all modern systems should do that) and you can copy-and-paste stuff from this tutorial (in your browser) into either a terminal window (in which you're SSH'd into your VPS) or into a text editor. Note, if you find it difficult to paste into a terminal window, try using CTRL+SHIFT+V (CTRL+V is already used as a short-cut for something else in UNIX terminals since long before the Windows world started using CTRL+C and CTRL+V).

When I provide files you need to copy, look for the 'tokens' or placeholders with values you need to substitute (search-and-replace) in square brackets like this, [token name], with your own values. Any text editor worth using should let you do that!

Create a Virtual Private Server

The first step is to create a place to host instances of your own LibreSoftware services.

You can use a local piece of computing hardware of sufficient capacity in your own home or organisation (e.g. a redundant depreciated desktop should be sufficiently powerful), recognising that it needs to be reliable because anyone using your solution will not be able to access it if it's not running. But, doing that is no joke - making reliable systems is hard.

The more cost-effective self-hosting approach in our experience is to lease a low cost commodity Linux Virtual Private Server (aka a VPS) running Ubuntu Linux 22.04 (or the latest "Long Term Support" version - the next one, 24.04, will be released in April 2024). That's what we assume you're running for all our recent tutorials.

We have used quite a few Linux VPSs commodity providers. Known good options include Digital Ocean, Linode, Vultr, Hetzner, and TurnkeyLinux. There are many (hundreds of) other credible options. We recommend you find one hosted in the network epicentre (which isn't necessarily the same as the 'geographic' epicentre) of your audience. In the past year, we shifted almost all of our hosting to Hetzner as they've got the benefit of not being US-owned (They're German, and are therefore less likely to expose us to the egregiously over-reaching US Cloud and Patriot Acts). Shifting to Hetzner also reduced our infrastructure costs by about 50% compared to Digital Ocean. It also represents a 95% savings over Amazon AWS or Microsoft Azure hosting (can't imagine how anyone could justify hosting with either of them).

If you have trouble getting a VPS, you might find this video I created for provisioning a VPS, using Digital Ocean as an example, useful. In my experience, the process for provisioning VPSs on other platforms is very similar. You'll find this process much easier than using either Microsoft Azure or Amazon AWS, which we strongly recommend against using.

VPS Properties:

We recommend that you provision a VPS with the following spec - these will suffice capacity-wise for all the tutorials we provide, although depending on your load, you might want to beef them up, which you can do as the need arises - ou should be able to upgrade those specs in real time if required, except for your disk space. You can, however, provision a secondary storage space (you can start small and increase it as you need to). I will cover setting this up, as it'll make your life far far easier in the medium-long term.

4-8 GB RAM - system volatile memory for running applications
2-4 Virtual CPUs - processing capacity
80-160 GB Disk space (NVMe storage is faster than SSD which is faster than spinning disk space) - long term storage for data
running Ubuntu Linux 22.04 (the current Long Term Support version) - the operating system
extra storage - 20-40GB extra space (can be expanded on fairly short notice) - a long term storage option separate to your operating system drive - this is very very useful in the event that your app inadvertently fills your hard drive. You'll thank yourself for doing this (I an assure you from bitter experience). It's always a good idea to learn from others' past mistakes rather than repeat them!

You'll need to create an account for yourself on your chosen hosting provider (it's a good idea to use Two Factor Authentication, aka 2FA, on your hosting account so that no one can log in as you and, say, delete your server unexpectedly - you'll find instructions on how to set up 2FA on your hosting provider's site) and create an Ubuntu 22.04 (or the most recent 'Long Term Support' (LTS) version) in the 'zone' nearest to you (or your primary audience, if that's different).

If you don't already have an SSH key on your computer, I encourage you to create one and specify the public key in the process of creating your server - specifying the 'public key' of your SSH identity during the server creation process that should allow you to log in without needing a password!

You'll need to note the server's IPv4 address (it'll be a series of 4 numbers, 0-254, separated by full stops, e.g. 103.99.72.244), and you should also be aware that your server will have a newer IPv6 address, which will be a set of 8 four hex character values (each hex character can have one of 16 values: 0-9,A-F) separated by colons, e.g. 2604:A880:0002:00D0:0000:0000:20DE:9001. With one or the other of those IPs, you should be able to log into your new server via SSH. If you're on a UNIX command line (e.g. a Linux or MacOS desktop), do this in a terminal (On Windows, I understand people use a tool called Putty for SSH, in which case follow the app's instructions - or you'll find many tutorials on using it with a quick web search):

ssh [your server IPv4 or IPv6]

followed by the ENTER key (that'll be true for any line of commands I provide).

In some cases, depending on your hosting provider, you'll have a password to enter, or if you've specified your pre-existing public SSH key, you shouldn't need to enter a password at all, you should be logged in. To check what user you care, you can type

whoami

If it returns root (there's also a convention of using a '#' as the command prompt), you're the root or super-admin of the server. If not, you're a normal user (some hosting providers have a convention of giving you a default user called "ubuntu" or perhaps "debian") with a prompt that is, by convention, a '$'.

Now that you're logged in, it's worth doing an upgrade of your server's Ubuntu system! Do that as follows (this works regardless of whether your a root user or an unprivileged user with 'sudo' ability):

sudo apt update && sudo apt dist-upgrade

Usually the user, even if it's not the root user, will have the ability to use the sudo command modifier - that means "do this action as the root (aka the 'Super User', thus 'su' in 'sudo' for short) user" - if you're a non-root user, you'll likely be asked to enter your password as a security precaution the first time you run a command prefaced by sudo. Enter it, and it should run the command. Plus, the system shouldn't bother you for it again unless you leave your terminal unused for a while (usually 5 minutes) and come back to it.

At this point, I also like to install a cool software package called 'etckeeper' which records configuration changes on your VPS for future reference (it can be life-saving if trying to recover from an administrative mess-up!):

sudo apt install etckeeper git

which will also install some dependencies, including the very important (and relevant later on) 'git' version control system.

Key variables for you VPS

To set up your services, you'll need a few crucial bits of information related to your system's identity and external systems you'll need it to interact with. For example, as mentioned before, you'll need a domain name. For the rest of this tutorial, we'll use the convention of representing those variables as a name inside [], for example, the domain name you've picked, [domain name].

Here's a list of variables you'll need to know to complete the rest of this tutorial:

[ipv4] and [ipv6] - your VPS' IPv4 and IPv6 addresses (the latter can be ignored if your cloud provider doesn't support IPv6 addresses) as described above.
[domain name] - the fully qualified domain names or subdomains of a base [domain name] by which you want your services to be accessed. You must have full domain management ability on this domain. Example: nextcloud.oeru.org - that's the nextcloud subdomain of the oeru.org domain.
Authenticating SMTP details - this is required so your services can send emails to users - crucial things like email address validation and password recovery emails...
- [smtp server] - the domain name or IPv4 or IPv6 address of an SMTP server
- [smtp port] - the port number on the server that is listening for your connection. By convention it's likely to be 465 or 587, or possibly 25.
- [smtp reply-to-email] - a monitored email to which people can send email related to this WordPress site, e.g. notifications@[domain name]
- [smtp user] - the username (often an email address) used to authenticate against your SMTP server, provided by your email provider.
- [smtp password] - the accompanying password, provided by your email provider.
[your email] - an email address to which system-related emails can be sent to you, perhaps something like webmaster@[domain name].
[vps username] - the username you use on your server (by convention, these are one word, and all lower case).

Get your Domain lined up

You will want to have a domain to point at your server, so you don't have to remember the IP number. There're are thousands of domain "registrars" in the world who'll help you do that... You just need to "register" a name, and you pay yearly fee (usually between USD10-30 depending on the country and the "TLD" (Top Level Domain. There're national ones like .nz, .au, .uk, .tv, .sa, .za, etc., or international domains (mostly associated with the US) like .com, .org, .net, and a myriad of others. Countries decide on how much their domains wholesale for and registrars add a margin for the registration service).

Here in NZ, I use the services of Metaname (they're local to me in Christchurch, and I know them personally and trust their technical capabilities). If you're not sure who to use, ask your friends. Someone's bound to have recommendations (either positive or negative, in which case you'll know who to avoid).

Once you have selected and registered your domain, you can 'manage your Zone' to set up (usually through a web interface provided by the registrar) an A Record which associates your website's name to the IPv4 address of your server. So you should just be able to enter your server's IPv4 address, the domain name (or sub-domain) you want to use for the web service you want to set up.

Nowadays, if your Domain Name host offers it (some don't, meaning you might be better off with a different one), it's also important to define an IPv6 record, which is called an AAAA Record... you put in your IPv6 address instead of your IPv4 one.

You might be asked to set a "Time-to-live" (which has to do with the length of time Domain Name Servers are asked to "cache" the association that the A Record specifies) in which case you can put in 3600 seconds or an hour depending on the time units your registrar's interface requests... but in most cases that'll be set to a default of an hour automatically.

Editing files

In the rest of this tutorial, we're going to be editing quite a few files via the command line. If you're new to this, I recommend using the 'nano' text editor which is installed by default on Ubuntu Linux systems. It's fairly simple, and all of its options are visible in the text-based interface. I tend to use a far more powerful but far less beginner-friendly editor called 'vim'. There're other editors people might choose, too. To use your preferred editor for the rest of the tutorial, enter the following to set an environment variable EDIT, specifying your preferred editor, e.g.:

EDIT=$(which nano)

or, if you're like me

EDIT=$(which vim)

To test (at any time) whether you session still knows your $EDIT command, run

echo $EDIT

if it returns the path to your preferred editor, you're good to go. If not, just reassert (run again) the EDIT= line from above!

Note: if you log out and back in again, change users, or create a new terminal tab/session, you'll need to reassert the EDIT value.

Set up an unprivileged user for yourself

You should be able to test that your A and AAAA Records have been set correctly by logging into your server via SSH using your domain name rather than the IPv4 or IPv6 address you used previously. It should (after you accept the SSH warning that the server's name has a new name) work the same way your original SSH login did.

This will log you into your server as it did the first time, either as 'root' or the default unprivileged user. It's not considered good practice to access your server as root (it's too easy to completely screw it up by accident). It's a good idea to create your own separate 'non-root' user who has 'sudo' privileges and the ability to log in via SSH. If you are currently logged in as 'root', you can create a normal user for yourself via (replace [vps username] with your chosen username - in my case, I'd use U=dave):

U=[vps username]
adduser $U
adduser $U ssh
adduser $U admin
adduser $U sudo

You'll also want to a set a password for user [vps username] (we have a tutorial on creating good passwords):

passwd $U

then become that user temporarily (note, the root user can 'become' another user without needing to enter a password) and create an SSH key and, in the process, the .ssh directory (directories starting with a '.' are normally 'hidden' - you can show them in a directory listing via ls -a) for the file into which to put your public SSH key:

su $U

after which you need to re-run your EDIT command: EDIT=$(which nano)

and then run ssh-keygen -t rsa -b 2048
$EDIT ~/.ssh/authorized_keys

and in that file, copy and paste (without spaces on either end) your current computer's public ssh key (never publish your private key anywhere!), save and close the file.

and then leave the 'su' state, back to the superuser:

CTRL+D or type exit

From that point, you should be able to SSH to your server via ssh [vps username]@[domain name] without needing to enter a password.

These instructions use 'sudo' in front of commands because I assume you're using a non-root user. The instructions will still work fine even if you're logged in as 'root' (the 'sudo' will be ignored as it's unnecessary).

Configure the VPS

First things first. Let's make sure you've got the time zone set appropriately for your instance. It'll probably default to 'UTC' (Greenwich Mean Time). For our servers, I tend to pick 'Pacific/Auckland' which is our time zone. Run this

sudo dpkg-reconfigure tzdata

and pick the appropriate timezone. You can just leave it running UTC, but you might find it tricky down the track if, for example, you're looking at logs and having to constantly convert the times into your timezone.

Configuring your firewall

In the name of safety from the get-go, let's configure our firewall. We work on the basis of explicitly allowing in only what we want to let in (i.e. a 'default deny' policy).

First we'll enable the use of SSH through the firewall (not doing this could lock us out of your machine!)

sudo ufw allow ssh

while we're here, we'll also enable data transfer from the internal (to the VPS) Docker virtual network and the IP range it uses for Docker containers:

sudo ufw allow in on docker0
sudo ufw allow from 172.0.0.0/8 to any

Then we'll enable forwarding from internal network interfaces as required for Docker containers to be able to talk to the outside world:

sudo $EDIT /etc/default/ufw

and copy the line DEFAULT_FORWARD_POLICY="DROP" tweak it to look like this (commenting out the default, but leaving it there for future reference!):

#DEFAULT_FORWARD_POLICY="DROP"
DEFAULT_FORWARD_POLICY="ACCEPT"

and then save and exit the file (CTRL-X and then 'Y' if your editor is nano).

You also have to edit /etc/ufw/sysctl.conf

sudo $EDIT /etc/ufw/sysctl.conf

and remove the "#" at the start of the following lines, so they look like this:

# Uncomment this to allow this host to route packets between interfaces
net/ipv4/ip_forward=1
net/ipv6/conf/default/forwarding=1
net/ipv6/conf/all/forwarding=1

Then we need to restart the network stack to apply that configuration change

sudo systemctl restart systemd-networkd

(on older Ubuntu systems this would have been done via sudo service networking restart...)

Next we have to enable the (default on Ubuntu) UFW firewall to start at boot time to keep your machine relatively safe.

sudo $EDIT /etc/ufw/ufw.conf

And set the ENABLED variable near the top:

ENABLED=yes

Now you can formally start UFW now:

sudo ufw enable

Install the Nginx

Next we need to install the Nginx web server and reverse-proxy, as well as the Let's Encrypt SSL certificate generator, both of which are crucial for any secure web services you might want to host. Nginx is a more efficient and flexible alternative to the older Apache web server you might've seen elsewhere (Nginx recently surpassed Apache as the most widely used web server on the Internet).

sudo apt install nginx-full letsencrypt ssl-cert

You'll get a couple pop-up windows in your terminal, just hit ENTER to accept the defaults. Having installed it, we need to create firewall rules to allow external services to see it:

sudo ufw allow 'Nginx Full'

You can check if the firewall rules you requested have been enabled:

sudo ufw status

Outgoing VPS Email (optional)

Although it's not absolutely necessary (you can do this section later if you're in a big hurry), it's very useful for your server to be able to send out emails, like status emails to administrators (perhaps you) about things requiring their attention, e.g. the status of backups, pending security updates, expiring SSL certificates, etc.

To do this, we'll set up the industrial strength Postfix SMTP server, which is pretty quick and easy. First we install Postfix and a command line mail client for testing purposes.

sudo apt install postfix bsd-mailx

During the install, you'll be asked to select a bunch of configuration parameters. Select the defaults except:

Select "Internet Site with Smarthost",
fill in the domain name for your server [domain name],
the [smtp server] name and [smtp port] (in the form [smtp server]:[smtp port], e.g. smtp.oeru.org:587 ) of your "smarthost" who'll be doing the authenticating SMTP for you, and
the email address to which you want to receive system-related messages, [your email].

After that's done, we set a default address for the server to mail to, to [your email] selected above. First

sudo $EDIT /etc/aliases

We need to make sure the "root" user points to a real email address. Add a line at the bottom which says (replacing [your email] with your email :) )

root: [your email]

After which you'll need to convert the aliases file into a form that postfix can process, simply by running this:

sudo newaliases

Then we have to define the authentication credentials required to convince your mail server that you're you!

sudo $EDIT /etc/postfix/relay_password

and enter a single line in this format:

[smtp server] [smtp user]:[smtp password]

as an example, this is more or less what I've got for my system. Note that the [smtp user] in my case is an email address (this is common with many smtp system - the user is the same as the email address):

smtp.oeru.org smtp-work@fossdle.org:YourObscurePassw0rd

then save the file and, like the aliases file, run the conversion process (which uses a slightly different mechanism):

sudo postmap /etc/postfix/relay_password

Finally, we'll edit the main configuration file for Postfix to tell it about all this stuff:

sudo $EDIT /etc/postfix/main.cf

If your SMTP server uses port 25 (the default for unencrypted SMTP) you don't have to change anything, although most people nowadays prefer to use StartTLS or otherwise encrypted transport to at least ensure that your SMTP authentication details (at least) are transferred encrypted. That means using port 587 or 465. If you're using either of those ports, find the "relayhost = [your server name]" line... and add your port number after a colon, like this

relayhost = [smtp server]:[smtp port]

or, for example:

relayhost = smtp.oerfoundation.org:465

Then we have to update the configuration for Postfix to ensure that it knows about the details we've just defined (this command will automatically back up the original default configuration so you can start from scratch with the template below):

sudo mv /etc/postfix/main.cf /etc/postfix/main.cf.orig && sudo $EDIT /etc/postfix/main.cf

You can just copy-and-paste the following into it, substituting your specific values for the [tokens]. Note: the IPv6 designations in the line mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128 are not tokens - you can leave those unchanged.

# See /usr/share/postfix/main.cf.dist for a commented, more complete version
 
# Debian specific:  Specifying a file name will cause the first
# line of that file to be used as the name.  The Debian default
# is /etc/mailname.
#myorigin = /etc/mailname
 
smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu)
biff = no
 
# appending .domain is the MUA's job.
append_dot_mydomain = no
 
# Uncomment the next line to generate "delayed mail" warnings
#delay_warning_time = 4h
readme_directory = no
 
# See http://www.postfix.org/COMPATIBILITY_README.html -- default to 3.6 on
# fresh installs.
compatibility_level = 3.6
 
# TLS parameters
smtpd_tls_cert_file=/etc/ssl/certs/ssl-cert-snakeoil.pem
smtpd_tls_key_file=/etc/ssl/private/ssl-cert-snakeoil.key
smtpd_tls_security_level=may
 
smtp_tls_CApath=/etc/ssl/certs
#smtp_tls_security_level=may
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache
 
smtpd_relay_restrictions = permit_mynetworks permit_sasl_authenticated defer_unauth_destination
myhostname = [domain name]
alias_maps = hash:/etc/aliases
alias_database = hash:/etc/aliases
myorigin = /etc/mailname
mydestination = $myhostname, localhost
relayhost = [smtp server]:[smtp port]
mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128
mailbox_size_limit = 0
recipient_delimiter = +
inet_interfaces = all
inet_protocols = all
 
# added to configure accessing the relay host via authenticating SMTP
smtp_sasl_auth_enable = yes
smtp_sasl_password_maps = hash:/etc/postfix/relay_password
smtp_sasl_security_options = noanonymous
smtp_tls_security_level = encrypt
 
# if you're using Ubuntu prior to 20.04, uncomment (remove the #) the
# earlier line smtp_tls_security_level = may to save errors in 'postfix check'
# and comment this line (by adding a # at the start)
smtp_tls_wrappermode = yes

Once you've created that main.cf file, you can double check that your config is valid:

sudo postfix check

and if it's all ok, you can get Postfix to re-read its configuration:

sudo postfix reload

You can then try sending an email so see if it works!

By default, a command line application called "mail" is installed as part of the bsd-mailx package we installed alongside postfix. You can use it to send test email from the command line on your host to verify you've got things working correctly! The stuff in <> are the keys to hit at the end of the line...

mail you@email.domain<ENTER>

Subject: Testing from your.relay.server.domain<ENTER>
Testing postfix remote host<ENTER>
<CTRL-D>
Cc:<ENTER>

Typing (hold down the Control or Ctrl key on your keyboard and press the "d" key) will finish your message, showing you a "CC:" field, in which you can type in other email addresses if you want to test sending to multiple addresses. When you then hit , it will attempt to send this email. It might take a few minutes to work its way through to the receiving email system (having to run the gauntlet of spam and virus filters on the way).

You can also always check the postfix system logs to see what postfix thinks about it using the command:

sudo less +G /var/log/mail.log

if your system doesn't have a /var/log/mail.log, never fear! Try this instead:

sudo less +G /var/log/syslog

In either case, hit to have the log update in real time.

Installing the Docker Engine, Docker Compose, and Let's Encrypt

First let's install the Docker Engine, which (these days) comes with Docker Compose and the Let's Encrypt scripts that let you procure no-cost Secure Sockets Layer certificates to secure access to your server. You can follow the official Docker Engine install instructions, but I've summarised them here (If the following doesn't work for you, go back to the official instructions, because something might've changed since I wrote this).

First we want to make sure no old Docker engines are installed on this server (this probably won't do anything, but no harm in running it):

for pkg in docker.io docker-doc docker-compose docker-compose-v2 podman-docker containerd runc; do sudo apt-get remove $pkg; done

Second, we want to set up Docker's 'APT' repository, so you can keep Docker up-to-date with their latest versions (usually more up-to-date than those shipped with Ubuntu):

First we Add Docker's official GPG key (copy and paste all of this at your command line):

sudo apt-get update
sudo apt-get install ca-certificates curl gnupg
sudo install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
sudo chmod a+r /etc/apt/keyrings/docker.gpg

Then we have to add the repository to Apt our system's sources and install the Docker Engine:

echo \
  "deb [arch="$(dpkg --print-architecture)" signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \
  "$(. /etc/os-release && echo "$VERSION_CODENAME")" stable" | \
  sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt-get update
sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

With this approach, any updates made by the Docker community will be installed as part of your regular server upgrades.

Having installed the most recent release of the Docker engine, you should find that you've now got Docker Compose built in. Test that by running

docker compose version

which should return something like

Docker Compose version v2.18.1

If that's the case, superb, everything's great.

Backwards compatibility for Docker Compose

Since version 2 of the Docker Engine, the 'Docker Compose' capability has been incorporated into the base system. Prior to version 2, using Docker Compose required installing a separate app, and it was run by typing docker-compose rather than docker compose... so something I do now, to accommodate my muscle memory of typing docker-compose is to create a tiny script that lets me keep using that command, but have it call, instead, the new docker compose functionality. I do this via

sudo $EDIT /usr/local/bin/docker-compose

into which I put the following:

#!/bin/bash
D=`which docker`
$D compose "$@"

After saving that, we have to make the script 'executable' via

sudo chmod a+x /usr/local/bin/docker-compose

So you should now be able to run

docker-compose version

and get the same result that you did above for docker compose version...

Docker use by non-root user

If the above docker commands didn't work for you... and if you want to run Docker commands without being the root user or using sudo, as we usually do, you need to do a few more steps...

create a 'docker' group on your system (this might already exist, but doing this again won't hurt): sudo groupadd docker
add your user to it: sudo usermod -aG docker $USER
refresh your shell so that it recognises your user's membership in the docker group: newgrp docker

You should now be to run a test as a non-root user docker run hello-world

Setting up places for your Docker configurations and persistent Data

The next step is to set up the file structure for holding your Docker configurations and the data your Docker containers will access. This is my convention, so you're welcome to do things different, but this is a 'known good' approach.

Now we create the set of directories I use for holding Docker Compose configurations (/home/docker) and the persistent data the Docker containers create (/home/data)

D=[domain]
sudo mkdir -p /home/data/$D
sudo mkdir -p /home/docker/$D

It's helpful to make sure that your non-root user can also read and write files in these directories:

U=[vps username]
sudo chown -R $U /home/docker
sudo chown -R $U /home/data

Configuring Nginx reverse proxy for your service

Above, we installed Nginx as well as the Let's Encrypt scripts. Now we'll configure them as it's useful to have them working before you set up your services.

In order for you, outside of your server, to see your specific LibreSoftware services, you will need to set up a secure external 'reverse proxy' on your host VPS which will accept requests for each of those services from the Internet and pass those requests securely to the two sets of Docker containers providing the services. These will answer to https://[domain name], noting that a given VPS can answer on behalf of more than one domain name or subdomain name, where each references a different instance of a service or altogether different services. For example, a given VPS could host multiple services, like, say, a password manager, a WordPress blog, a Mautic email automation system, an Authentik Single-Sign On service, and a Discourse forum, where each has a separate domain name (or sub domain name) and each has an Nginx reverse proxy configuration that directs requests to the appropriate domain to the appropriate set of Docker containers.

We use Let's Encrypt to provide the SSL certificates (each is a file with a specially generated, very long string) which we use to limit access to our services to encrypted (secure) connections (protecting both our users and ourselves from external enemies), usually one for each service domain name and corresponding Nginx configuration file. Some Nginx configurations will have multiple domains names for which a given service accepts connections, but it consolidates them to the 'canonical' (official) domain name. For example, the OER Foundation's WordPress website will respond to any of the following:

but all requests to any of those 4 options (note the difference between http:// and https:// where 's' refers to 'Secure' or encrypted) will all be redirected - transparently by the Nginx configuration, which will be reflected in your browser's address bar - to the canonical web address (or URL), https://oerfoundation.org, that we've chosen because it's short and secure.

Nginx will not run unless the SSL certificates you reference in your configurations are valid. Given that we need to request them with a working Nginx prior to them being created puts us in an awkward position. We use a trick to get around it: we temporarily reference the default 'self-signed' SSL certificates (sometimes called 'Snakeoil certs' because that's the placeholder name they're given) that every new Linux system generates when it's installed, that are valid certificates (and thus acceptable to Nginx) but *they won't work with our domains, as they're generic and not 'signed' by an external party, like Let's Encrypt, meaning that your browser won't like them. But that's ok, as you browser will never need to see them, and Let's Encrypt's systems won't look at them either. We'll swap the Snakeoil certs out as soon as we've successfully created the Let's Encrypt ones, and your browser will be happy, and all will be well with the world.

This should happen automatically when you install the Let's Encrypt packages, but just to be sure, run this (it won't harm anything if you run it more than once):

sudo make-ssl-cert generate-default-snakeoil

which creates your default 'Snakeoil' certificates. They are technically valid certificates, but they aren't signed by any

Let's Encrypt setup

Step one of using Let's Encrypt is to make sure the Let's Encrypt scripts are installed (note: sometimes they're referred to as 'certbot' - it's the same code):

sudo apt install letsencrypt

Let's Encrypt and Nginx need to work together. Nginx stores all of its configuration in the directory /etc/nginx. The first thing we'll do is create a place for Let's Encrypt Nginx-specific configuration details:

sudo mkdir /etc/nginx/includes

Then we create that configuration file itself:

sudo $EDIT /etc/nginx/includes/letsencrypt.conf

into which we copy-and-paste the following (no [tokens] to replace in this one!)

# Rule for legitimate ACME Challenge requests
location ^~ /.well-known/acme-challenge/ {
    default_type "text/plain";
    # this can be any directory, but this name keeps it clear
    root /var/www/letsencrypt;
}
 
# Hide /acme-challenge subdirectory and return 404 on all requests.
# It is somewhat more secure than letting Nginx return 403.
# Ending slash is important!
location = /.well-known/acme-challenge/ {
    return 404;
}

As described in the file we've just created, Let's Encrypt will look for a secret code we create to verify that we own the domain we're requesting an SSL certificate for, so we have to make sure it exists:

sudo mkdir /var/www/letsencrypt

You will need to install an Nginx reverse proxy configuration file for each web-based service you run on your VPS, but those will usually be service-specific in their make up, so I won't discuss them here.

But, for the record (to make your lives a bit easier), this is what you'll do after you've got a suitable Nginx reverse proxy configuration in place and ready to run to acquire a Let's Encrypt SSL certificate.

Requesting Let's Encrypt certificates

To request Let's Encrypt SSL certificates for your service, run the following, replacing the [domain name] reference with the address you've selected for your service. Note that 'certbot' is the script provided by the Let's Encrypt package - historically, it could also be called via 'letsencrypt', although apparently the latter is now deprecated:

sudo certbot --webroot -w /var/www/letsencrypt -d [domain name]

Note - if you want to address your instance from multiple domains, use one (or more) -d [another domain] - just make sure that

all those domains already point to your VPS, and
those domains are included in the Nginx proxy configuration above.

otherwise the Let's Encrypt certbot request will fail!

Here's what you're likely to see as output from the first run of the letsencrypt script - note that it will ask you for an email address (so it can send you warnings if your certificate is going to expire, e.g. due to a problem with renewal (like if you make a configuration change that breaks the renewal process)).

Saving debug log to /var/log/letsencrypt/letsencrypt.log
Enter email address (used for urgent renewal and security notices)
 (Enter 'c' to cancel): webmaster@fossdle.org
 
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Please read the Terms of Service at
https://letsencrypt.org/documents/LE-SA-v1.3-September-21-2022.pdf. You must
agree in order to register with the ACME server. Do you agree?
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
(Y)es/(N)o: y
 
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Would you be willing, once your first certificate is successfully issued, to
share your email address with the Electronic Frontier Foundation, a founding
partner of the Let's Encrypt project and the non-profit organization that
develops Certbot? We'd like to send you email about our work encrypting the web,
EFF news, campaigns, and ways to support digital freedom.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
(Y)es/(N)o: y
Account registered.
Requesting a certificate for [domain name]
 
Successfully received certificate.
Certificate is saved at: /etc/letsencrypt/live/[domain name]/fullchain.pem
Key is saved at:         /etc/letsencrypt/live/[domain name]/privkey.pem
This certificate expires on (some future date).
These files will be updated when the certificate renews.
Certbot has set up a scheduled task to automatically renew this certificate in the background.
 
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
If you like Certbot, please consider supporting our work by:
 * Donating to ISRG / Let's Encrypt:   https://letsencrypt.org/donate
 * Donating to EFF:                    https://eff.org/donate-le
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Ideally, you'll see a message like the above. If not, and there's an error, the error messages they provide are usually very useful and accurate. Fix the problem and try again. Note, your SSL certificate will have the name of your [domain name], even if it also provide support for [second domain name] (or third, fourth, etc.).

Once you have a Let's Encrypt certificate, you can update our Nginx configuration:

sudo $EDIT /etc/nginx/sites-available/[domain name]

and swap all occurrences of

    ssl_certificate /etc/ssl/certs/ssl-cert-snakeoil.pem;
    ssl_certificate_key /etc/ssl/private/ssl-cert-snakeoil.key;
#    ssl_certificate /etc/letsencrypt/live/[domain name]/fullchain.pem;
#    ssl_certificate_key /etc/letsencrypt/live/[domain name]/privkey.pem;

#    ssl_certificate /etc/ssl/certs/ssl-cert-snakeoil.pem;
#    ssl_certificate_key /etc/ssl/private/ssl-cert-snakeoil.key;
    ssl_certificate /etc/letsencrypt/live/[domain name]/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/[domain name]/privkey.pem;

which enables your new domain-specific SSL certificate. Check that Nginx is happy with your change:

sudo nginx -t

and if so,

sudo service nginx reload

You domain should now be enabled for https:// access. Note that going to http://[domain name] should automatically redirect you to https://[domain name] because you care about your user's security!

And that's it - your server is now ready to receive specific configurations for a myriad of Libre web services! Have a look around this site and check out some of the other tutorials - for many of them, you'll now be able to skip to near the end, as you've done most of the groundwork already!

Add new comment

Install NextCloud Hub and OnlyOffice on Ubuntu 22.04 with Docker Compose

dave — Sun, 28 May 2023 23:50:43 +0000

Install NextCloud Hub and OnlyOffice on Ubuntu 22.04 with Docker Compose

Blog tags

ubuntu linux

mariadb

docker

docker-compose

nextcloud

onlyoffice

nginx

let's encrypt

redis

productivity

dave Mon 29/05/2023 - 11:50

This is another update of my previous posts (installing NextCloud with Collabora Office Online on Ubuntu 16.04 and then NextCloud with OnlyOffice on Ubuntu 18.04). I'm updating it thanks to my colleague in edtech, Stephen Downes' heroic videos showing how he went through this process using my 18.04 instructions on 22.04, running into a few minor issues... this update seeks to remedy the problems he encountered with the older tutorial. Remember, all these Free and Open Source Software projects are progressing and improving relentlessly, which means the they way they behaved a few years ago is not likely to be exactly the way they behave now. In particular, NextCloud is leaping from strength to strength, benefiting from the well-founded concern held by many in the EU about data sovereignty and the market domination (and exploitation) of US-based multinationals like Amazon, Google, Microsoft, Dropbox, and others.

There're a few productivity packages that can be used in conjunction with NextCloud to provide comparable functionality to, for example, GoogleDocs + GoogleDrive or Microsoft Office 365 + Microsoft OneDrive, including Collabora Office (which we've used in the past). But the best companion productivity suite for NextCloud, in my opinion, is OnlyOffice. The application itself (for the tech focused reader, they've built an entirely new application ecosystem primarily using modern Javascript frameworks) is impressive in both capabilities and polish. The only real caveat I've come across is that it uses, by default, the 'fauxpen' standard formats developed by Microsoft rather than the true open standard formats of OpenDocumentFormat. But in a world where, sadly, most people don't even know what a file format is, any software that doesn't read and write the incumbent monopolist's format with great fidelity is dead in the water. On that count, OnlyOffice is impressive. NextCloud + OnlyOffice - even better together (without a single US multinational tech giant involved. NextCloud development is led from Germany, OnlyOffice's development is led by a team in Latvia)!

The beauty of the open source software model is that we can connect complementary applications, like NextCloud and OnlyOffice - developed by completely separate communities - to create a tightly integrated, highly functional, diverse computing platform. This combination, along with a bunch of other NextCloud "apps", is the equal of something like Google Apps (which includes Google Docs and Google Drive), but is under your control, not Google's. To me, that's a crucial difference.

With the release of NextCloud 26.0 (current as of this writing), NextCloud has the option of installing NextCloud Office (which is just a bundled OnlyOffice installation) alongside it, creating something called "NextCloud Hub". It's pretty impressive, but I've found that managing the OnlyOffice install is problematic and somewhat inflexible, so I've opted to stay with having an independent (rather than bundled) OnlyOffice instance, but it's on the same server but managed as a different service. That's what we'll be setting up here!

Tips for this tutorial
Create a Virtual Private Server
- VPS Properties:
Key variables for you NextCloud and OnlyOffice instances
- Get your Domain lined up
Editing files
- Set up an unprivileged user for yourself
Configure the VPS
- Configuring your firewall
- Install the Nginx
Outgoing VPS Email (optional)
Installing Docker Compose and Let's Encrypt
Installing MariaDB
Configuring Nginx reverse proxy for NextCloud and OnlyOffice
Let's Encrypt setup
- NextCloud Proxy Configuration
- OnlyOffice Proxy Configuration
- Requesting Let's Encrypt certificates
Prepare your Docker Compose host
NextCloud Install
- Install the NextCloud Docker recipe
- The NextCloud Nginx configuration
- The OnlyOffice Docker configuration
Firing up your NextCloud!
- The NextCloud source code (if necessary)
Configuring database access
Configuring the Admin user
Configuring Outgoing Email
Setting up OnlyOffice
Configuring OnlyOffice Integration with NextCloud
Keeping the whole thing up-to-date
Backing up NextCloud
Backup OnlyOffice

Tips for this tutorial

This tutorial is aimed at adventuresome would-be system administrators. I endeavour not to assume any specialised knowledge on your part, and try to provide useful tips and exposition along the way to help you build a valid mental model of what you're doing. At the same, this is not a trivial process. Luckily, if you try it out, and decide not to follow through, so long as you delete your VPS, you should not be out-of-pocket by more than a few cents.

If this is your first attempt at 'self-hosting', and you do follow through, this could be the start of a new era in your technical status - you could realised that self-hosting 'agency' you always wanted. People with that skill set are in hot demand among most organisations, especially in the NGO/charitable spaces. Plus, I'll be very impressed by your moxie!

When I provide files you need to copy, look for the placeholders with values you need to substitute (search-and-replace) in square brackets - [] - with your own values. I assume you'll be able to do that in a text editor on your desktop.

Create a Virtual Private Server

The first step is to create a place to host the NextCloud and OnlyOffice instances. You can run them on a local piece of hardware of sufficient capacity, but make sure you've got a fast and symmetrical (as fast to upload as to download!) connection. If (as with most residential Internet services) your upload is much slower than your download (often 1:10 ratio) your server is going to be very slow for external people, especially if streaming video. Also, don't undertake this unless you have a flat-rate data connection.

The more cost-effective approach in our experience, is to secure a low cost commodity Linux Virtual Private Server running Ubuntu Linux 22.04 (the latest "Long Term Support" version). That's what we'll assume you're running for this tutorial. We have used quite a few Linux VPSs commodity providers. Known good options are Digital Ocean (who recently raised their prices significantly), Linode, Vultr, Hetzner, and TurnkeyLinux. There are many (hundreds) of other credible options. We recommend you find one hosted in the network epicentre (which isn't necessarily the same as the 'geographic' epicentre) of your audience. For the record, we've just shifted our hosting to Hetzner as they've got the benefit of not being US-owned (They're German, and therefore don't expose us to the egregiously over-reaching US Cloud and Patriot Acts) and their pricing is pretty unbeatable.

If you have trouble getting a VPS, you might find this video I created for provisioning a VPS, using Digital Ocean as an example. In my experience, the process for provisioning VPSs on other platforms is very similar. You'll find this process much easier than using either Microsoft Azure or Amazon AWS, which we do not recommend. Their systems are unnecessarily complex, proprietary (they will lock you in), and 10-20 times more expensive than commodity hosting options already listed.

VPS Properties:

We recommend that, for a NextCloud instance of modest size (say up to 50 users) you provision a VPS with the following spec. You should be able to upgrade those specs in realtime if required, except for your disk space. You can, however, provision a secondary storage space (you can start small and increase it as you need to). I will cover setting this up, as it'll make your life far far easier in the medium-long term.

4-8 GB RAM
2-4 Virtual CPUs
80-160 GB Disk space (NVME disk is faster than SSD which is faster than spinning disk space)
running Ubuntu Linux 22.04 (the current Long Term Support version)
extra storage - 20-40GB extra space (can be expanded on fairly short notice)

You'll need to create an account for yourself on your chosen hosting provider (it's a good idea to use Two Factor Authentication, aka 2FA, on your hosting account so that no one can log in as you and, say, delete your server unexpectedly - you'll find instructions on how to set up 2FA on your hosting provider's site) and create an Ubuntu @2.04 (or the most recent 'Long Term Support' (LTS) version) - 24.04 is likely to come out in April 2024) in the 'zone' nearest to you (or your primary audience, if that's different).

You'll need to note the server's IPv4 address (it'll be a series of 4 numbers, 0-254, separated by full stops, e.g. 103.99.72.244), and you should also be aware that your server will have a newer IPv6 address, which will be a set of 8 four hex character values (each hex character can have one of 16 values: 0-9,A-F) separated by colons, e.g. 2604:A880:0002:00D0:0000:0000:20DE:9001. With one or the other of those IPs, you should be able to log into your new server via SSH. If you're on a UNIX command line (e.g. a Linux or MacOS desktop), do this in a terminal. On Windows, I understand people use a tool called Putty for SSH, in which case follow the app's instructions.

ssh [your server IPv4 or IPv6]

followed by the ENTER key (that'll be true for any line of commands I provide).

whoami

sudo apt update && sudo apt dist-upgrade

sudo apt install etckeeper

which will also install some dependencies, including the very important (and relevant later on) 'git' version control system.

Key variables for you NextCloud and OnlyOffice instances

To set up your services, you'll need a few crucial bits of information related to your system's identity and external systems you'll need it to interact with. For example, as mentioned before, you'll need a domain name. For the rest of this tutorial, we'll use the convention of representing those variables as a name inside [], or, for the domain name you've picked, [domain name].

Here's a list of variables you'll need to know to complete the rest of this tutorial:

[ipv4] and [ipv6] - your VPS' IPv4 and IPv6 addresses (the latter can be ignored if your cloud provider doesn't support IPv6 addresses) as described above.
[nextcloud domain] and [onlyoffice domain] - the fully qualified domain names or subdomains of a base [domain name] by which you want your services to be accessed. You must have full domain management ability on this domain. Example: nextcloud.oeru.org - that's the nextcloud subdomain of the oeru.org domain. *Authenticating SMTP details - this is required so your services can send emails to users - crucial things like email address validation and password recovery emails...
- [smtp server] - the domain name or IPv4 or IPv6 address of an SMTP server
- [smtp port] - the port number on the server that is listening for your connection. By convention it's likely to be 465 or 587, or possibly 25.
- [smtp reply-to-email] - a monitored email to which people can send email related to this WordPress site, e.g. notifications@[domain name]
- [smtp user] - the username (often an email address) used to authenticate against your SMTP server, provided by your email provider.
- [smtp password] - the accompanying password, provided by your email provider.
[your email] - an email address to which system-related emails can be sent to you, perhaps something like webmaster@[domain name].
[vps username] - the username you use on your server (by convention, these are one word, and all lower case).
[redis password] - this is a random secret that secure access to your webserver's cached data - I use a randomly generated alphanumeric password.
[onlyoffice secret] - this comes from your actual install, and you can get it when the time comes.
The MariaDB credentials for your NextCloud system (which stores a lot of stuff in MariaDB or MySQL by default)
- [db name] - the name of your MariaDB database for NextCloud - usually 'nextcloud'.
- [db user] - the user who can manage your database.
- [db password] - the user's password.

Get your Domain lined up

Editing files

EDIT=$(which nano)

or, if you're like me

EDIT=$(which vim)

To test (at any time) whether you session still knows your $EDIT command, run

echo $EDIT

if it returns the path to your preferred editor, you're good to go. If not, just reassert the EDIT= line from above!

Note: if you log out and back in again, change users, or create a new terminal tab/session, you'll need to reassert the EDIT value.

Set up an unprivileged user for yourself

U=[vps username]
adduser $U
adduser $U ssh
adduser $U admin
adduser $U sudo

You'll also want to a set a password for user [vps username] (we have a tutorial on creating good passwords):

passwd $U

su $U

after which you need to re-run your EDIT command: EDIT=$(which nano)

and then run ssh-keygen -t rsa -b 2048
$EDIT ~/.ssh/authorized_keys

and in that file, copy and paste (without spaces on either end) your current computer's public ssh key (never publish your private key anywhere!), save and close the file.

and then leave the 'su' state, back to the superuser:

CTRL+D or type exit

From that point, you should be able to SSH to your server via ssh [vps username]@[domain name] without needing to enter a password.

Configure the VPS

sudo dpkg-reconfigure tzdata

Configuring your firewall

In the name of safety from the get-go, let's configure our firewall. We work on the basis of explicitly allowing in only what we want to let in (i.e. a 'default deny' policy).

First we'll enable the use of SSH through the firewall (not doing this could lock us out of your machine!)

sudo ufw allow ssh

while we're here, we'll also enable data transfer from the internal (to the VPS) Docker virtual network and the IP range it uses for Docker containers:

sudo ufw allow in on docker0
sudo ufw allow from 172.0.0.0/8 to any

Then we'll enable forwarding from internal network interfaces as required for Docker containers to be able to talk to the outside world:

sudo $EDIT /etc/default/ufw

and copy the line DEFAULT_FORWARD_POLICY="DROP" tweak it to look like this (commenting out the default, but leaving it there for future reference!):

#DEFAULT_FORWARD_POLICY="DROP"
DEFAULT_FORWARD_POLICY="ACCEPT"

and then save and exit the file (CTRL-X and then 'Y' if your editor is nano).

You also have to edit /etc/ufw/sysctl.conf and remove the "#" at the start of the following lines, so they look like this:

sudo $EDIT /etc/ufw/sysctl.conf

# Uncomment this to allow this host to route packets between interfaces
net/ipv4/ip_forward=1
net/ipv6/conf/default/forwarding=1
net/ipv6/conf/all/forwarding=1

Then we need to restart the network stack to apply that configuration change

sudo systemctl restart systemd-networkd

(on older Ubuntu systems this would have been done via sudo service networking restart...)

Next we have to enable the UFW firewall to start at boot time.

sudo $EDIT /etc/ufw/ufw.conf

And set the ENABLED variable near the top:

ENABLED=yes

Now you can formally start UFW now:

sudo ufw enable

Install the Nginx

sudo apt install nginx-full letsencrypt ssl-cert

You'll get a couple pop-up windows in your terminal, just hit ENTER to accept the defaults. Having installed it, we need to create firewall rules to allow external services to see it:

sudo ufw allow 'Nginx Full'

You can check if the firewall rules you requested have been enabled:

sudo ufw status

Outgoing VPS Email (optional)

sudo apt install postfix bsd-mailx

During the install, you'll be asked to select a bunch of configuration parameters. Select the defaults except:

Select "Internet Site with Smarthost",
fill in the domain name for your server [domain name],
the [smtp server] name and [smtp port] (in the form [smtp server]:[smtp port], e.g. smtp.oeru.org:587 ) of your "smarthost" who'll be doing the authenticating SMTP for you, and
the email address to which you want to receive system-related messages, [your email].

After that's done, we set a default address for the server to mail to, to [your email] selected above. First

sudo $EDIT /etc/aliases

We need to make sure the "root" user points to a real email address. Add a line at the bottom which says (replacing [your email] with your email :) )

root: [your email]

After which you'll need to convert the aliases file into a form that postfix can process, simply by running this:

sudo newaliases

Then we have to define the authentication credentials required to convince your mail server that you're you!

sudo $EDIT /etc/postfix/relay_password

and enter a single line in this format:

[smtp server] [smtp user]:[smtp password]

smtp.oerfoundation.org smtp-work@fossdle.org:SomeObscurePassw0rd

then save the file and, like the aliases file, run the conversion process (which uses a slightly different mechanism):

sudo postmap /etc/postfix/relay_password

Finally, we'll edit the main configuration file for Postfix to tell it about all this stuff:

sudo $EDIT /etc/postfix/main.cf

relayhost = [smtp server]:[smtp port]

or, for example:

relayhost = smtp.oerfoundation.org:465

sudo mv /etc/postfix/main.cf /etc/postfix/main.cf.orig && sudo $EDIT /etc/postfix/main.cf

You can just copy-and-paste the following into it, substituting your specific values for the [tokens].

# See /usr/share/postfix/main.cf.dist for a commented, more complete version
 
# Debian specific:  Specifying a file name will cause the first
# line of that file to be used as the name.  The Debian default
# is /etc/mailname.
#myorigin = /etc/mailname
 
smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu)
biff = no
 
# appending .domain is the MUA's job.
append_dot_mydomain = no
 
# Uncomment the next line to generate "delayed mail" warnings
#delay_warning_time = 4h
readme_directory = no
 
# See http://www.postfix.org/COMPATIBILITY_README.html -- default to 3.6 on
# fresh installs.
compatibility_level = 3.6
 
# TLS parameters
smtpd_tls_cert_file=/etc/ssl/certs/ssl-cert-snakeoil.pem
smtpd_tls_key_file=/etc/ssl/private/ssl-cert-snakeoil.key
smtpd_tls_security_level=may
 
smtp_tls_CApath=/etc/ssl/certs
#smtp_tls_security_level=may
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache
 
smtpd_relay_restrictions = permit_mynetworks permit_sasl_authenticated defer_unauth_destination
myhostname = [domain name]
alias_maps = hash:/etc/aliases
alias_database = hash:/etc/aliases
myorigin = /etc/mailname
mydestination = $myhostname, localhost
relayhost = [smtp server]:[smtp port]
mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128
mailbox_size_limit = 0
recipient_delimiter = +
inet_interfaces = all
inet_protocols = all
 
# added to configure accessing the relay host via authenticating SMTP
smtp_sasl_auth_enable = yes
smtp_sasl_password_maps = hash:/etc/postfix/relay_password
smtp_sasl_security_options = noanonymous
smtp_tls_security_level = encrypt
 
# if you're using Ubuntu prior to 20.04, uncomment (remove the #) the
# earlier line smtp_tls_security_level = may to save errors in 'postfix check'
# and comment this line (by adding a # at the start)
smtp_tls_wrappermode = yes

Once you've created that main.cf file, you can double check that your config is valid:

sudo postfix check

and if it's all ok, you can get Postfix to re-read its configuration:

sudo postfix reload

You can then try sending an email so see if it works!

mail you@email.domain<ENTER>

Subject: Testing from your.relay.server.domain<ENTER>
Testing postfix remote host<ENTER>
<CTRL-D>
Cc:<ENTER>

You can also always check the postfix system logs to see what postfix thinks about it using the command:

sudo less +G /var/log/mail.log

if your system doesn't have a /var/log/mail.log, never fear! Try this instead:

sudo less +G /var/log/syslog

In either case, hit to have the log update in real time.

Installing Docker Compose and Let's Encrypt

First let's install Docker Compose (and its dependencies, like the whole Docker subsystem) and the Let's Encrypt scripts that let you procure no-cost Secure Sockets Layer certificates to secure access to your server.

sudo apt install docker-compose letsencrypt

Now we create the set of directories I use for holding Docker Compose configurations (/home/docker) and the persistent data the Docker containers create (/home/data)

D=[nextcloud domain]
sudo mkdir -p /home/data/$D
sudo mkdir -p /home/docker/$D

followed by

D=[onlyoffice domain]
sudo mkdir -p /home/data/$D
sudo mkdir -p /home/docker/$D

It's helpful to make sure that your non-root user can also read and write files in these directories:

U=[vps username]
sudo chown -R $U /home/docker
sudo chown -R $U /home/data

Installing MariaDB

MariaDB is effectively a drop-in alternative to MySQL and we prefer it because it's not controlled by Oracle and has a more active developer community. On Ubuntu, MariaDB pretends to be MySQL for compatibility purposes, so don't be weirded out by the interchangeable names below. Install the server and the client like this.

sudo apt install mariadb-server mariadb-client

You should now be able to type sudo mysql at the command prompt, and it'll log you into the MariaDB console (to get out type \q or exit)

Tweak the configuration so that it's listening on

sudo vim /etc/mysql/mariadb.conf.d/50-server.cnf

and copy the bind-address line and adjust so it looks like this - we want MariaDB to be listening on all interfaces, not just localhost (127.0.0.1)...

# Instead of skip-networking the default is now to listen only on
# localhost which is more compatible and is not less secure.
#bind-address           = 127.0.0.1
bind-address            = 0.0.0.0

Then restart MariaDB:

sudo service mysql restart

It should now be listening on port 3306 on all interfaces, i.e. 0.0.0.0. Your instance will be protected from anyone outside of your VPS connecting to it by the fact that external access to port 3306 isn't allowed by your ufw firewall.

To check it's running, you can run

sudo netstat -punta | grep 3306

and you should see something like

tcp 0 0 0.0.0.0:3306 0.0.0.0:* LISTEN 8459/mysqld

which is the 'mysqld' (the MySQL-compatible database daemon provided by MariaDB).

Now set up the database which will hold NextCloud's data. Log into the MySQL client on the host:

sudo mysql

You'll need to gin up a password for your "nextcloud" database user. I usually use pwgen (sudo apt install pwgen) - for example running this command will give you a single 19 character password without special characters (just numbers and letters):

pwgen -s 19 1

Giving you something like this (but if it's truly random, almost certainly not exactly this):

bYIOSrvR9aGwL5FRGFU

At the prompt (which will look something like MariaDB [(none)]>) enter the following lines (putting your password in place of [passwd]):

CREATE DATABASE nextcloud CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
CREATE USER "nextcloud"@"%" IDENTIFIED BY "[passwd]";
GRANT ALL ON nextcloud.* to "nextcloud"@"%";
FLUSH PRIVILEGES;

Then enter \q to exit.

Configuring Nginx reverse proxy for NextCloud and OnlyOffice

Above, we installed Nginx as well as the Let's Encrypt scripts. Now we'll configure them as it's useful to have them working before you set up your services.

In order for you, outside of your server, to see the NextCloud and OnlyOffice services, you will need to set up a secure external 'reverse proxy' on your host VPS which will accept requests for those two services from the Internet and pass those requests securely to the two sets of Docker containers providing the services. These will answer to https://[nextcloud domain] (for NextCloud) and https://[onlyoffice domain] for OnlyOffice.

Let's Encrypt will provide the SSL certificates (each is a file with a specially generated, very long string) which we use to limit access to our services to encrypted (secure) connections (protecting both our users and ourselves from external enemies).

Note: many thanks to Stephen Harlow (who crash-tested this tutorial!) for pointing out that you might need to run the following if you're not finding the 'Snakeoil certs' on your system (running them just to be safe shouldn't cause any issues):

sudo make-ssl-cert generate-default-snakeoil

Let's Encrypt setup

sudo mkdir /etc/nginx/includes

Then we create that configuration file itself:

sudo $EDIT /etc/nginx/includes/letsencrypt.conf

into which we copy-and-paste the following (no [tokens] to replace in this one!)

# Rule for legitimate ACME Challenge requests
location ^~ /.well-known/acme-challenge/ {
    default_type "text/plain";
    # this can be any directory, but this name keeps it clear
    root /var/www/letsencrypt;
}
 
# Hide /acme-challenge subdirectory and return 404 on all requests.
# It is somewhat more secure than letting Nginx return 403.
# Ending slash is important!
location = /.well-known/acme-challenge/ {
    return 404;
}

sudo mkdir /var/www/letsencrypt

NextCloud Proxy Configuration

To configure the NextCloud proxy, you need to create this configuration file in your /etc/nginx/sites-available/ directory.

Create a file with a meaningful name for your NextCloud Proxy, something like "nextcloud" (I use the domain name I've chosen, e.g. for docs.oeru.org I call the proxy file "docs.oeru.org" - keeps everything clear, and I can have multiple instances on the same server if I want...). Let's go with in this instance (change it if you prefer)

sudo $EDIT /etc/nginx/sites-available/nextcloud

with the following contents, replacing [nextcloud domain] with your selected domain name, but leave off the [ ] (those are just there to make sure nginx errors if you've missed replacing any) - and the port number 8080 if you've opted to change to a different one!:

server {
    listen 80;
    listen [::]:80;
 
    # note, you can add additional domain names, separated by a space, to which this config will answer.
    server_name [nextcloud domain];
 
    include includes/letsencrypt.conf;
 
    # enforce https
    location / {
        return 302 https://$server_name$request_uri;
    }
}
 
server {
    listen 443 ssl;
    listen [::]:443 ssl;
 
    # note, you can add additional domain names, separated by a space, to which this config will answer.
    server_name [nextcloud domain];
 
    ## Access and error logs.
    access_log /var/log/nginx/[nextcloud domain]_access.log;
    error_log /var/log/nginx/[nextcloud domain]_error.log;
 
    # these are temporary certificates, used only long enough to secure Let's Encrypt certs as below.
    ssl_certificate /etc/ssl/certs/ssl-cert-snakeoil.pem;
    ssl_certificate_key /etc/ssl/private/ssl-cert-snakeoil.key;
 
    # these need to be commented out until after the Let's Encrypt
    # certificates have been acquired
    #ssl_certificate /etc/letsencrypt/live/[nextcloud domain]/fullchain.pem;
    #ssl_certificate_key /etc/letsencrypt/live/[nextcloud domain]/privkey.pem;
 
    # from http://axiacore.com/blog/enable-perfect-forward-secrecy-nginx/
    ssl_session_cache shared:SSL:10m;
    ssl_session_timeout  10m;
    # limit_req_zone $binary_remote_addr zone=one:10m rate=1r/s;
    # forward secrecy settings
    ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
    ssl_prefer_server_ciphers on;
    ssl_ciphers "EECDH+ECDSA+AESGCM EECDH+aRSA+AESGCM EECDH+ECDSA+SHA384 EECDH+ECDSA+SHA256 EECDH+aRSA+SHA384 EECDH+aRSA+SHA256 EECDH+aRSA+RC4 EECDH EDH+aRSA RC4 !aNULL !eNULL !LOW !3DES !MD5 !EXP !PSK !SRP !DSS !RC4";
    ssl_dhparam /etc/ssl/certs/dhparam.pem;
 
    # The following 2 rules are only needed for the user_webfinger app.
    # Uncomment it if you're planning to use this app.
    rewrite ^/.well-known/host-meta /public.php?service=host-meta last;
    rewrite ^/.well-known/host-meta.json /public.php?service=host-meta-json last;
 
    # The following rule is only needed for the Social app.
    # Uncomment it if you're planning to use this app.
    rewrite ^/.well-known/webfinger /public.php?service=webfinger last;
 
    location ^~ / {
        proxy_pass http://127.0.0.1:8080;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "Upgrade";
        proxy_set_header Host $http_host;
        proxy_read_timeout 36000s;
        proxy_buffering off;
        proxy_max_temp_file_size 15000m;
    }
    client_max_body_size 1G;
    fastcgi_buffers 64 4K;
    add_header Strict-Transport-Security "max-age=31536000; includeSubdomains;";
    # Remove X-Powered-By, which is an information leak
    fastcgi_hide_header X-Powered-By;
}

Note: you'll need to create the file cited in the proxy configuration: /etc/ssl/certs/dhparam.pem

You can do this as follows (install the necessary software, backup any possible existing version as a matter of prudence, and create a new one):

sudo apt update && sudo apt install openssl
sudo [ -f "/etc/ssl/certs/dhparam.pem" ] && sudo mv /etc/ssl/certs/dhparam.pem /etc/ssl/certs/dhparam.pem.bak
sudo openssl dhparam -out /etc/ssl/certs/dhparam.pem 2048

Once those are created, you have to make sure that they're "enabled" (replacing with your file names, of course):

cd /etc/nginx/sites-enabled sudo ln -sf ../sites-available/nextcloud .

To confirm that there aren't any typos or issues that might make nginx unhappy, run

sudo nginx -t

If all's well, get nginx to reread its configuration with the new files (if not, it might be because you missed replacing one of the [tokens]):

sudo service nginx reload

OnlyOffice Proxy Configuration

The OnlyOffice proxy configuration uses a very similar process to the one above. You just need to create another configuration file:

upstream docservice {
   server 127.0.0.1:9880;
}
 
map $http_host $this_host {
   "" $host;
   default $http_host;
}
 
map $http_x_forwarded_proto $the_scheme {
   default $http_x_forwarded_proto;
   "" $scheme;
}
 
map $http_x_forwarded_host $the_host {
    default $http_x_forwarded_host;
    "" $this_host;
}
 
map $http_upgrade $proxy_connection {
  default upgrade;
  "" close;
}
 
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $proxy_connection;
proxy_set_header X-Forwarded-Host $the_host;
proxy_set_header X-Forwarded-Proto $the_scheme;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
 
server {
    listen 80;
    listen [::]:80;
 
    server_name [onlyoffice domain];
 
    # for let's encrypt renewals!
    include /etc/nginx/includes/letsencrypt.conf;
 
    ## Access and error logs.
    access_log /var/log/nginx/[onlyoffice domain]_access.log;
    error_log /var/log/nginx/[onlyoffice domain]_error.log;
 
    # redirect all HTTP traffic to HTTPS.
    location / {
        return  302 https://$server_name$request_uri;
    }
}
 
# This configuration assumes that there's an nginx container talking to the mautic PHP-fpm container,
# and this is a reverse proxy for that Mautic instance.
server {
    listen 443 ssl;
    listen [::]:443 ssl;
 
    server_name [onlyoffice domain];
 
    ssl_certificate /etc/ssl/certs/ssl-cert-snakeoil.pem;
    ssl_certificate_key /etc/ssl/private/ssl-cert-snakeoil.key;
    #ssl_certificate /etc/letsencrypt/live/[onlyoffice domain]/fullchain.pem;
    #ssl_certificate_key /etc/letsencrypt/live/[onlyoffice domain]/privkey.pem;
    ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
    # to create this, see https://raymii.org/s/tutorials/Strong_SSL_Security_On_nginx.html
    ssl_dhparam /etc/ssl/certs/dhparam.pem;
    keepalive_timeout 20s;
    # for let's encrypt renewals!
    include /etc/nginx/includes/letsencrypt.conf;
 
    proxy_http_version 1.1;
    proxy_buffering off;
 
    ## Access and error logs.
    access_log /var/log/nginx/[onlyoffice domain]_access.log;
    error_log /var/log/nginx/[onlyoffice domain]_error.log;
 
    add_header Strict-Transport-Security max-age=31536000;
    # add_header X-Frame-Options SAMEORIGIN;
    add_header X-Content-Type-Options nosniff;
 
    # see https://github.com/ONLYOFFICE/document-server-proxy/blob/master/nginx/proxy-https-to-http.conf
    location / {
        proxy_pass http://docservice;
        proxy_http_version 1.1;
    }
}

After that's done, we'll repeat what we did for the NextCloud config:

sudo cd /etc/nginx/sites-enabled sudo ln -sf ../sites-available/onlyoffice . sudo nginx -t

and, if there're no errors, run

sudo service nginx reload

Now we're ready to request Let's Encrypt certificates.

Requesting Let's Encrypt certificates

To request Let's Encrypt SSL certificates for your NextCloud and OnlyOffice services, run the following, replacing the [token], of course (note that 'certbot' is the script provided by the Let's Encrypt package - historically, it could also be called via 'letsencrypt', although apparently the latter is now deprecated):

sudo certbot --webroot -w /var/www/letsencrypt -d [nextcloud domain]

Note - if you want to address your instance from multiple domains, use one (or more) -d [another domain] - just make sure that

all those domains already point to your VPS, and
those domains are included in the Nginx proxy configuration above.

otherwise the Let's Encrypt certbot request will fail!

Saving debug log to /var/log/letsencrypt/letsencrypt.log
Enter email address (used for urgent renewal and security notices)
 (Enter 'c' to cancel): webmaster@fossdle.org
 
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Please read the Terms of Service at
https://letsencrypt.org/documents/LE-SA-v1.3-September-21-2022.pdf. You must
agree in order to register with the ACME server. Do you agree?
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
(Y)es/(N)o: y
 
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Would you be willing, once your first certificate is successfully issued, to
share your email address with the Electronic Frontier Foundation, a founding
partner of the Let's Encrypt project and the non-profit organization that
develops Certbot? We'd like to send you email about our work encrypting the web,
EFF news, campaigns, and ways to support digital freedom.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
(Y)es/(N)o: y
Account registered.
Requesting a certificate for [nextcloud domain]
 
Successfully received certificate.
Certificate is saved at: /etc/letsencrypt/live/[nextcloud domain]/fullchain.pem
Key is saved at:         /etc/letsencrypt/live/[nextcloud domain]/privkey.pem
This certificate expires on (some future date).
These files will be updated when the certificate renews.
Certbot has set up a scheduled task to automatically renew this certificate in the background.
 
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
If you like Certbot, please consider supporting our work by:
 * Donating to ISRG / Let's Encrypt:   https://letsencrypt.org/donate
 * Donating to EFF:                    https://eff.org/donate-le
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Ideally, you'll see a message like the above. If not, and there's an error, the error messages they provide are usually very useful and accurate. Fix the problem and try again. Note, your SSL certificate will have the name of your [nextcloud domain], even if it also provide support for [second domain name] (or third, fourth, etc.).

Once you have a Let's Encrypt certificate, you can update our NGINX configuration:

sudo $EDIT /etc/nginx/sites-available/[nextcloud domain]

and swap all occurrences of

    ssl_certificate /etc/ssl/certs/ssl-cert-snakeoil.pem;
    ssl_certificate_key /etc/ssl/private/ssl-cert-snakeoil.key;
#    ssl_certificate /etc/letsencrypt/live/[nextcloud domain]/fullchain.pem;
#    ssl_certificate_key /etc/letsencrypt/live/[nextcloud domain]/privkey.pem;

#    ssl_certificate /etc/ssl/certs/ssl-cert-snakeoil.pem;
#    ssl_certificate_key /etc/ssl/private/ssl-cert-snakeoil.key;
    ssl_certificate /etc/letsencrypt/live/[nextcloud domain]/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/[nextcloud domain]/privkey.pem;

which enables your new domain-specific SSL certificate. Check that NGINX is happy with your change:

sudo nginx -t

and if so,

sudo service nginx reload

You domain should now be enabled for https:// access. Note that going to http://[nextcloud domain] should automatically redirect you to https://[nextcloud domain] because you care about your user's security! :grin:

Now you'll have to repeat the same process for the [onlyoffice domain]. When that's done... Onward!

Prepare your Docker Compose host

We make use of the NextCloud community's stable Docker container which they keep (more or less) up-to-date. Similarly, the OnlyOffice developers maintain a Docker container, too. We will run them both on this same server as separate services via Docker Compose. The two sets of Docker containers will look like this:

a suite of NextCloud containers:

the main PHP-FPM container (which provides most of the functionality for NextCloud using the PHP scripting engine,
an identical container to the PHP one which runs the cron service (which does periodic administrative tasks relevant to NextCloud)
a Redis container (which provides performance improving caching for NextCloud), and
an Nginx webserver container which makes it easier to manage the configuration and paths of the NextCloud instance. It means that on the hosting server, we only need to run a proxying web server, which is easy.

the single OnlyOffice container which, despite the Docker convention of each container running only a single services, runs the whole OnlyOffice stack, which includes PostgreSQL, Nginx, Rabbit-MQ, Python, and NodeJS.

The way I prefer to implement this set of containers is to use:

sudo apt install docker-compose

to set up the entire Docker and Docker Compose system on your server. If you desire, you can also set up a newer version of Docker Compose, namely the 2.x series. The command line interface and feedback on actions is nicer with this newer version, but otherwise, there's little difference (a few other services I run require 2.x, like Mailcow, so I've had experience with both).

Then set up a place for your Docker containers and the associated persistent data (your Docker containers should hold no important data - you should be able to delete and recreate them entirely without losing any important data or configuration):

sudo mkdir /home/docker
sudo mkdir /home/docker/nextcloud
sudo mkdir /home/docker/onlyoffice
sudo mkdir /home/data
sudo mkdir /home/data/nextcloud
sudo mkdir /home/data/nextcloud/nginx
sudo mkdir /home/data/onlyofficesudo
chown -R ${USER}:${USER} /home/docker /home/data

My personal convention is to name both docker and data directories after the specific domain name of the service to which they apply - makes it easier when, for example, I have multiple instances of NextCloud on a single server. The above is intended to be straight forward for folks only running one of each - but feel free to modify for your requirements. If you do so, remember to ripple that through the rest of these instructions!

NextCloud Install

Install the NextCloud Docker recipe

Now we have a place to put the really key bit - the code for running NextCloud and OnlyOffice via Docker Compose. First, let's set up NextCloud (this also installs the OnlyOffice server):

cd /home/docker/nextcloud

You'll have to create a file, e.g via

$EDIT docker-compose.yml

and fill it with this (substituting the values in [] to suit your details - and changing the paths in /home/data if you've used something different than the default above!):

version: '3'
services:
  nginx:
    container_name: nginx-server
    image: nginx
    ports:
      - 127.0.0.1:8080:80
    volumes:
      - /home/data/nextcloud/nginx/nginx.conf:/etc/nginx/nginx.conf:ro
      - /home/data/nextcloud/nextcloud:/var/www/html
    links:
      - app
    environment:
      - VIRTUAL_HOST
    restart: unless-stopped
  app:
    container_name: app-server
    image: nextcloud:fpm
    stdin_open: true
    tty: true
    links:
      - redis
    expose:
      - '80'
      - '9000'
    volumes:
      - /home/data/nextcloud/nextcloud:/var/www/html
    environment:
      - REDIS_HOST=redis
      - REDIS_HOST_PASSWORD=[redis password]
    extra_hosts:
      - '[onlyoffice domain]:[ipv4]'
    restart: unless-stopped
  cron:
    image: nextcloud:fpm
    volumes:
      - /home/data/nextcloud/nextcloud:/var/www/html
    user: www-data
    entrypoint: |
      bash -c 'bash -s <<EOF
      trap "break;exit" SIGHUP SIGINT SIGTERM
      while /bin/true; do
        /usr/local/bin/php /var/www/html/cron.php
        sleep 900
      done
      EOF'
    restart: unless-stopped
  redis:
    image: redis:alpine
    command: redis-server --requirepass [redis password]
    volumes:
      - /home/data/nextcloud/redis:/data
    restart: unless-stopped

The "port" specified above, 8080, for nginx is arbitrary - I picked it to ensure it doesn't don't conflict with ports being used by other containers on my server - you can use this value if you want, or use sudo netstat -punta (you might need to install the package that provides netstat first, sudo apt install net-tools) to see what ports are currently claimed by other services on your server (if there are any) and pick one that doesn't clash! If it scroll past too fast, you can pipe it into less to allow you to scroll and search like this: sudo netstat -punta | less - hit "q" to exit or "/" to initiate a text search. Or, if you want verify that a specific port is not already being used, you can do this (in this case for port 8080) via sudo netstat -punta | grep 8080 - if it returns any results, something is already listening on that port. If not, it's available.

The NextCloud Nginx configuration

You will also need to provide the "nginx.conf" file referenced in the nginx section of the Docker Compose configuration. Do that via

$EDIT /home/data/nextcloud/nginx/nginx.conf

and copy-and-paste the following incantation (you shouldn't need to change anything in this one) - there're notes in it offering some explanations:

worker_processes auto;
 
error_log  /var/log/nginx/error.log warn;
pid        /var/run/nginx.pid;
 
 
events {
    worker_connections  1024;
}
 
 
http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;
 
    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';
 
    access_log  /var/log/nginx/access.log  main;
 
    sendfile        on;
    #tcp_nopush     on;
 
    keepalive_timeout  65;
 
    set_real_ip_from  10.0.0.0/8;
    set_real_ip_from  172.16.0.0/12;
    set_real_ip_from  192.168.0.0/16;
    real_ip_header    X-Real-IP;
 
    #gzip  on;
 
    map $http_host $this_host {
        "" $host;
        default $http_host;
    }
 
    map $http_x_forwarded_proto $the_scheme {
        default $http_x_forwarded_proto;
        "" $scheme;
    }
 
    map $http_x_forwarded_host $the_host {
        default $http_x_forwarded_host;
        "" $this_host;
    }
 
    upstream php-handler {
        server app-server:9000;
    }
 
    server {
        listen 80;
 
        # Add headers to serve security related headers
        # Before enabling Strict-Transport-Security headers please read into this
        # topic first.
        #add_header Strict-Transport-Security "max-age=15768000; includeSubDomains; preload;" always;
        #
        # WARNING: Only add the preload option once you read about
        # the consequences in https://hstspreload.org/. This option
        # will add the domain to a hardcoded list that is shipped
        # in all major browsers and getting removed from this list
        # could take several months.
        add_header Referrer-Policy "no-referrer" always;
        add_header X-Content-Type-Options "nosniff" always;
        add_header X-Download-Options "noopen" always;
        add_header X-Frame-Options "SAMEORIGIN" always;
        add_header X-Permitted-Cross-Domain-Policies "none" always;
        add_header X-Robots-Tag "noindex, nofollow" always;
        add_header X-XSS-Protection "1; mode=block" always;
 
        # Remove X-Powered-By, which is an information leak
        fastcgi_hide_header X-Powered-By;
 
        # Path to the root of your installation
        root /var/www/html;
 
        location = /robots.txt {
            allow all;
            log_not_found off;
            access_log off;
        }
 
        # The following 2 rules are only needed for the user_webfinger app.
        # Uncomment it if you're planning to use this app.
        #rewrite ^/.well-known/host-meta /public.php?service=host-meta last;
        #rewrite ^/.well-known/host-meta.json /public.php?service=host-meta-json last;
 
        # The following rule is only needed for the Social app.
        # Uncomment it if you're planning to use this app.
        #rewrite ^/.well-known/webfinger /public.php?service=webfinger last;
 
        location = /.well-known/carddav {
            return 301 $scheme://$host:$server_port/remote.php/dav;
        }
 
        location = /.well-known/caldav {
            return 301 $scheme://$host:$server_port/remote.php/dav;
        }
 
        # set max upload size
        client_max_body_size 10G;
        fastcgi_buffers 64 4K;
 
        # Enable gzip but do not remove ETag headers
        gzip on;
        gzip_vary on;
        gzip_comp_level 4;
        gzip_min_length 256;
        gzip_proxied expired no-cache no-store private no_last_modified no_etag auth;
        gzip_types application/atom+xml application/javascript application/json application/ld+json application/manifest+json application/rss+xml application/vnd.geo+json application/vnd.ms-fontobject application/x-font-ttf application/x-web-app-manifest+json application/xhtml+xml application/xml font/opentype image/bmp image/svg+xml image/x-icon text/cache-manifest text/css text/plain text/vcard text/vnd.rim.location.xloc text/vtt text/x-component text/x-cross-domain-policy;
 
        # Uncomment if your server is build with the ngx_pagespeed module
        # This module is currently not supported.
        #pagespeed off;
 
        location / {
            rewrite ^ /index.php;
        }
 
        location ~ ^\/(?:build|tests|config|lib|3rdparty|templates|data)\/ {
            deny all;
        }
        location ~ ^\/(?:\.|autotest|occ|issue|indie|db_|console) {
            deny all;
        }
 
        location ~ ^\/(?:index|remote|public|cron|core\/ajax\/update|status|ocs\/v[12]|updater\/.+|oc[ms]-provider\/.+)\.php(?:$|\/) {
            fastcgi_split_path_info ^(.+?\.php)(\/.*|)$;
            set $path_info $fastcgi_path_info;
            try_files $fastcgi_script_name =404;
            include fastcgi_params;
            fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
            fastcgi_param PATH_INFO $path_info;
            # fastcgi_param HTTPS on;
 
            # Avoid sending the security headers twice
            fastcgi_param modHeadersAvailable true;
 
            # Enable pretty urls
            fastcgi_param front_controller_active true;
            fastcgi_pass php-handler;
            fastcgi_intercept_errors on;
            fastcgi_request_buffering off;
        }
 
        location ~ ^\/(?:updater|oc[ms]-provider)(?:$|\/) {
            try_files $uri/ =404;
            index index.php;
        }
 
        # Adding the cache control header for js, css and map files
        # Make sure it is BELOW the PHP block
        location ~ \.(?:css|js|woff2?|svg|gif|map)$ {
            try_files $uri /index.php$request_uri;
            add_header Cache-Control "public, max-age=15778463";
            # Add headers to serve security related headers (It is intended to
            # have those duplicated to the ones above)
            # Before enabling Strict-Transport-Security headers please read into
            # this topic first.
            #add_header Strict-Transport-Security "max-age=15768000; includeSubDomains; preload;" always;
            #
            # WARNING: Only add the preload option once you read about
            # the consequences in https://hstspreload.org/. This option
            # will add the domain to a hardcoded list that is shipped
            # in all major browsers and getting removed from this list
            # could take several months.
            add_header Referrer-Policy "no-referrer" always;
            add_header X-Content-Type-Options "nosniff" always;
            add_header X-Download-Options "noopen" always;
            add_header X-Frame-Options "SAMEORIGIN" always;
            add_header X-Permitted-Cross-Domain-Policies "none" always;
            add_header X-Robots-Tag "noindex, nofollow" always;
            add_header X-XSS-Protection "1; mode=block" always;
 
            # Optional: Don't log access to assets
            access_log off;
        }
 
        location ~ \.(?:png|html|ttf|ico|jpg|jpeg|bcmap|mp4|webm)$ {
            try_files $uri /index.php$request_uri;
            # Optional: Don't log access to other assets
            access_log off;
        }
        add_header Strict-Transport-Security "max-age=31536000; includeSubdomains;";
    }
}

That should be all the configuration you need to make the NextCloud Docker containers go.

The OnlyOffice Docker configuration

We also need to something similar (but easier) for OnlyOffice. This is the docker-compose.yml that I use:

cd /home/docker/onlyoffice $EDIT docker-compose.yml

and copy-and-paste this into it (replacing the []):

version: '3'
services:
  onlyoffice:
    image: onlyoffice/documentserver:latest
    restart: unless-stopped
    ports:
      - 127.0.0.1:9880:80
    environment:
#      - JWT_SECRET=[onlyoffice secret]
    volumes:
      - /home/data/onlyoffice/data:/var/www/onlyoffice/Data
      - /home/data/onlyoffice/logs:/var/log/onlyoffice
      - /home/data/onlyoffice/lib:/var/lib/onlyoffice
      - /home/data/onlyoffice/db:/var/lib/postgresql
    extra_hosts:
      - "[nextcloud domain]:[ipv4]"

Now, we can fire it up provisionally:

docker-compose up -d && docker-compose logs -f

We'll find the value of [onlyoffice secret] a bit later, below.

To get back to a command prompt without killing the running Docker container execute a CTRL-C.

Firing up your NextCloud!

Phew - congratulations on getting here! We've reached the moment of truth where we need to see if this whole thing will work!

We need to make sure we're back in the NextCloud Docker directory we set up:

cd /home/docker/nextcloud

Then you can run:

docker-compose up -d && docker-compose logs -f

This will trigger the initial download of the docker container images you've specified in your docker-compose.yml file. All going well, after a few minutes (longer or shorter depending on the speed of your server's connection) you should have download the Nginx, Redis, and NextCloud Docker images, and then the script will attempt to start them (bringing them "up" in daemon mode with the -d, meaning they'll keep running even if you log out) and then, if successful, the logs -f command will run and show you a stream of log messages from the containers, each preceded by the container name to which it corresponds. This should help you debug any problems that occur during the process (ideally, none).

Once you see log messages streaming past, and no obvious "container exited" or other error messages (which will usually contain the word "error" a lot), you should be able to point your browser at your selected domain name and have your fist visit to your NextCloud in your browser! Just point your web browser at https://[nextcloud domain] (replacing with your domain, of course. You should also try going to http://[nextcloud domain] (note the missing 's' from http) which should automatically redirect you to https://[nextcloud domain] as the reverse proxy file instructs.

Again, to get back to a command prompt without killing the running Docker containers execute a CTRL-C.

The NextCloud source code (if necessary)

Normally the source code for NextCloud's current stable version is transparently downloaded and installed by the NextCloud Docker container the first time it's instantiated. If it is, you'll see a bunch of files and directories in your /home/data/nextcloud/nextcloud folder. If so, you're fine and you can move on to the next step.

If not, you can always find the most recent stable release's source code here. I tend to prefer the .tar.bz2 archive format, so I get it from this link: https://download.nextcloud.com/server/releases/latest.tar.bz2 (which, fingers crossed, should remain valid indefinitely - if not, check the previous link or look for 'Download' on the NextCloud website.).

We need to get that file and extract it in /home/data/nextcloud, so do the following (if wget isn't already installed, get it via sudo apt install wget):

cd /home/data/nextcloud
wget https://download.nextcloud.com/server/releases/latest.tar.bz2
tar xvfj latest.tar.bz2

which will create a directory 'nextcloud' with the latest (stable) version of the NextCloud source code in that directory.

Then reassert the file permissions just to be sure

sudo chown -R www-data nextcloud

After that, you should be able to point your browser at your domain (the containers are already running) and see if it starts the install process as it should. [/code]

You can figure out what version that is by running:

cat nextcloud/version.php | grep VersionString

With my latest install, I get the result:

$OC_VersionString = '26.0.2';

Note, I tend to hold on to install archives for safety's sake, so I generally do the following to tidy up (still in the nextcloud data directory), replacing [version] with the advertised most recent stable version of NextCloud (26.0.2 in my case):

mkdir attic
mv latest.tar.bz2 attic/nextcloud_[version].tar.bz2

Now you've got the source code for NextCloud where you containers are configured to look for it!

Configuring database access

On doing so, if all is well, you should be directed through the database set up process for your NextCloud instance. You'll be asked for your database details, which should be:

database IP: 172.17.0.1 - this is the default IP of the Docker host server.
database name: [db name]
database user: [db user]
database password: [db password]

Configuring the Admin user

Once that's set and working, NextCloud will install all the relevant database tables and initial data. You'll be asked to set up an admin user account, which can be "admin" and some strong password you create (you can use the pwgen utility you used earlier) - I'd recommend recording it somewhere. I would not recommend making your own account, in your name, the main admin account. Instead, I recommend creating a second account, with administrator privileges, for yourself, but leave the admin account purely for administrative activities.

Configuring Outgoing Email

To allow your NextCloud instance to send outgoing email, so that your site can alert you to security updates that need to be applied, or so that any of your NextCloud users can request a password reset if they've forgot theirs. For this, you'll need the authenticating SMTP account details from the start of this process. You'll need:

SMTP server : an IP address or a domain name
SMTP username: a username or an email address
SMTP password: a strong password already configured for the username on that server
SMTP login security: whether login is via TLS, SSL, or unsecure (!!), and
SMTP login method: plain, encrypted, "login" or some other value.

You should be able to test your email settings to make sure the details you've entered are valid. If you need to adjust these settings later, you can go to the admin menu (top right of the web browser interface) and go to Admin->Basic Settings - should have a path of https://[nextcloud domain]/settings/admin.

Setting up OnlyOffice

The OnlyOffice server should already be running - if you point your browser at https://[onlyoffice domain] you should see a page like the attached screenshot with the OnlyOffice Logo and a title of "OnlyOffice Docs Community Edition".

Configuring OnlyOffice Integration with NextCloud

To configure your NextCloud to use your OnlyOffice, the OnlyOffice will require that NextCloud knows its "secret". To generate the secret, run this in /home/docker/onlyoffice:

docker-compose exec onlyoffice /var/www/onlyoffice/documentserver/npm/json -f /etc/onlyoffice/documentserver/local.json 'services.CoAuthoring.secret.session.string'

The resulting secret string, which will look something like QC7QmEqUpXmmnwXZcvBQ needs to be added to your OnlyOffice docker-compose.yml file to ensure that the same code is used everytime you start OnlyOffice (if it isn't set, it'll be generated each time you restart OnlyOffice and your NextCloud will need a different 'secret' each time - a major inconvenience).

$EDIT /home/docker/onlyoffice/docker-compose.yml

Add it in place of [onlyoffice secret] and also remove the '#' that's commenting out the line - again, thanks to Stephen Harlow for pointing out that this is required! Then restart OnlyOffice via

docker-compose up -d

Docker will see that the container's configuration has changed and will restart the container.

Next you need to be logged into your NextCloud as an administartive user (your own user is fine if you've given it admin privileges).

You should have an "admin" menu (assuming you've created your user with Administrator privileges) at the top right of the web interface. If you go to Apps, you can install the new "Hub bundle" available under the "App bundles" option (see attached image). If you don't want the whole bundle you can just use the search box to search for "OnlyOffice" or go to the "Office & text" App category and enable the OnlyOffice "official" app, at which point it will automatically download the latest version of the connector app and install it (it should appear in your /home/data/nextcloud/apps directory)

Once you've done that, go to your top right menu again, selecting Admin, and you should see "OnlyOffice" as an option in the left column (which starts with "Basic settings"). Selecting that, you'll need to enter the following:

"Document Editing Service address": https://[onlyoffice domain]
"Secret key": [onlyoffice secret]

You don't need the 'advanced settings'.

When you're done, click "Save".

You can also select formats you'd like OnlyOffice to open and edit files of those types are clicked or created. I've selected the following: doc, docx, odp, ods, odt, ppt, pptx, xls, xlsx, and in the second section: csv and txt.

You can also make other editor customisations as you desire. The only Editor customisation setting I haven't selected is "Display Chat menu button" because NextCloud Hub provides an integrated Chat service, making this one within OnlyOffice an unnecessary distraction.

Once finished configuring, you should have the ability to go back to the home of your NextCloud install, which should show you your top-level folders. If you click the "+" next to the home icon (top left of the folder pane) you should now have the option to create (in addition to "Upload file", "New folder", "New text file") a "New Document", "New Spreadsheet", and "New Presentation". Clicking those should give you the OnlyOffice interface for the designated content type.

Similarly, you can use the "Upload file" to upload a document in a format that is supported by OnlyOffice. Once uploaded, clicking on the filename should open it for editing in the appropriate OnlyOffice interface.

It is saved as it is changed, so you shouldn't need to save it explicitly.

Keeping the whole thing up-to-date

So, as you're no doubt aware, both NextCloud and OnlyOffice are always being improved and updated. I certainly encourage you to keep your installations up-to-date.

While you'll periodically be alerted that NextCloud apps have available updates (these can be upgraded through the browser interface) updates to the NextCloud and OnlyOffice systems themselves need to be undertaken by upgrading their containers. Luckily it's easy to do although I strongly urge you to ensure you have a very recent backup of both database and uploaded files - they're the files in /home/data/nextcloud/data and /home/data/onlyoffice/ (note, backups of OnlyOffice are complicated somewhat by the fact that you can't reliably back up running PostgreSQL instance simply by backing up its files - see a solution below). Prior to updating my containers, I normally create an archive of my NextCloud code in the event the upgrade goes wrong and I need to recover quickly. I do it like this. In my /home/data/nextcloud directory, I create a new 'attic' directory:

cd /home/data/nextcloud && sudo mkdir attic

then I create an archive of my entire NextCloud source directory in the attic directory:

DATE=date +%Y%m%d && sudo tar cvfz attic/nextcloud-${DATE}.tgz --exclude "data" nextcloud

If you need to recover files from it, you can untar it (assuming you have sufficient disk space! Best to be mindful of that!) via

sudo mkdir tmp && cd tmp && tar xvfz ../attic/nextcloud-${DATE}.tgz

although if you're doing it in a different SSH session (e.g. after the fact) you might have to manually enter the DATE part of the filename.

Once you've got a backup of your NextCloud source code, updating the container should be as easy as either doing another

docker-compose pull

and then restarting the service with the new containers via

docker-compose up -d

which will remove any old containers (this won't remove any data you want to save if you followed the directions above! But remember to do it in the right directory!) and start up the new versions you've just pulled.

Use docker-compose logs -f to watch the logs - you'll likely see useful debugging information in the unlikely event that something goes wrong in the upgrade process.

Backing up NextCloud

To back up your instance on your server, you need two things: a file system backup of your /home/data/nextcloud directory, and database dumps of your database.

There're lots of ways to back up your files (I've recently updated to using a system called Restic to make off-server incremental encrypted backups - I plan to document this in a future howto! - although there're other documented approaches - leave a comment below if you'd like to learn more about my approach!).

Backing up your MariaDB databases is as easy installing automysqlbackups:

sudo apt install automysqlbackups

You'll find daily versioned dumps of your MariaDB database(s) in /var/lib/automysqlbackups on your VM host's filesystem. To run an ad hoc backup (which will replace the previous backup from that day, if there is one) just run

sudo automysqlbackups

Backup OnlyOffice

Along with backing up the files in your /home/data/onlyoffice directory, you'll also want a proper "dump" of your PostgreSQL backup (you can write simple bash scripts to do this regularly, automatically), particularly prior to doing an upgrade (to allow for recovery if something goes badly wrong, which is always possible). You can achieve this by going to

cd /home/docker/onlyoffice

and running this

DATE=$(date +%Y%m%d) && FILE=/home/data/onlyoffice/backup/fullbackup-${DATE}.sql && docker-compose exec onlyoffice sudo -u postgres pg_dumpall > ${FILE} && gzip ${FILE}

which will assign the current date to DATE, the relevant filename to FILE, and then put the backup SQL into a dated file called $FILE and compress the result with gzip :)

At some point, I'll modify my normal versioned PostgreSQL-in-a-Docker-Container dated database backup scripts to cater for this solution and make the result available on https://git.oeru.org - it'll probably be a small modification to this script: https://git.oeru.org/oeru/docker-compose-dbbackup in case someone wants to beat me to it!

Add new comment

Updating OER Foundation Web Services for February 2023

dave — Thu, 02 Feb 2023 02:09:50 +0000

Updating OER Foundation Web Services for February 2023 dave Thu 02/02/2023 - 15:09

It's been about six months since my last update and, wow, a lot has changed. The OER Foundation (OERF) has embarked on a new initiative inspired by our old friend the Fediverse and the ramifications to on our sustainability thanks to our newer nemesis, Covid19: the Free and Open Source Software (FOSS) Digital Learning Ecosystem (DLE). Here's a paper we wrote which describes the initial FOSSDLE concept in detail, including technologies, costs, and a case study. But the FOSSDLE concept is evolving rapidly.

Thanks to increasing inequity (with the haves getting more, and have-nots less) and the prospects for addressing that imbalance (education, especially for women in the developing world), we need to do something new and different. Something new - to most folks at least - is the relatively rapid rise from obscurity of the 'Fediverse', coincident with Elon Musk taking over Twitter. For those unfamiliar with the term Fediverse (a portmanteau of 'Federated Universe'), it refers to a family of independently developed decentralised FOSS social media technologies, instances of which are 'federated' (link and talk to one another) by means of automated data transfer that adheres to an open standard called "ActivityPub". We have taken note of the diverse range of applications making up the Fediverse, and the power of being able to view the digital world through each of those lenses, on ones own terms. We think that the pedagogical opportunities the Fediverse represents are rich and varied.

In addition to running a wide set of Fediverse-enabled technologies, we've also been working to improve our measurement and monitoring game - ensuring that our services are looked after, so we're the first to know when there's a problem. We also want to be able to measure our impact in a way that's compatible with user freedom and privacy... so that's resulted in still more new services.

Finally, we've identified a number of new FOSS technologies we consider to be better than those we previously considered 'best-of-breed', and so, in the 'rapid evolution' approach of FOSS technologies, we've implemented them and made the move.

Here's where things stand now: in February 2023, we run the following production systems:

For the FOSSDLE Commons, we have:

A series of Fediverse apps:

a Wordpress Multisite - https://fossdle.org - our main site, with subsites for related purposes, like support and other documentation. Posts are shared via the Fediverse using the Wordpress ActivityPub plugin. Wordpress, a vastly extensible FOSS blogging engine, is the single, most widely use platform on the World Wide Web - 43% of all websites and 65% of websites that use a Content Management System (CMS) are built on Wordpress. The next two closest web platforms are also FOSS - Joomla (4.6% of CMSs) and Drupal (3%). Wordpress is unrivalled.
a Mastodon instance - https://social.fossdle.org - superficially similar to Twitter, but decentralised and open and many other subtle (but important) differences.
a PixelFed - https://pic.fossdle.org - similar to Instagram.
a Lemmy instance - https://link.fossdle.org - a link aggregator similar to Reddit.
an OwnCast instance - https://stream.fossdle.org - a streaming platform like Twitch, but streams and viewer comments are Fediverse content.
a WriteFreely instance - https://write.fossdle.org - a no-nonsense blogging system similar to a decentralised Fediverse-enabled Medium.

In the near term, we plan to add

a PeerTube instance - a distributed YouTube-like video hosting service (update: stream.fossdle.org).
a Mobilizon instance - a distributed Event management and registration service (update: events.oeru.org).

We have also added a number of non-Fediverse apps to support the FOSSDLE Commons initiative.

a Matrix server - https://matrix.fossdle.org, with an Element client - https://chat.fossdle.org - Matrix is a hugely powerful open standard for defining 'chat' applications, and it's also a reference implementation of that standard. It's a fully FOSS alternative to Slack or MS Teams, and integrates with IRC and just about anything else under the sun.
an Authentik instance - https://auth.fossdle.org - an authentication and authorisation platform for Single Sign-On.
an instance of HedgeDoc - https://md.fossdle.org - a collaborative editing environment for MarkDown content.
a Wekan instance - https://kanban.fossdle.org - a Kanban project planning and management board similar to Trello.
a (Bit|Vault)Warden - https://safe.fossdle.org - a full-featured password manager like LastPass (but secure) and 1Password, et al.
a NextCloud instance, with an integrated OnlyOffice instance - https://hub.fossdle.org - for file sharing and collaborative project work. Like Google Drive, Dropbox, or MS OneDrive.
to support our NextCloud, we have integrated an OnlyOffice instance. Together, they behave like a feature-rich Google Suite, but refreshingly Google-free - https://onlyoffice.oeru.org
a Discourse instance - https://forum.fossdle.org - Discourse is the market leading, modern, full-featured (and fully FOSS) forum.

In addition to these FOSSDLE-supporting services, we continue to maintain the following for the OERu and partners:

two more WordPress Multisites: our main course delivery site, where each course is a 'subsite' - https://course.oeru.org our train-the-educators blog and sandbox course site. Each educator gets a 'blog' subsite for their own use, and a 'course sandbox' subsite - https://edt4ol.oeru.org
a trio of standalone WordPress instances for the OERF and related initiatives: the OERF's own organisational website - https://oerfoundation.org the Centre for Open Educational Practice (COEP) - https://coep.nz the OERF's initiative to help educators in the developing world cope with the challenges of COVID19 - https://oer4covid.oeru.org
a pair of Drupal sites (they've been upgraded to Drupal 9.x) the OERF's information tech site (this site!) - https://tech.oeru.org, and our H5P learning object builder site - https://h5p.oeru.org
another Discourse instance - Discourse is the market leading, modern, full-featured (and fully FOSS) forum: our companion site for folks taking OERu courses for both assignments and collaboration with other learners - https://forums.oeru.org Note: we redeployed our underutilised our OERu community collaboration and support site for educators as the FOSSDLE forum above.
another NextCloud for file sharing and many other uses. It's our main collaborative document management system and collaborative authorship and sharing media and documents - https://docs.oeru.org
a BigBlueButton instance - a large scale real-time video conferencing and educational collaboration platform - https://talk.oeru.org
a Mautic high-powered mailing list automation management system - https://mautic.oeru.org
a Rocket.Chat instance - a rich chat service comparable to Slack or Discord but fully open source (and we hold all our own data) - https://chat.oeru.org - Rocket.Chat, though very good, is not nearly as powerful or secure as Matrix, so we have shifted most of our activities to Matrix and Element, which also offers more mature mobile apps.
another Mastodon instance - Twitter-esque but de-centralised and non-commercial federated open source social network (part of the Fediverse) - https://mastodon.oeru.org
a Mailcow instance for managing all our email - Mailcow is a full multi-tenanted SMTP/IMAP/POP email system with active spam filtering, user self-service, DKIM configuration/management, webmail, & much more - even virus scanning for Windows clients - https://about.oerfoundation.org - it supplies email services for the oeru.org, oerfoundation.org, coep.nz, and fossdle.org domains, among others!
an instance of YourLS - a link shortener - https://oer.nz
a Matomo instance - website analytics (like Google Analytics, but without infringing on the EU's GDPR legislation) - https://stats.oeru.org
we added a Plausible instance - another very nice FOSS website analytics system which uses different technologies and is similarly privacy-respecting to Matomo. It also serves to provide a check of our Matomo analytics - https://stats.fossdle.org
a Mahara instance - online academic portfolio tool - https://portfolio.oerfoundation.org
a Moodle instance - market leading learning management system which we use primarily for offering quizzes and awarding certificates for participants - https://moodle.oeru.org
a SilverStripe instance - our main OERu website - https://oeru.org
a MediaWiki instance - collaboration platform for OER development built on the same platform on which Wikipedia is built. All our courses are developed here - https://wikieducator.org
a GitLab - a software developer version control/collaboration tool where we make all our code available - https://git.oeru.org
a LimeSurvey instance - survey tool - https://survey.oeru.org
another (BitWarden/VaultWarden](https://github.com/dani-garcia/vaultwarden) instance - password manager and cross-device sync service - https://safe.oeru.org (here's our howto for hosting your own)
an instance of Semantic Scuttle - a collaborative public website bookmarking service - https://bookmarks.oeru.org
another Wekan instance - for project planning via the Kanban methodology. Similar to Trello (among other tools) - https://kanban.oeru.org.
a Mobilizon instance - for managing events, and people following, discussing, and signing-up for and attending them - https://events.oeru.org
a RustDesk server instance, allowing us to provide anyone anywhere on just about any computing platform (Linux, Windows, MacOS, iOS, and Android) with live interactive shared desktop support. Equivalent to TeamViewer.
a server monitoring solution based on Grafana, Prometheus, Node-exporter, Alertmanager, cAdvisor, and other tools, which provide a graphical monitoring solution for each of our servers.
we recently added an instance of Uptime Kuma, a superb system for periodically (i.e. every minute) checking that all of our services are doing what they ought to be. It alerts us via Matrix integration if any service is misbehaving based on a variety of parameters. It also provides a pretty dashboard with green, orange, and (occasionally) red lights. It also allows us to monitor each service's performance over time.

We also maintain development and testing/staging instances of most of these services. We do frequent (daily or, in some cases, hourly) database backups and daily remote incremental encrypted filesystem backups onto a server with a LOT of storage. Aside from one co-located physical server with large disk capacity, our services are all hosted on virtual Linux servers (we mostly run Ubuntu Linux) provided by commodity cloud hosting providers via Docker and orchestrated via Docker Compose.

Hosting on behalf

We continue to host services on behalf of other organisations, including Commonwealth of Learning, Open Education Global, and the government of Samoa, in particular their Ministry of Education, Sport, and Culture's Innovative Lifelong Learning Lab (the MiLLL).

For the MiLLL project:

a pair of WordPress Multisites, one for hosting open educational resource-based course sites - https://course.milll.ws - and one hosting individual sites for Samoan primary and secondary schools - https://schools.milll.ws
a BigBlueButton instance supporting learners, educators, and government staff's video conferencing requirements - https://bbb.milll.ws
a Mastodon instance especially for Samoans wanting to participate in digital social media - https://mastodon.milll.ws
an instance of Rocket.Chat - https://chat.milll.ws
an instance of Discourse - https://forum.milll.ws
an instance of Moodle - https://moodle.milll.ws
an instance of BitWardern/VaultWarden - https://safe.milll.ws

For the Commonwealth of Learning:

a WordPress Multisite for hosting OER-based course sites for the Pacific Partnership in Open Distance and Flexible Learning initiative - https://pacificopencourses.col.org

For Open Education Global:

a WordPress Multisite for hosting OER-based course sites - https://course.oeglobal.org

Costs and Usage

In the past year we have served tens of thousands of registered users and over 200,000 anonymous learners access our courses (the full content of which are open to all without requiring authentication).

It's also important to note that all of these services can be provided at no cost to our collaborators and learners as there are no per-seat license fees (nor any license fees) associated with any of the services. Our only costs are related to the 14 (at last count) generic 'cloud'-based Virtual Private Servers (VPSs) (all running a FOSS Linux operating system) we hire from a host of competing commodity hosting providers including Digital Ocean, Hetzner, and Linode. We actively avoid using any vendor-specific proprietary services so that we can shift providers with minimal hassle and service interruption if required.

Our entire annual IT infrastructure cost, including for our 'on behalf' hosting partners, is now hovering around USD10,000 per year for all of the above. What's more our usage monitoring suggests that most of our servers were operating well below 50% capacity, meaning that we have a plenty of additional headroom.

Add new comment

Join the Fediverse: installing Mastodon 4.0 on Ubuntu 22.04 with Docker Compose

dave — Thu, 03 Nov 2022 21:31:11 +0000

Join the Fediverse: installing Mastodon 4.0 on Ubuntu 22.04 with Docker Compose dave Fri 04/11/2022 - 10:31

In the past week or two, with Elon Musk completing his purchase and take-over of Twitter, there has been a torrent of defections from the centralised, closed, for-profit platform to its antithesis, the Fediverse - it's a portmanteau of "federation" and "universe". The Fediverse, in short, is a set of Free and Open Source Software (FOSS) applications created by different quite independent communities of developers, sharing one thing in common: the (somewhat magical) ActivityPub open standard for data characterisation and exchange. They are deployed independently by interested individuals and communities to form a 'decentralised network' of ActivityPub-enabled services (here's a quick explanation video). Of these many applications (with new ones popping up every day!), probably the most ambitious and widely known is Mastodon (which also features a short video explanation).

For those not (yet) familiar with it, Mastodon is a 'micro-blogging' platform superficially somewhat similar to Twitter, but different in many important details. It is also a collection of FOSS applications, notably an 'instance', aka a server (the subject of this how-to), and a variety of 'clients' that support people interacting with one or more instances. A user installs a client on their phone or desktop, or simply uses a web browser, opening the instance's web address (URL) in a tab. Either the tab or the client becomes their portal for engaging with the Mastodon instance and the various ActivityPub streams the user selects to see via that instance. That selection is done via 'following' other users on the same or different instances, whether those instances are also Mastodon or one of the other Fediverse applications (here's a fairly complete list). Users can further 'shape' their feeds by employing filters selecting or eliminating those containing specific 'hashtags' (a hashtag is just a word - or CamelCaseCollectionOfWords - in a post or 'toot' that has a '#' hash symbol at the front of it) and/or by blocking/muting specific other Fediverse participants whose contributions they prefer to avoid.

We, here at the OER Foundation, have been running Mastodon for 6 years now, but the sudden spike in interest and changes to the various dependent technologies since our previous Mastodon howto have motivated me to create this streamlined, updated version. We're very keen to see lots of individual academic institutions running Mastodon instances on behalf of their learners and broader educational communities! The cost is fairly trivial (USD30-50/month) for an instance with a thousand or two users, especially for an institution.

Tips for this tutorial
Create a Virtual Private Server
- VPS Properties:
Key variables for you Mastodon
- Get your Domain lined up
- Set up an unprivileged user for yourself
Configure the VPS
- Configuring your firewall
- Install the NGINX
Outgoing VPS Email
Docker Compose and Let's Encrypt
Install the Mastodon Docker recipe
NGINX reverse proxy configuration
Build your Mastodon
Create your Admin user
Basic Instance configuration
Other considerations

Tips for this tutorial

If this is your first attempt at 'self-hosting', and you do follow through, this could be the start of a new era in your technical status - you could realised that 'agency' you always wanted. Plus, I'll be very impressed by your esprit de corps and tenancity!

Create a Virtual Private Server

The first step is to create a place to host the Mastodon instance. You can run it on a local piece of hardware of sufficient capacity, but make sure you've got a fast and symmetrical (as fast to upload as to download!) connection. If (as with most residential Internet services) your upload is much slower than your download (often 1:10 ratio) your server is going to be very slow for external people, especially if streaming video. Also, don't undertake this unless you have a flat-rate data connection.

If you have trouble getting a VPS, you might find this video I created for provisioning a VPS, using Digital Ocean as an example. In my experience, the process for provisioning VPSs on other platforms is very similar. You'll find this process much easier than using either Microsoft Azure or Amazon AWS, which we do not recommend. Their systems are unnecessarily complex, proprietary (they will lock you in), and about 10 times more expensive than commodity hosting options.

VPS Properties:

We recommend that, for a Mastodon instance of modest size (say up to a few hundred active users) you provision a VPS with the following spec (go as high as you can afford to). You should be able to upgrade those specs in realtime if required, except for your disk space. You can, however, provision a secondary storage space (you can start small and increase it as you need to). I will cover setting this up, as it'll make your life far far easier in the medium-long term.

8-16 GB RAM (a small instance can be run on 4GB RAM, so you could start with that, as RAM can be upgraded easily)
2-4 Virtual CPUs
80-160 GB Disk space (NVME disk is faster than SSD which is faster than spinning disk space)
running Ubuntu Linux 22.04 (the current Long Term Support version)
extra storage - 20-40GB extra space (can be expanded on fairly short notice)

You'll need to create an account for yourself on your chosen hosting provider (it's a good idea to use Two Factor Authentication, aka 2FA, on your hosting account so that no one can log in as you and, say, delete your server unexpectedly - you'll find instructions on how to set up 2FA on your hosting provider's site) and create an Ubuntu @2.04 (or the most recent 'Long Term Support' (LTS) version) - 24.04 is likely to come out in April 2024) in the 'zone' nearest to you (or your primary audience, if that's different).

You'll need to note the server's IPv4 address (it'll be a series of 4 numbers, 0-254, separated by full stops, e.g. 103.99.72.244), and you should also be aware that your server will have a newer IPv6 address, which will be a set of 8 four hex character values (each hex character can have one of 16 values: 0-9,A-F) separated by colons, e.g. 2604:A880:0002:00D0:0000:0000:20DE:9001. With one or the other of those IPs, you should be able to log into your new server via SSH. If you're on a UNIX command line (e.g. a Linux or MacOS desktop), do this in a terminal. On Windows, I understand people use a tool called Putty for SSH, in which case follow the app's instructions.

ssh [your server IPv4 or IPv6]

followed by the ENTER key (that'll be true for any line of commands I provide).

whoami

If it returns root (there's also a convention of using a '#' as the command prompt), you're the root or super-admin of the server. If not, you're a normal user (some hosting providers have a convention of giving you a default user of "ubuntu" user perhaps "debian") with a prompt that is, by convention a '$'.

Now that you're logged in, it's worth doing an upgrade of your server's Ubuntu system! Do that as follows:

sudo apt-get update && sudo apt-get dist-upgrade

Usually the user, even if it's not the root user, will have the ability to use the sudo command modifier - that means "do this action as the root (aka the 'Super User', thus 'su' for short) user" - you may be asked to enter your password as a security precaution the first time you run a command prefaced by sudo. Enter it, and it should run the command. Plus, the system shouldn't bother you for it again unless you leave your terminal unused for a while and come back to it.

At this point, I also like to install some cool software called 'etckeeper' which records configuration changes on your VPS for future reference (and recovering from administrative mess-ups):

sudo apt-get install etckeeper

which will also install some dependencies, including the very important (and relevant later on) 'git' version control system.

Key variables for you Mastodon

To set up you Mastodon, you'll need a few crucial bits of information related to your system's identity and external systems you'll need it to interact with. For example, as mentioned before, you'll need a domain name. For the rest of this tutorial, we'll use the convention of representing those variables as a name inside [], or, for the domain name you've picked, [domain name].

Here's a list of variables you'll need to know to complete the rest of this tutorial:

[ipv4] and [ipv6] - your VPS' IPv4 and IPv6 addresses (the latter can be ignored if your cloud provider doesn't support IPv6 addresses) as described above.
[domain name] - the fully qualified domain name or subdomain by which you want your instance to be accessed. You must have full domain management ability on this domain. Example: mastodon.oeru.org - that's the mastodon subdomain of the oeru.org domain. Authenticating SMTP details - this is required so your web service can send emails to users - crucial things like email address validation and password recovery emails...
[smtp server] - the domain name or IPv4 or IPv6 address of an SMTP server
[smtp port] - the port number on the server that is listening for your connection. By convention it's likely to be 465 or 587, or possibly 25.
[smtp reply-to-email] - a monitored email to which people can send email related to this WordPress site, e.g. notifications@[domain name]
[smtp user] - the username (often an email address) used to authenticate against your SMTP server, provided by your email provider.
[smtp password] - the accompanying password, provided by your email provider.
[your email] - an email address to which system-related emails can be sent to you, perhaps something like webmaster@[domain name].
[vps username] - the username you use on your server (by convention, these are one word, and all lower case).

Get your Domain lined up

Set up an unprivileged user for yourself

This will log you into your server as it did the first time, either as 'root' or the default unprivileged user. It's not considered good practice to access your server as root (it's too easy to completely screw it up by accident). Either way, best practice is to create your own separate 'non-root' user who has 'sudo' privileges and the ability to log in via SSH. If you are currently logged in as 'root', you can create a normal user for yourself via (replace [vps username] with your chosen username - in my case, I'd use U=dave):

U=[vps username]
adduser $U
adduser $U ssh
adduser $U admin
adduser $U sudo

You'll also want to a set a password for user [vps username] (we have a tutorial on creating good passwords):

passwd $U

su $U
ssh-keygen -t rsa -b 2048
nano ~/.ssh/authorized_keys

and in that file, copy and paste (without spaces on either end) your current computer's public ssh key (never publish your private key anywhere!), save and close the file.

From that point, you should be able to SSH to your server via ssh [vps username]@[domain name] without needing to enter a password.

Configure the VPS

sudo dpkg-reconfigure tzdata

and pick the appropriate timezone. You can just leave it running UTC, but you might find it tricky down the track if you're looking at logs and having to constantly convert the times into your timezone.

EDIT=`which nano`

or, if you're like me

EDIT=`which vim`

so that subsequent references to $EDIT will invoke your preferred editor. Note the command which nano is a script which finds the full path to the named command, in this case 'nano'. Putting it inside the backquotes `` means 'replace with the value the script returns', so it sets the value of EDIT to the path of the nano command.

Configuring your firewall

Let's configure our firewall. We work on the basis of allowing in only what we want to let in (i.e. 'default deny').

First we'll enable the use of SSH through the firewall (not doing this could lock us out of your machine!)

sudo ufw allow ssh

while we're here, we'll also enable data transfer from the internal (to the VPS) Docker virtual network and the IP range it uses for Docker containers:

sudo ufw allow in on docker0
sudo ufw allow from 172.0.0.0/8 to any

Then we'll enable forwarding from internal network interfaces as required for Docker containers to be able to talk to the outside world:

sudo $EDIT /etc/default/ufw

and copy the line DEFAULT_FORWARD_POLICY="DROP" tweak it to look like this (commenting out the default, but leaving it there for future reference!):

#DEFAULT_FORWARD_POLICY="DROP"
DEFAULT_FORWARD_POLICY="ACCEPT"

and then save and exit the file (CTRL-X and then 'Y' if your editor is nano).

You also have to edit /etc/ufw/sysctl.conf and remove the "#" at the start of the following lines, so they look like this:

sudo $EDIT /etc/ufw/sysctl.conf

# Uncomment this to allow this host to route packets between interfaces
net/ipv4/ip_forward=1
net/ipv6/conf/default/forwarding=1
net/ipv6/conf/all/forwarding=1

Then we need to restart the network stack to apply that configuration change

sudo systemctl restart systemd-networkd

Next we have to enable the UFW firewall to start at boot time.

sudo $EDIT /etc/ufw/ufw.conf

And set the ENABLED variable near the top:

ENABLED=yes

Now you can formally start UFW now:

sudo ufw enable

Install the NGINX

Next we need to install the NGINX webserver and reverse-proxy, as well as the Let's Encrypt SSL certificate generator, both of which are crucial for any secure webservices you might want to host. NGINX is a more efficient and flexible alternative to the older Apache webserver you might've seen elsewhere.

sudo apt-get install nginx-full letsencrypt

You'll get a couple pop-up windows in your terminal, just hit ENTER to accept the defaults. Having installed it, we need to create firewall rules to allow external services to see it:

sudo ufw allow 'Nginx Full'

You can check if the firewall rules you requested have been enabled:

sudo ufw status

Outgoing VPS Email

It's very useful for your server to be able to send out emails, like status emails to administrators (perhaps you) about things requiring their attention, e.g. the status of backups, pending security updates, expiring SSL certificates, etc. To do this, we'll set up the industrial strength Postfix SMTP server, which is pretty quick and easy. First we install Postfix.

sudo apt-get install postfix bsd-mailx

During the install, you'll be asked to select a bunch of configuration parameters. Select the defaults except:

Select "Internet Site with Smarthost",
fill in the domain name for your server [domain name],
the [smtp server] name and [smtp port] (in the form [smtp server]:[smtp port], e.g. smtp.oeru.org:587 ) of your "smarthost" who'll be doing the authenticating SMTP for you, and
the email address to which you want to receive system-related messages, [your email].

After that's done, we set a default address for the server to mail to, to [your email] selected above. First

sudo $EDIT /etc/aliases

We need to make sure the "root" user points to a real email address. Add a line at the bottom which says (replacing [your email] with your email :) )

root: [your email]

After which you'll need to convert the aliases file into a form that postfix can process, simply by running this:

sudo newaliases

Then we have to define the authentication credentials required to convince your mail server that you're you!

sudo $EDIT /etc/postfix/relay_password

and enter a single line in this format:

[smtp server] [smtp user]:[smtp password]

smtp.oerfoundation.org smtp-work@fossdle.org:SomeObscurePassw0rd

then save the file and, like the aliases file, run the conversion process (which uses a slightly different mechanism):

sudo postmap /etc/postfix/relay_password

Finally, we'll edit the main configuration file for Postfix to tell it about all this stuff:

sudo $EDIT /etc/postfix/main.cf

relayhost = [smtp server]:[smtp port]

or, for example:

relayhost = smtp.oerfoundation.org:465

sudo mv /etc/postfix/main.cf /etc/postfix/main.cf.orig && sudo $EDIT /etc/postfix/main.cf

You can just copy-and-paste the following into it, substituting your specific values for the [tokens].

# See /usr/share/postfix/main.cf.dist for a commented, more complete version
 
# Debian specific:  Specifying a file name will cause the first
# line of that file to be used as the name.  The Debian default
# is /etc/mailname.
#myorigin = /etc/mailname
 
smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu)
biff = no
 
# appending .domain is the MUA's job.
append_dot_mydomain = no
 
# Uncomment the next line to generate "delayed mail" warnings
#delay_warning_time = 4h
 
readme_directory = no
 
# See http://www.postfix.org/COMPATIBILITY_README.html -- default to 3.6 on
# fresh installs.
compatibility_level = 3.6
 
# TLS parameters
smtpd_tls_cert_file=/etc/ssl/certs/ssl-cert-snakeoil.pem
smtpd_tls_key_file=/etc/ssl/private/ssl-cert-snakeoil.key
smtpd_tls_security_level=may
smtp_tls_CApath=/etc/ssl/certs
#smtp_tls_security_level=may
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache
smtpd_relay_restrictions = permit_mynetworks permit_sasl_authenticated defer_unauth_destination
myhostname = [domain name]
alias_maps = hash:/etc/aliases
alias_database = hash:/etc/aliases
myorigin = /etc/mailname
mydestination = $myhostname, work.fossdle.org, localhost.fossdle.org, localhost
relayhost = [smtp server]:[smtp port]
mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128
mailbox_size_limit = 0
recipient_delimiter = +
inet_interfaces = all
inet_protocols = all
 
# added to configure accessing the relay host via authenticating SMTP
smtp_sasl_auth_enable = yes
smtp_sasl_password_maps = hash:/etc/postfix/relay_password
smtp_sasl_security_options = noanonymous
smtp_tls_security_level = encrypt
 
# if you're using Ubuntu prior to 20.04, uncomment (remove the #) the
# earlier line smtp_tls_security_level = may to save errors in 'postfix check'
# and comment this line (by adding a # at the start)
smtp_tls_wrappermode = yes

Once you've created that main.cf file, you can double check that your config is valid:

sudo postfix check

and if it's all ok, you can get Postfix to re-read its configuration:

sudo postfix reload

You can then try sending an email so see if it works!

$ mail you@email.domain<ENTER>

Subject: Testing from your.relay.server.domain<ENTER>
Testing postfix remote host<ENTER>
<CTRL-D>
Cc:<ENTER>

You can also always check the postfix system logs to see what postfix thinks about it using the command:

sudo less +G /var/log/mail.log

if your system doesn't have a /var/log/mail.log, never fear! Try this instead:

sudo less +G /var/log/syslog

In either case, hit to have the log update in real time.

Docker Compose and Let's Encrypt

sudo apt install docker-compose letsencrypt

Now we create the set of directories I use for holding Docker Compose configurations (/home/docker) and the persistent data the Docker containers create (/home/data)

D=[domain name]
sudo mkdir -p /home/data/$D
sudo mkdir -p /home/docker/$D

It's helpful to make sure that your non-root user can also read and write files in these directories:

U=[vps username]
sudo chown -R $U /home/docker
sudo chown -R $U /home/data

Install the Mastodon Docker recipe

Now we have a place to put the really key bit - the code for running Mastodon via Docker Compose - first we go to the right place to set things up:

cd /home/docker

and then we use an amazing tool called 'git' to download the canonical code created by the Mastodon developers. Of course, replace [domain name] in the following with the one you've selected (for the record, this command should work without sudo because we changed the ownership of the director in the previous step to belong to your unprivileged user).

git clone https://github.com/tootsuite/mastodon.git [domain name]

Then you can go into the directory

cd [domain name]

Have a look around. Use ls -la to see what files and directories (including 'hidden' ones, whose names start with a '.') the project includes. They key files in this case are the .env.production.sample and docker-compose.yml. You can ignore the rest for the time being.

The first thing we want to do is create a .env.production file - this will hold all the crucial details of your Mastodon instance. Do it like this:

cp .env.production.sample .env.production

and then edit it (if EDIT isn't defined, because you're in a different session now, you might need to reset it as described above):

$EDIT .env.production

The following is what I've got (replace the [tokens] as usual). Make sure you get your SMTP details right - if you don't, you can fix them, but you won't get any emails from the instance until you do! Note that there're five (5) special numbers that you'll need to set later - but don't worry about them for the moment.

# This is a sample configuration file. You can generate your configuration
# with the `rake mastodon:setup` interactive setup wizard, but to customize
# your setup even further, you'll need to edit it manually. This sample does
# not demonstrate all available configuration options. Please look at
# https://docs.joinmastodon.org/admin/config/ for the full documentation.
 
# Note that this file accepts slightly different syntax depending on whether
# you are using `docker-compose` or not. In particular, if you use
# `docker-compose`, the value of each declared variable will be taken verbatim,
# including surrounding quotes.
# See: https://github.com/mastodon/mastodon/issues/16895
 
# Federation
# ----------
# This identifies your server and cannot be changed safely later
# ----------
LOCAL_DOMAIN=[domain name]
LOCAL_HTTPS=true
ALTERNATE_DOMAINS=[second domain name]
 
# Redis
# -----
REDIS_HOST=redis
REDIS_PORT=6379
 
# PostgreSQL
# ----------
DB_HOST=db
DB_USER=postgres
DB_NAME=postgres
DB_PASS=
DB_PORT=5432
 
# Elasticsearch (optional)
# ------------------------
ES_ENABLED=false
#ES_HOST=localhost
#ES_PORT=9200
# Authentication for ES (optional)
#ES_USER=elastic
#ES_PASS=password
 
# Secrets
# -------
# Make sure to use `rake secret` to generate secrets
# -------
PAPERCLIP_SECRET=LongSecretNumberOne
SECRET_KEY_BASE=LongSecretNumberTwo
OTP_SECRET=LongSecretNumberThree
 
# Web Push
# --------
# Generate with `rake mastodon:webpush:generate_vapid_key`
# --------
#VAPID_PRIVATE_KEY=
#VAPID_PUBLIC_KEY=
VAPID_PRIVATE_KEY=QuiteALotShorterSecretNumberOne
VAPID_PUBLIC_KEY=SlightlyShorterSecretNumberTwo
 
 
# Sending mail
# ------------
SMTP_SERVER=[smtp server]
SMTP_PORT=[smtp port]
SMTP_LOGIN=[smtp user]
SMTP_PASSWORD=[smtp password]
SMTP_FROM_ADDRESS=[smtp reply-to-email]
 
# File storage (optional)
# -----------------------
# #S3_ENABLED=true
#S3_BUCKET=files.example.com
#AWS_ACCESS_KEY_ID=
#AWS_SECRET_ACCESS_KEY=
#S3_ALIAS_HOST=files.example.com

Once you've save that, we can create the docker-compose.yml file. I recommend creating a backup version of the original file in case you want to refer back to it:

cp docker-compose.yml docker-compose.yml.orig

Then you can edit the file via

$EDIT docker-compose.yml

Replace any [tokens] with your values. Note that the version of Mastodon at the time of this writing is 4.0.1 - to use a different version (e.g. if you are reading this tutorial after Eugen releases Mastodon 4.x (see the current release) you'll just want to replace all occurrences of 4.0.1 below with the appropriate version number.

Note, the 'build' commands are commented out here as we're using pre-built containers rather than building them locally (as that takes a lot of time and potentially requires a fair bit of disk space, at least temporarily). If you want to do a build, you'll want to uncomment those lines.

version: '3'
services:
  db:
    restart: unless-stopped
    image: postgres:14-alpine
    shm_size: 256mb
    networks:
      - internal_network
    healthcheck:
      test: ['CMD', 'pg_isready', '-U', 'postgres']
    volumes:
      - /home/data/[domain name]/postgres14:/var/lib/postgresql/data
    environment:
      - 'POSTGRES_HOST_AUTH_METHOD=trust'
 
  redis:
    restart: unless-stopped
    image: redis:6-alpine
    networks:
      - internal_network
    healthcheck:
      test: ['CMD', 'redis-cli', 'ping']
    volumes:
      - /home/data/[domain name]/redis:/data
 
## this is commented out because most Mastodon admins don't use it.
## that's because the disk space it uses grows quickly and without bound...
# es:
#    restart: unless-stopped
#    image: docker.elastic.co/elasticsearch/elasticsearch-oss:7.10.2
#    environment:
#      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
#      - "cluster.name=es-mastodon"
#      - "discovery.type=single-node"
#      - "bootstrap.memory_lock=true"
#    networks:
#      - internal_network
#    healthcheck:
#      test: ["CMD-SHELL", "curl --silent --fail localhost:9200/_cluster/health || exit 1"]
#    volumes:
#      - /home/data/[domain name]/elasticsearch:/usr/share/elasticsearch/data
#    ulimits:
#      memlock:
#        soft: -1
#        hard: -1
 
  web:
    #build: .
    image: tootsuite/mastodon:v4.0.1
    restart: unless-stopped
    env_file: .env.production
    command: bash -c "rm -f /mastodon/tmp/pids/server.pid; bundle exec rails s -p 3000"
    networks:
      - external_network
      - internal_network
    healthcheck:
      # prettier-ignore
      test: ['CMD-SHELL', 'wget -q --spider --proxy=off localhost:3000/health || exit 1']
    ports:
      - '127.0.0.1:3000:3000'
    depends_on:
      - db
      - redis
    volumes:
      - /home/data/[domain name]/public/system:/mastodon/public/system
 
  streaming:
    #build: .
    image: tootsuite/mastodon:v4.0.1
    restart: unless-stopped
    env_file: .env.production
    command: node ./streaming
    networks:
      - external_network
      - internal_network
    healthcheck:
      # prettier-ignore
      test: ['CMD-SHELL', 'wget -q --spider --proxy=off localhost:4000/api/v1/streaming/health || exit 1']
    ports:
      - '127.0.0.1:4000:4000'
    depends_on:
      - db
      - redis
 
  sidekiq:
    #build: .
    image: tootsuite/mastodon:v4.0.1
    restart: unless-stopped
    env_file: .env.production
    command: bundle exec sidekiq
    depends_on:
      - db
      - redis
    networks:
      - external_network
      - internal_network
    volumes:
      - /home/data/[domain name]/public/system:/mastodon/public/system
    healthcheck:
      test: ['CMD-SHELL', "ps aux | grep '[s]idekiq\ 6' || false"]
 
## If you need more workers uncomment the following. Add up to 3 more sidekiq containers (up to 5 total)
#  sidekiq2:
#    #build: .
#    image: tootsuite/mastodon:v4.0.1
#    restart: unless-stopped
#    env_file: .env.production
#    command: bundle exec sidekiq
#    depends_on:
#      - db
#      - redis
#    networks:
#      - external_network
#      - internal_network
#    volumes:
#      - /home/data/[domain name]/public/system:/mastodon/public/system
#    healthcheck:
#      test: ['CMD-SHELL', "ps aux | grep '[s]idekiq\ 6' || false"]
 
networks:
  external_network:
  internal_network:
    internal: true

Note: the port numbers in the docker-compose.yml file need to match those in the NGINX reverse proxy configuration.

Once you've got this file configured, you should be ready to 'pull' your Docker containers! To do that, run

docker-compose pull

Once that's finished, you'll need to get three secret numbers: SECRET_KEY_BASE, OTP_SECRET, and PAPERCLIP_SECRET. Run the following three times and record the numbers (doesn't matter which one goes where, but once you set them, you don't want to change or lose them!):

docker-compose run --rm web bundle exec rake secret

Then you need to get the two VAPID keys - running this will provide both:

docker-compose run --rm web bundle exec rake mastodon:webpush:generate_vapid_key

Copy these into your .env.production - I encourage you to record them elsewhere as well - a password manager is a good place!

$EDIT .env.production

After that, your Mastodon is ready to roll! But there's one more crucial step - we need to set up the reverse proxy server which provides the secure (encrypted) access to your instant for you and all your users.

NGINX reverse proxy configuration

For the reverse proxy, we'll use the NGINX web server we installed earlier. It stores all of its configuration in the directoy /etc/nginx.

The first thing we'll do is create a couple directories we need:

This is where Let's Encrypt will look for a secret code we put here to verify that we own the domain we're requesting an SSL certificate for:

sudo mkdir /var/www/letsencrypt

Then we create a place to the Let's Encrypt NGINX configuration details

sudo mkdir /etc/nginx/includes

And then we create that configuration file

sudo $EDIT /etc/nginx/includes/letsencrypt.conf

into which we copy-and-paste the following (no [tokens] to replace in this one!)

# Rule for legitimate ACME Challenge requests
location ^~ /.well-known/acme-challenge/ {
    default_type "text/plain";
    # this can be any directory, but this name keeps it clear
    root /var/www/letsencrypt;
}
 
# Hide /acme-challenge subdirectory and return 404 on all requests.
# It is somewhat more secure than letting Nginx return 403.
# Ending slash is important!
location = /.well-known/acme-challenge/ {
    return 404;
}

Next we have to create the reverse proxy configuration file for our Mastodon domain name (and any other domain names we might want to use):

sudo $EDIT /etc/nginx/sites-available/[domain name]

Again, we have the convention of using the [domain name] to identify this file. It's self-documenting. Copy-and-paste the following into, making [token] substitutions. Note that the port numbers (usually after a ':') need to correspond to the port numbers specified in the docker-compose.yml file.

map $http_upgrade $connection_upgrade {
  default upgrade;
  '' close;
}
 
proxy_cache_path /var/cache/nginx levels=1:2 keys_zone=CACHE:10m inactive=7d max_size=1g;
 
# this configuration redirects any attempts to connect to your instance insecurely (via http rather than https)
# to your preferred [domain name] via https.
server {
    listen 80;
    listen [::]:80;
    # you can optionally include one or more other domains (e.g. [second domain name]) here
    # - just separate by spaces.
    server_name [domain name];
    root /var/www/html;
 
    include includes/letsencrypt.conf;
 
    # change the file name of these logs to include your server name
    # if hosting many services...
    access_log /var/log/nginx/[domain name]_access.log;
    error_log /var/log/nginx/[domain name]_error.log;
 
    # redirect all HTTP traffic to HTTPS.
    location / {
        return 302 https://[domain name]$request_uri;
    }
}
 
## optional - if you want your Mastodon to respond to a [second domain name]
## this will perform a redirection from this domain to your main domain.
##
## to use it, uncomment the first # in each line following
#server {
#    listen 443 ssl http2;
#    listen [::]:443 ssl http2;
#    server_name [second domain name];
#
#    ssl_certificate /etc/ssl/certs/ssl-cert-snakeoil.pem;
#    ssl_certificate_key /etc/ssl/private/ssl-cert-snakeoil.key;
##    ssl_certificate /etc/letsencrypt/live/[domain name]/fullchain.pem;
##    ssl_certificate_key /etc/letsencrypt/live/[domain name]/privkey.pem;
#    ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
#    # from https://0x39b.fr/post/nginx_security/
#    ssl_session_timeout 1d;
#    ssl_session_cache shared:SSL:50m;
#    #ssl_session_tickets off;
#    ssl_prefer_server_ciphers on;
#    ssl_ciphers 'ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256';
#    # OCSP Stapling ---
#    # fetch OCSP records from URL in ssl_certificate and cache them
#    ssl_stapling on;
#    ssl_stapling_verify on;
#    # to create this, see https://raymii.org/s/tutorials/Strong_SSL_Security_On_nginx.html
#    ssl_dhparam /etc/ssl/certs/dhparam.pem;
#
#    # for let's encrypt renewals!
#    include includes/letsencrypt.conf;
#
#    keepalive_timeout    70;
#    sendfile             on;
#    client_max_body_size 80M;
#
#    # change the file name of these logs to include your server name
#    # if hosting many services...
#    access_log /var/log/nginx/[domain name]_access.log;
#    error_log /var/log/nginx/[domain name]_error.log;
#
#    # redirect all HTTP traffic to HTTPS.
#    location / {
#        return 302 https://[domain name]$request_uri;
#    }
#}
## end optional *second* domain name
 
server {
    listen 443 ssl http2;
    listen [::]:443 ssl http2;
    # if you have a [second domain]
    server_name [domain name];
 
    ssl_certificate /etc/ssl/certs/ssl-cert-snakeoil.pem;
    ssl_certificate_key /etc/ssl/private/ssl-cert-snakeoil.key;
#    ssl_certificate /etc/letsencrypt/live/[domain name]/fullchain.pem;
#    ssl_certificate_key /etc/letsencrypt/live/[domain name]/privkey.pem;
    ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
    # from https://0x39b.fr/post/nginx_security/
    ssl_session_timeout 1d;
    ssl_session_cache shared:SSL:50m;
    #ssl_session_tickets off;
    ssl_prefer_server_ciphers on;
    ssl_ciphers 'ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256';
    # OCSP Stapling ---
    # fetch OCSP records from URL in ssl_certificate and cache them
    ssl_stapling on;
    ssl_stapling_verify on;
    # to create this, see https://raymii.org/s/tutorials/Strong_SSL_Security_On_nginx.html
    ssl_dhparam /etc/ssl/certs/dhparam.pem;
 
    # for let's encrypt renewals!
    include includes/letsencrypt.conf;
 
    keepalive_timeout    70;
    sendfile             on;
    client_max_body_size 80M;
 
    # change the file name of these logs to include your server name
    # if hosting many services...
    access_log /var/log/nginx/[domain name]_access.log;
    error_log /var/log/nginx/[domain name]_error.log;
 
    # from https://github.com/tootsuite/documentation/blob/master/Running-Mastodon/Production-guide.md
    gzip on;
    gzip_vary on;
    gzip_proxied any;
    gzip_comp_level 6;
    gzip_buffers 16 8k;
    gzip_http_version 1.1;
    gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript;
 
 
    add_header Strict-Transport-Security "max-age=31536000; includeSubDomains";
 
    location / {
        try_files $uri @proxy;
    }
 
    location ~ ^/(emoji|packs|system/accounts/avatars|system/media_attachments/files) {
        add_header Cache-Control "public, max-age=31536000, immutable";
        add_header Strict-Transport-Security "max-age=31536000";
        try_files $uri @proxy;
    }
 
    location /sw.js {
        add_header Cache-Control "public, max-age=0";
        add_header Strict-Transport-Security "max-age=31536000";
        try_files $uri @proxy;
    }
 
    location @proxy {
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto https;
        proxy_set_header Proxy "";
        proxy_pass_header Server;
 
        proxy_pass http://127.0.0.1:3000;
        proxy_buffering off;
        proxy_redirect off;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection $connection_upgrade;
 
        tcp_nodelay on;
    }
 
    location /api/v1/streaming {
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto https;
        proxy_set_header Proxy "";
 
        proxy_pass http://127.0.0.1:4000;
        proxy_buffering off;
        proxy_redirect off;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection $connection_upgrade;
 
        tcp_nodelay on;
    }
 
    error_page 500 501 502 503 504 /500.html;
}

We initially comment out the SSL certificate paths for our [domain name] in the above configuration because we haven't yet created those certificates, and NGINX won't run with missing certificates (a trap for young players!). So we temprarily substitute the default (not domain-specific) "snakeoil" certificates that are provided with every Linux installation to act as valid certificate placeholders to run NGINX so that we can then request Let's Encrypt certificates.

After we've save the NGINX configuration, we need to make sure it's also in the sites-enabled directory, so NGINX will see it (it ignores those merely in sites-available, which is sort of a holding pen for potential site configurations):

sudo ln -sf /etc/nginx/sites-available/[domain name] /etc/nginx/sites-enabled/

before we can test our NGINX confguration for errors, we need to address the file, /etc/ssl/certs/dhparam.pem, our config file references but which doesn't yet exist by creating it like this:

sudo openssl dhparam -out /etc/ssl/certs/dhparam.pem 4096

Note this process can take quite a long time, like 10-40 minutes depending on your VPS and the 'entropy' it generates. If you're short of time, cancel the one running (CTRL+C) and run it again with 2048 rather than 4096 specified. It'll make your installation marginally less secure.

after which we can ask NGINX to test its configuration for errors:

sudo nginx -t

Fix any errors you might find (e.g. typos, missing punctuation, etc.) and after nginx -t tells you you've got valid configurations, reload NGINX to enable the new configuration:

sudo service nginx reload

Once that's working, your server is configured to respond to external requests for you site via [domain name] via http:// (not encrypted), and via https:// (encrypted, although pointing your browser at https://[domain name] right now will get your warnings that you've got a mismatch between your certificate and your domain name.). So, of course, the next step is to generate a certificate for [domain name].

Let's Encrypt can award a certificate to you because it can confirm that you (who control [domain name]) also control the server. They verify it by pointing their infrastructure at your [domain name] - recalling that your domain is pointed at your VPS' IPv4 (the A record above) and (if used) IPv6 (the AAAA record above) - and checking in pre-defined location (see the letsencrypt command below) to see if, at that location they can find a secret number that you've asked them to write there. If they find it, they can trust that you control both the [domain name] and the VPS it's pointing to.

Here's the command that will request Let's Encrypt's systems to run that check - it will verify that all the domain names specified (with a -d flag are a) pointing at your VPS, and b) their scripts can see the secret number the specified location, in this case in /var/www/letsencrypt):

letsencrypt certonly --webroot -w /var/www/letsencrypt -d [domain name] -d [second domain name]

Saving debug log to /var/log/letsencrypt/letsencrypt.log
Enter email address (used for urgent renewal and security notices)
 (Enter 'c' to cancel): webmaster@fossdle.org
 
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Please read the Terms of Service at
https://letsencrypt.org/documents/LE-SA-v1.3-September-21-2022.pdf. You must
agree in order to register with the ACME server. Do you agree?
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
(Y)es/(N)o: y
 
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Would you be willing, once your first certificate is successfully issued, to
share your email address with the Electronic Frontier Foundation, a founding
partner of the Let's Encrypt project and the non-profit organization that
develops Certbot? We'd like to send you email about our work encrypting the web,
EFF news, campaigns, and ways to support digital freedom.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
(Y)es/(N)o: y
Account registered.
Requesting a certificate for [domain name] and [second domain name]
 
Successfully received certificate.
Certificate is saved at: /etc/letsencrypt/live/[domain name]/fullchain.pem
Key is saved at:         /etc/letsencrypt/live/[domain name]/privkey.pem
This certificate expires on 2023-01-31.
These files will be updated when the certificate renews.
Certbot has set up a scheduled task to automatically renew this certificate in the background.
 
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
If you like Certbot, please consider supporting our work by:
 * Donating to ISRG / Let's Encrypt:   https://letsencrypt.org/donate
 * Donating to EFF:                    https://eff.org/donate-le
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Once you have a Let's Encrypt certificate, you can update our NGINX configuration:

sudo $EDIT /etc/nginx/sites-available/[domain name]

and swap all occurrences of

    ssl_certificate /etc/ssl/certs/ssl-cert-snakeoil.pem;
    ssl_certificate_key /etc/ssl/private/ssl-cert-snakeoil.key;
#    ssl_certificate /etc/letsencrypt/live/[domain name]/fullchain.pem;
#    ssl_certificate_key /etc/letsencrypt/live/[domain name]/privkey.pem;

#    ssl_certificate /etc/ssl/certs/ssl-cert-snakeoil.pem;
#    ssl_certificate_key /etc/ssl/private/ssl-cert-snakeoil.key;
    ssl_certificate /etc/letsencrypt/live/[domain name]/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/[domain name]/privkey.pem;

which enables your new domain-specific SSL certificate. Check that NGINX is happy with your change:

sudo nginx -t

and if so,

sudo service nginx reload

You domain should now be fully enabled for https:// access. Note that going to http://[domain name] should automatically redirect you to https://[domain name] because you care about your user's security! :grin:

We're in the final stretch now!

Build your Mastodon

To actually launch your Mastodon instance you first have to create the structure for database in the PostreSQL container. You can do that Virtual

docker-compose run --rm web rails db:migrate

This will temporarily spin up your 'web' container (which, in turn depends on your PostgreSQL container, aka 'db') and run the 'migration' script which either updates (or creates, if it's not already there) your database tables.

Ok - it's finally time to launch your Mastodon instance. Running this will fire up all of your containers using the values in your .env.production file.

docker-compose up -d && docker-compose logs -f --tail=100

After it initiates the containers with the up -d command, it opens a logging interface which allow you to watch the log messages from all the containers (only including the most recent 100 lines at the time you run it), which is very helpful in the event that something goes wrong - you should be able to see what it is.

You can always stop your Mastodon instance via

docker-compose stop or individual containers via, for example, the 'sidekiq' container, docker-compose stop sidekiq.

Now your Mastodon should be running. You can point your browser at https://[domain name] and you should be greeted by a sight similar to the first screenshot above (although it'll have your site's [domain name] rather than social.fossdle,org, which is the site I recently deployed and was the basis for setting up this tutorial). Your login page will just have generic information as it hasn't been configured yet, and doesn't have any users, administrative or otherwise.

If your site is missing the "mastodon" image below the Log In form, it might be that your 'assets' haven't been pre-compiled. If that's the case, just run this:

docker-compose run --rm web rails assets:precompile

which will re-compile them.

Create your Admin user

What you now need to do is create a user with [mastodon username] for the administrator of the site. That can be your own user or a dedicated admin user. It's up to you. I tend to make my personal user the admin user. You'll just follow the instructions on the start page to create a new account. All going well, you'll receive an email from the site asking you to verify your email address, and then you'll get a welcome email. At this point, you can log in, although you'll just be a generic user, not an admin. To change your user into an admin user, you'll need to run

docker-compose exec -e RAILS_ENV=production streaming bin/tootctl accounts modify [mastodon username] --role admin

If it's successful, you should see

OK

At that point, you can do a refresh of your logged in session in your Mastodon instance, and you'll be the Administrator! Congratulations! You've done it.

Now, you'll just want to do some basic configuration of your instance... and then you can let others know about it!

Basic Instance configuration

Once you've got your admin user working, you'll want to go to the "settings" for your user (the little 'gear' near the bottom right of the right column in the default interface, or at the top right of the left column of the advanced interface) and from the resulting menu, you should be able to select the 'Administration' option. The first thing to do is to go to 'Settings' and fill in as much as you need to - I've included a screen shot of our settings page on social.fossdle.org for your reference. You should also set some 'instance rules' - I've included another screenshot showing our instance rules - you can always check them out for yourself - you're welcome to borrow (any|every)thing from them.

After all that's set up, you're ready to go! You can start following other users - one useful trick to know about is that you can go to any other instances you know about via their web address (or go to the main Mastodon help site and look for other instances) and you can explore the 'profile directory' on any other instance. Clicking on a user's name (bold text) will pop up a box in your browser allowing you to elect to follow that user by typing in your Mastodon handle - it'll be [mastodon username]@[domain name] - in the provided form, and then selecting the 'Follow' button. Alternatively, you can copy-and-paste a user handle (below a user's name in their profile directory) into the Search bar in your Mastodon interface - it should show you the same profile info in your own Mastodon's interface window, and you should be able to follow with a single click on the "Follow" button or the little 'follow user' icon (greyed out head with a + after it). If you're already following a user, you'll see a blue 'user being followed' icon (with a blue x next to it) that, if clicked, will unfollow that user.

Other considerations

Ok, you're in business, but there're a few loose ends to tidy up (after the well-earned euphoria of your new found power and place die down a bit!). These are things that any production web service needs to have, to ensure that you're looking after both your own (and, just as important!, your fellow users') data and generally running a secure, tight ship.

Backing up the Mastodon Database

The first is that you need to have regular backups of the PostgreSQL database that underlies the whole Mastodon instance. I've published a backup utility that we use at the OER Foundation to back up our Docker containerised PostgreSQL instances every hour! By the way, PostgreSQL is an amazing database, both FOSS and world-class, that's used by many of our FOSS services! We would use it over, say, the vastly more expensive proprietary database, MS SQL Server, any day of the week. In many ways, PostgreSQL is more capable, and quite a lot faster besides.

I'll aim to write another (shorter!) tutorial on how to implement this system in the next week or two.

Backing up your VPS' files and configurations, incuding your Mastodon

Also, you'll need to make sure that all the important files (configuration and data) on your VPS are being backed up, and ideally sent somewhere remote to your VPS in case something happens to it (rare though that might be), like it gets cracked by ne'er-do-wells or your hosting company suddenly goes belly-up (haven't heard of it happening, but there's always a first time!)...

We use an amazing FOSS tool called Restic. It allows us to make automated remote incremental backups of the VPS filesystem (including the frequent database dumps created by the database backup script above), and it even encrypts them on the way to protect your users' data even if the backup server is somehow compromised. This is best practice. I've also created a script to deploy Restic, and will need to write another tutorial to provide some more explanation.

To use this script, you'll need somewhere (that's accessible from the internet) to which you can send our backups! That's either a server with big hard disks in someone's home (with an externally visible network and a properly configured router), or some other backup location, like a commodity object store (often these are advertised as 'AWS S3-compatible') or a big internet-addressible block storage device. The Restic site will provide some guidance on this.

Upgrading your Mastodon instance to newer versions

Last, Mastodon's developers seldom rest for long. Eugen and the rest of the Mastodon developer community are constantly looking at how they can improve Mastodon, or fix any issues that might emerge. While I was composing this tutorial, the current release version went from 3.5.3 -> 4.0.0. -> 4.0.1 in a matter of a couple hours. You don't need to upgrade your instance every time there's a new release, although you do want to apply any security-specific upgrades as quickly as possible to minimise the window of time that your instance is vulnerable!

I'll need to write another tutorial on how to do updates, but you should find basic instructions on each release! If your instance is not current at the time of a new release, make sure you read the intervening release notes, too as there are sometimes special instructions for a given jump from one release to the next.

Always make sure you have valid backups, ideally on the same VPS, before you do an upgrade so that you can roll back in the event the upgrade fails for some reason!

Well done reading through this screed of techno-text! I hope it was a lot quicker to read than it was to write (sheesh! I think I need a lie-down). All the best with your adventures in the Fediverse as a full-fledged contributing member!!

Blog comments

I get permission denied as…

nick (not verified) Mon 21/11/2022 - 04:42

I get permission denied as user or root on:
Then we'll enable forwarding from internal network interfaces as required for Docker containers to be able to talk to the outside world:

sudo $EDIT /etc/default/ufw

Did you define an editor …

dave Wed 08/02/2023 - 22:35

Did you define an editor (which happens prior to that line, e.g. EDIT=which nano or EDIT=which vim)?

Well written article however…

David Field (not verified) Tue 20/12/2022 - 02:43

Well written article however, i think you ned to set passwords for the Postgres and Elastic in .env.production as docker-compose run --rm web rails db:migrate fails.

Also I think if you want the resulting docker containers to work on Ubuntu 22.04 then you'll need to add

sudo iptables -t nat -A POSTROUTING ! -o docker0 -s 172.0.0.0/8 -j MASQUERADE

Thanks for that, David. Will…

dave Wed 08/02/2023 - 22:33

Thanks for that, David. Will double check that the next time I do a Mastodon setup - but, for what it's worth, this configuration works for me without the NAT iptables rule, and I didn't have issues with the db:migrate passwords (I generally disable Elastic, for what it's worth)...

Add new comment

OERu Web Services as of August 2022

dave — Mon, 08 Aug 2022 04:38:07 +0000

OERu Web Services as of August 2022 dave Mon 08/08/2022 - 16:38

In the intervening 17 months since my last update the OER Foundation (OERF) has continued to develop new online services for its global community of learners and educators. All of these services are themselves built with - and hosted on - Free and Open Source Software or FOSS. Each application is the work and responsibility of its own global developer community. The OERF participates in these communities at various levels as we avail ourselves of this wealth of brilliant software for the benefit of our users.

In addition to refining our OERu services over the past year, adding a few and sloughing off a few, we have also begun to assist other organisations by hosting systems on their behalf, usually with the aim of handing them over to their control after a period of system administration tuition.

As of this writing, in August 2022, we run the following production systems:

a couple of WordPress Multisites: our main course delivery site, where each course is a 'subsite' - https://course.oeru.org our train-the-educators blog and sandbox course site. Each educator gets a 'blog' subsite for their own use, and a 'course sandbox' subsite - https://edt4ol.oeru.org
a trio of standalone WordPress instances for the OERF and related initiatives: the OERF's own organisational website - https://oerfoundation.org the Centre for Open Educational Practice (COEP) - https://coep.nz the OERF's initiative to help educators in the developing world cope with the challenges of COVID19 - https://oer4covid.oeru.org
a pair of Drupal sites (they've been upgraded to Drupal 9.x) the OERF's information tech site (this site!) - https://tech.oeru.org, and our H5P learning object builder site - https://h5p.oeru.org
a couple Discourse instances - Discourse is the market leading, modern, full-featured (and open source) forum: our collaboration and support site for educators developing Open Educational Resources (OERs) - https://community.oeru.org our companion site for folks taking OERu courses for both assignments and collaboration with other learners - https://forums.oeru.org
a couple NextClouds for file sharing and many other uses: our main collaborative document management system and collaborative authorship and sharing media and documents - https://docs.oeru.org our internal collaboration system - https://hub.oeru.org
to support our NextCloud, we have integrated OnlyOffice allowing it to behave like a feature-rich Google Suite, but refreshingly Google-free - https://onlyoffice.oeru.org
a BigBlueButton instance - a large scale real-time video conferencing and educational collaboration platform - https://talk.oeru.org
a Mautic high-powered mailing list automation management system - https://mautic.oeru.org
a Rocket.Chat instance - a rich chat service comparable to Slack or Discord but fully open source (and we hold all our own data) - https://chat.oeru.org
a Keycloak instance for our still work-in-progress consolidated authentication and identity management system (aka Single Sign-on) - https://login.oeru.org
a Mastodon instance - Twitter-esque but de-centralised and non-commercial federated open source social network (part of the Fediverse) - https://mastodon.oeru.org
a Mailcow instance for managing all our email - Mailcow is a full multi-tenanted SMTP/IMAP/POP email system with active spam filtering, user self-service, DKIM configuration/management, webmail, & much more - even virus scanning for Windows clients - https://about.oerfoundation.org - it supplies email services for the oeru.org, oerfoundation.org, coep.nz domains, among others!
an instance of YourLS - a link shortener - https://oer.nz
a Matomo instance - website analytics (like Google Analytics, but without infringing on the EU's GDPR legislation) - https://stats.oeru.org
a Mahara instance - online academic portfolio tool - https://portfolio.oerfoundation.org
a Moodle instance - market leading learning management system which we use primarily for offering quizzes and awarding certificates for participants - https://moodle.oeru.org
a SilverStripe instance - our main OERu website - https://oeru.org
a MediaWiki instance - collaboration platform for OER development built on the same platform on which Wikipedia is built. All our courses are developed here - https://wikieducator.org
a GitLab - a software developer version control/collaboration tool where we make all our code available - https://git.oeru.org
a LimeSurvey instance - survey tool - https://survey.oeru.org
a BitWarden/VaultWarden](https://github.com/dani-garcia/vaultwarden) instance - password manager and cross-device sync service - https://safe.oeru.org (here's our howto for hosting your own)
an instance of Semantic Scuttle - a collaborative public website bookmarking service - https://bookmarks.oeru.org

This past year, having elected to use OnlyOffice for productivity purposes, we retired our Etherpad-light and CollaboraOffice instances.

We also added

a Wekan instance - for project planning via the Kanban methodology. Similar to Trello (among other tools) - https://kanban.oeru.org.
a Mobilizon instance - for managing events, and people following, discussing, and signing-up for and attending them - https://events.oeru.org
a RustDesk server instance, allowing us to provide anyone anywhere on just about any computing platform (Linux, Windows, MacOS, iOS, and Android) with live interactive shared desktop support
a server monitoring solution based on Grafana, Prometheus, Node-exporter, Alertmanager, cAdvisor, and other tools, which provide a graphical monitoring solution for each of our servers.

We also maintain development and testing/staging instances of most of these services. Our services are all hosted on virtual Linux servers (we mostly run Ubuntu Linux) provided by commodity cloud hosting providers via Docker and orchestrated via Docker Compose. As we reported last year, in time it's likely we'll move to "just-in-time" scaling via Kubernetes, but for now that'd be overkill.

Hosting on behalf

In the past year, we've started hosting services on behalf of other organisations, including Commonwealth of Learning, Open Education Global, and the government of Samoa, in particular their Ministry of Education, Sport, and Culture's Innovative Lifelong Learning Lab (the MiLLL).

For the MiLLL project:

a pair of WordPress Multisites, one for hosting open educational resource-based course sites - https://course.milll.ws- and one hosting individual sites for Samoan primary and secondary schools - https://schools.milll.ws
a BigBlueButton instance supporting learners, educators, and government staff's video conferencing requirements - https://bbb.milll.ws
a Mastodon instance especially for Samoans wanting to participate in digital social media - https://mastodon.milll.ws
an instance of Rocket.Chat - https://chat.milll.ws
an instance of Discourse - https://forum.milll.ws
an instance of Moodle - https://moodle.milll.ws
an instance of BitWardern/VaultWarden - https://safe.milll.ws

For the Commonwealth of Learning:

a WordPress Multisite for hosting OER-based course sites for the Pacific Partnership in Open Distance and Flexible Learning initiative - https://pacificopencourses.col.org

For Open Education Global:

a WordPress Multisite for hosting OER-based course sites - https://course.oeglobal.org

Costs and Usage

In the past year we have served many thousands of registered users and over 200,000 anonymous learners access our courses (the full content of which are open to all without requiring authentication).

Our entire annual IT infrastructure cost, including for our 'on behalf' hosting partners, was comfortably less than USD10,000. What's more our usage monitoring suggests that we were operating below 10% capacity, meaning that we have a lot of additional headroom.

Add new comment

Installing and Upgrading Moodle with Docker Compose on Ubuntu 22.04

dave — Sun, 22 May 2022 23:13:24 +0000

Installing and Upgrading Moodle with Docker Compose on Ubuntu 22.04

Blog tags

ubuntu linux

22.04

docker compose

docker

mysql

mariadb

nginx

let's encrypt

dave Mon 23/05/2022 - 11:13

Note: work in progress

Moodle is probably (there's little agreement among the pundits) the market-leading Free and Open Source Software (FOSS) Learning Management System (LMS). It's pretty much everywhere. Here's how you can set up and maintain your own Moodle instance(s).

Prerequisites
Installing Moodle on Ubuntu 22.04 with Docker Compose
Backing up your data
- MariaDB database dumps
Upgrading Moodle with Git

Prerequisites

If you would like to host your own (and we certainly encourage it!) using the same approach we use at the OER Foundation here is how you get started:

Set up a Virtual Private Server (VPS) and configure it to run Ubuntu Linux and Docker Compose.
Make sure any administrative users can log into the VPS securely via Secure Shell (SSH).

From that point, use the following process.

Installing Moodle on Ubuntu 22.04 with Docker Compose

The process for completing the install involves

setting up a domain name (or subdomain) to point at the server so people can easily find the Moodle site,
installing all the relevant code via 'Git' on the VPS' file system,
configuring the Docker Compose file, defining the set of Docker containers comprising this site's moving parts, and 'pulling' the component containers (to download the current versions of each to our VPS),
adding the 'default configuration' for the Dockerised NGINX instance, so it knows how to server Moodle from the adjacent containers,
configuring the NGINX webserver running on the VPS to act as a 'reverse proxy' as well as using to generate a Let's Encrypt Secure Sockets Layer (SSL) certificate to encrypt and otherwise secure users' interactions with the Moodle instance, and finally
configuring the Moodle instance itself.

Configure a domain name

Configuring a domain name requires us to register a domain with a 'domain registrar' - we use one called Metaname, based here in Christchurch, New Zealand - and using the registrar's browser based configuration system to create a 'zone file' which defines the server to which the domain points. Each registrar is likely to have their own bespoke zone file configuration tools, so consider this to be just one example.

Our convention at the OERF is to allocate a default domain name to each VPS we create. Each one hosts one or more Free and Open Source Software services. Each of those services has its own domain name. Usually we designate "A" and "AAAA" records for the default domain name pointing to the IPv4 and IPv6 addresses given to the VPS by the cloud services provider.

For the purposes of this tutorial, we'll create a site called moodletest.milll.ws which is the moodletest subdomain of the milll.ws (Ministry for Education, Sport, and Culture's 'innovative Lifelong Learning Lab', aka MiLLL, in (western) Samoa). We use a "CNAME" to point the subdomain at the VPS' official name, sandbox.milll.ws, which is how its A and AAAA records identify it (via its IPv4 and IPv6 addresses).

So our "Fully Qualified Domain Name" (FQDN) is moodletest.milll.ws.

To prepare for the rest of this tutorial, we're going to set a variable for your selected domain name. Do that by typing this at your command prompt. It will set this variable ''for this session'':

FQDN=your domain name

in our case, it'd be

FQDN=moodletest.milll.ws

The following will assuming that this value is set correctly.

Install the Moodle source code

Before we can install the Moodle source code, we have to prepare a sensible place for it to live. We do that by creating suitable directories for both the Docker configurations and the Moodle instance data and configuration as follows:

sudo mkdir -p /home/docker/${FQDN} /home/data/${FQDN}

That creates two directories as you can see. We need to go into the data directory:

cd /home/data/${FQDN}

and in there, we'll execute this git command to download the entire Moodle codebase:

sudo git clone https://github.com/moodle/moodle.git src

When that finishes, go into the source directory:

cd src

Here, we'll have to execute this new command, which we use to declare this a 'safe' git directory, which helps protect us from nefarious parties running git commands in dangerous places if they can get access to our server via some other security vulnerability...

sudo git config --global --add safe.directory /home/data/${FQDN}

Then we can run this comman:

git branch -a

This should give you a list of all the Moodle repository version 'tags'.

In this case, we're going to install a stable (but not the latest) version of Moodle - to do that we tell git what tag we're going to track:

sudo git branch --track MOODLE_310_STABLE origin/MOODLE_310_STABLE

And then, to make sure that our Docker container's nginx webserver can read these files (they need to be owned by the www-data users, and I (user 'dave') can run git commands without needing to use sudo. Replace with your own user name here!

sudo chown -R www-data:dave ../src
sudo chmod -R g+w ../src
ls -l
git status
git checkout MOODLE_310_STABLE

Configure Docker Compose to host Moodle

The first step to running anything with Docker Compose - which helps you combine a set of Docker containers together into a useful, maintainable service - is to create docker-compose.yml file. It usually contains the full 'recipe' for the service.

Go to the relevant docker directory and create and edit the file:

cd /home/docker/${FQDN}

sudo nano docker-compose.yml

And copy and past in this content (with FQDN occurrences and [secret password] placeholders to be replaced appropriately, of course):

version: '3'
 
services:
    mariadb:
        image: mariadb:10
        container_name: db
        environment: 
            MYSQL_RANDOM_ROOT_PASSWORD: 1
            MYSQL_DATABASE: moodle
            MYSQL_USER: moodle
            MYSQL_PASSWORD: [secret MariaDB password]
        volumes:
            - /home/data/FQDN/mariadb:/var/lib/mysql
    moodle:
        image: oeru/moodle:php74-fpm
        environment:
            MOODLE_DOMAIN: FQDN
            MOODLE_DB_HOST: mariadb
            MYSQL_PORT_3306_TCP: 3306
            MOODLE_DB_NAME: moodle
            MOODLE_DB_USER: moodle
            MOODLE_DB_PASSWORD: [secret MariaDB password]
        volumes:
            - /home/data/FQDN/src:/var/www/html
            - /home/data/FQDN/data:/var/www/moodledata
        restart: unless-stopped
        depends_on:
            - mariadb
    cron:
        image: oeru/moodle-cron:php74-fpm
        environment:
            MOODLE_DOMAIN: FQDN
            MOODLE_DB_HOST: mariadb
            MYSQL_PORT_3306_TCP: 3306
            MOODLE_DB_NAME: moodle
            MOODLE_DB_USER: moodle
            MOODLE_DB_PASSWORD: [secret MariaDB password]
        volumes:
            - /home/data/FQDN/src:/var/www/html
            - /home/data/FQDN/data:/var/www/moodledata
        restart: unless-stopped
        depends_on:
            - mariadb
    nginx:
        image: nginx
        depends_on:
            - moodle
        ports:
            - 127.0.0.1:8080:80
        volumes:
            - /home/data/FQDN/nginx:/etc/nginx/conf.d
            - /home/data/FQDN/src:/var/www/html
        restart: unless-stopped

And, naturally, save the file.

Note, the last stanza of the docker-compose.yml file, defining the NGINX component, specifies a port number on the host machine (i.e. 'localhost' or 127.0.0.1) of 8080, which in turn points to port 80 on the NGINX container. If this is the only Docker Compose configuration on your server, then 8080 is probably an avaiable port to use, but if you server has lots of other Docker Compose (or other sorts of services on it) port 8080 might not be available. You can check by running

sudo netstat -punta

If netstat isn't installed, you can install it via sudo apt install net-tools.

Running this command will give output that looks something like this:

Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      11199/nginx: master 
tcp        0      0 0.0.0.0:25              0.0.0.0:*               LISTEN      60021/master        
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      59473/sshd: /usr/sb 
tcp        0      0 127.0.0.53:53           0.0.0.0:*               LISTEN      11263/systemd-resol 
tcp        0      0 127.0.0.1:8080          0.0.0.0:*               LISTEN      94497/docker-proxy  
tcp        0      0 127.0.0.1:6010          0.0.0.0:*               LISTEN      13691/sshd: dave@pt 
tcp        0      0 127.0.0.1:6011          0.0.0.0:*               LISTEN      13097/sshd: dave@pt 
tcp        0      0 147.182.232.12:22       203.118.159.63:39248    ESTABLISHED 13044/sshd: dave [p 
tcp        0      0 147.182.232.12:22       202.4.38.248:54655      ESTABLISHED 94900/sshd: william 
tcp        0      0 147.182.232.12:22       202.4.38.248:53108      ESTABLISHED 91961/sshd: sialofi 
tcp        0     52 147.182.232.12:22       203.118.159.63:39650    ESTABLISHED 13638/sshd: dave [p 
tcp        0      0 147.182.232.12:22       202.4.38.248:62921      ESTABLISHED 92704/sshd: eleanor 
tcp6       0      0 :::80                   :::*                    LISTEN      11199/nginx: master 
tcp6       0      0 :::25                   :::*                    LISTEN      60021/master        
tcp6       0      0 :::22                   :::*                    LISTEN      59473/sshd: /usr/sb 
tcp6       0      0 ::1:6010                :::*                    LISTEN      13691/sshd: dave@pt 
tcp6       0      0 ::1:6011                :::*                    LISTEN      13097/sshd: dave@pt 
udp        0      0 127.0.0.53:53           0.0.0.0:*                           11263/systemd-resol

You'll notice that in this case, the 7th line says

tcp 0 0 127.0.0.1:8080 0.0.0.0:* LISTEN 94497/docker-proxy

which means that port 8080 is already in use with 127.0.0.1 on the server and trying to start my containers with that port defined would result in a 'port unavailable' error.

Here's a quick one-liner allowing you to check whether any particular port is in use right now (replace '8080' with whatever port number you're checking):

sudo netstat -punta | grep '8080'

if it returns nothing, that port is ''currently'' unused. If it returns a line like

tcp 0 0 127.0.0.1:8080 0.0.0.0:* LISTEN 94497/docker-proxy

it's in use and you'll need to choose another port, e.g. 8081 or 8070 or something else. Ports over 8000 are unlikely to be used by most normal software so should be fair game. You can verify that your selected port is available before launching your containers.

Download your containers from Docker Hub

You should then be able to 'pull' the relevant Docker images (might take a while depending on your server's location and available bandwidth):

sudo docker-compose pull

Once that's done, we need to make sure that our container running NGINX webserver has a valid Moodle-specific configuration.

Configure NGINX container for Moodle

To do that, we're going to create an NGINX default configuration for the Moodle containers by creating and editing the file:

sudo nano /home/data/${FQDN}/nginx/default.conf

copying and pasting in the following content. It shouldn't require any changes.

# This configuration is used by a Docker container running nginx, and is not intended to be
# directly connected to the Internet (for that we recommend using SSL on port 443).
# You should reverse proxy this container on the Docker host using whatever webserver
# you have running there.
#
server {
    listen 80 default_server;
 
    # default path to Mautic - from the point of view of the docker container
    root /var/www/html;
    index index.php index.html index.htm;
    fastcgi_keep_conn on; # keep alive to the FCGI upstream
    location / {
        # First attempt to serve request as file
        try_files $uri $uri/index.php;
 
        # moodle rewrite rules
        rewrite ^/(.*.php)(/)(.*)$ /$1?file=/$3 last;
    }
    # php parsing
 
    location ~ ^(.+\.php)(.*)$ {
        fastcgi_split_path_info  ^(.+\.php)(.*)$;
        fastcgi_index index.php;
        fastcgi_pass moodle:9000;
        include /etc/nginx/mime.types;
        include fastcgi_params;
        fastcgi_param PATH_INFO $fastcgi_path_info;
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
        fastcgi_buffer_size 128k;
        fastcgi_buffers 256 4k;
        fastcgi_busy_buffers_size 256k;
        fastcgi_temp_file_write_size 256k;
        client_max_body_size 100M;
    }
    add_header 'Access-Control-Allow-Origin' "*";
}

Once this is in place, you can start up the Docker containers like this (as your own user!):

docker-compose up -d && docker-compose logs -f

which will start the containers (the up -d) in 'daemon mode', where they continue to run even if you log out of the server, and then show you the logging from the containers (the logs -f) part. To exit the log view (which will update automatically as more log messages are created), hit the CTRL-C key combination. If there're any errors from your individual containers, they should visible here.

That's it, your Docker containers should be running. Once you're out of the log view, you can always check on your containers by going into the /home/docker/FQDN directory and executing:

docker-compose ps

which should show you a list of the currently running containers and their states. Here's an example:

           Name                         Command               State           Ports         
--------------------------------------------------------------------------------------------
db                           docker-entrypoint.sh mariadbd    Up      3306/tcp              
moodletestmilllws_cron_1     /entrypoint.sh /bin/sh -c  ...   Up      9000/tcp              
moodletestmilllws_moodle_1   /entrypoint.sh php-fpm           Up      9000/tcp              
moodletestmilllws_nginx_1    /docker-entrypoint.sh ngin ...   Up      127.0.0.1:8080->80/tcp

or, to get a bit more info, you can run

docker-compose top

which should show you a bit more information about the different running containers and their active processes. Here's an example:

db
UID    PID    PPID    C   STIME   TTY     TIME       CMD   
-----------------------------------------------------------
lxd   12056   12035   0   May23   ?     00:04:06   mariadbd
 
moodletestmilllws_cron_1
UID     PID    PPID    C   STIME   TTY     TIME                         CMD                     
------------------------------------------------------------------------------------------------
root   12244   12201   0   May23   ?     00:00:00   /bin/sh -c cron && tail -f /var/log/cron.log
root   12651   12244   0   May23   ?     00:00:02   cron                                        
root   12652   12244   0   May23   ?     00:00:13   tail -f /var/log/cron.log                   
 
moodletestmilllws_moodle_1
  UID       PID    PPID    C   STIME   TTY     TIME                              CMD                         
-------------------------------------------------------------------------------------------------------------
root       12250   12173   0   May23   ?     00:00:12   php-fpm: master process (/usr/local/etc/php-fpm.conf)
www-data   37755   12250   2   01:46   ?     00:00:00   php-fpm: pool www                                    
www-data   37757   12250   1   01:46   ?     00:00:00   php-fpm: pool www                                    
www-data   37758   12250   1   01:46   ?     00:00:00   php-fpm: pool www                                    
www-data   37759   12250   0   01:46   ?     00:00:00   php-fpm: pool www                                    
 
moodletestmilllws_nginx_1
  UID       PID    PPID    C   STIME   TTY     TIME                        CMD                    
--------------------------------------------------------------------------------------------------
root       13207   13179   0   May23   ?     00:00:00   nginx: master process nginx -g daemon off;
systemd+   13333   13207   0   May23   ?     00:00:01   nginx: worker process                     
systemd+   13334   13207   0   May23   ?     00:00:00   nginx: worker process

So, now we should have all the relevant containers for your Moodle instance running... Al that's left to do is hook up the reverse proxy configuration creating that last vital link: between your browser's request and the Moodle service (secured by Let's Encrypt)!

Create a secure reverse-proxy configuration

The first step to doing that is to create a reverse proxy configuration:

sudo nano /etc/nginx/sites-available/${FQDN}

Copy and paste the following into the file. Of course, you will want to replace every occurrence of 'FQDN' with your own domain name.

server {
    # add [IP-Address:]80 in the next line if you want to limit this to a single interface
    listen 80;
    listen [::]:80;
 
    server_name FQDN;
 
    root /var/www/html;
 
    # for let's encrypt renewals!
    include includes/letsencrypt.conf;
 
    access_log /var/log/nginx/FQDN_access.log;
    error_log /var/log/nginx/FQDN_error.log;
 
    # redirect all HTTP traffic to HTTPS.
    location / {
       return  302 https://$server_name$request_uri;
    }
}
 
# This configuration assumes that there's an nginx container talking to the moodle PHP-fpm container,
# and this is a reverse proxy for that Moodle instance.
fastcgi_cache_path /etc/nginx/cache levels=1:2 keys_zone=moodle:100m inactive=1d max_size=100m;
fastcgi_cache_key "$scheme$request_method$host$request_uri";
 
server {
    # add [IP-Address:]443 in the next line if you want to limit this to a single interface
    listen 443 ssl;
    listen [::]:443 ssl;
 
    ssl_certificate /etc/ssl/certs/ssl-cert-snakeoil.pem;
    ssl_certificate_key /etc/ssl/private/ssl-cert-snakeoil.key;
#    ssl_certificate /etc/letsencrypt/live/FQDN/fullchain.pem;
#    ssl_certificate_key /etc/letsencrypt/live/FQDN/privkey.pem;
    ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
 
    # A strong ssl_dhparam will help secure your server from certain kinds of attacks. 
    # It is commented out unless you've created it (if it's uncommented but doesn't exist
    # nginx won't start). To create one, see 
    # https://raymii.org/s/tutorials/Strong_SSL_Security_On_nginx.html
    # note: it can take 5-30 minutes to create one.
    #ssl_dhparam /etc/ssl/certs/dhparam.pem;
    #ssl_stapling on;
    #ssl_stapling_verify on;
 
    ssl_session_tickets off;
    resolver_timeout 5s;
    keepalive_timeout 20s;
 
    server_name FQDN;
 
    # for let's encrypt renewals!
    include includes/letsencrypt.conf;
 
    access_log /var/log/nginx/FQDN_access.log;
    error_log /var/log/nginx/FQDN_error.log;
 
    client_max_body_size 100M;
 
    location / {
        proxy_pass http://127.0.0.1:8080;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_set_header Host $http_host;
    }
}

Once you've saved that file, we need to make sure NGINX can see the configuration - we do that by creating a 'symbolic link' between our configuration file in the sites-available directory and the sites-enabled directory as follows:

sudo ln -sf /etc/nginx/sites-available/${FQDN} /etc/nginx/sites-enabled/

Before we can restart NGINX with this new configuration, we have to make sure that the 'letsencrypt.conf' cited in our configuration file exists. We do that by creating a new directory:

sudo mkdir /etc/nginx/includes

and creating the letsencrypt.conf file:

sudo nano /etc/nginx/includes/letsencrypt.conf

into which we copy and paste the following:

#############################################################################
# Configuration file for Let's Encrypt ACME Challenge location
# This file is already included in listen_xxx.conf files.
# Do NOT include it separately!
#############################################################################
#
# This config enables to access /.well-known/acme-challenge/xxxxxxxxxxx
# on all our sites (HTTP), including all subdomains.
# This is required by ACME Challenge (webroot authentication).
# You can check that this location is working by placing ping.txt here:
# /var/www/letsencrypt/.well-known/acme-challenge/ping.txt
# And pointing your browser to:
# http://xxx.domain.tld/.well-known/acme-challenge/ping.txt
#
# Sources:
# https://community.letsencrypt.org/t/howto-easy-cert-generation-and-renewal-with-nginx/3491
#
# Rule for legitimate ACME Challenge requests
location ^~ /.well-known/acme-challenge/ {
    default_type "text/plain";
    # this can be any directory, but this name keeps it clear
    root /var/www/letsencrypt;
}
# Hide /acme-challenge subdirectory and return 404 on all requests.
# It is somewhat more secure than letting Nginx return 403.
# Ending slash is important!
location = /.well-known/acme-challenge/ {
    return 404;
}

In turn, we also have to make sure that the directory stipulated in the letsencrypt.conf file exists - we'll attempt to create the directory, which won't hurt anything if it already exists:

sudo mkdir /var/www/letsencrypt

With that done, we can test NGINX's configuration:

sudo nginx -t

If it's free of errors (there might be a warning related to the snakeoil ''temporary'' certificates we've specified. That's ok - we just specified them so that we can get NGINX restarted in a way that let's us request an SSL certificate from Let's Encrypt!), we can get NGINX to reload its configuration, so that it'll know how to respond to requests for the FQDN domain!

sudo service nginx reload

If that returns without errors, we're in business! It's time to make our Let's Encrypt certificate. Creating one is easy:

sudo letsencrypt certonly --webroot -w /var/www/letsencrypt -d ${FQDN}

In our case, it's:

sudo letsencrypt certonly --webroot -w /var/www/letsencrypt -d moodletest.milll.ws

Running that should trigger output like this:

Saving debug log to /var/log/letsencrypt/letsencrypt.log
Enter email address (used for urgent renewal and security notices)
 (Enter 'c' to cancel): webmaster@milll.ws
 
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Please read the Terms of Service at
https://letsencrypt.org/documents/LE-SA-v1.2-November-15-2017.pdf. You must
agree in order to register with the ACME server. Do you agree?
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
(Y)es/(N)o: y
 
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Would you be willing, once your first certificate is successfully issued, to
share your email address with the Electronic Frontier Foundation, a founding
partner of the Let's Encrypt project and the non-profit organization that
develops Certbot? We'd like to send you email about our work encrypting the web,
EFF news, campaigns, and ways to support digital freedom.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
(Y)es/(N)o: y
Account registered.
Requesting a certificate for moodletest.milll.ws
 
Successfully received certificate.
Certificate is saved at: /etc/letsencrypt/live/moodletest.milll.ws/fullchain.pem
Key is saved at:         /etc/letsencrypt/live/moodletest.milll.ws/privkey.pem
This certificate expires on 2022-08-21.
These files will be updated when the certificate renews.
Certbot has set up a scheduled task to automatically renew this certificate in the background.
We were unable to subscribe you the EFF mailing list because your e-mail address appears to be invalid. You can try again later by visiting https://act.eff.org.
 
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
If you like Certbot, please consider supporting our work by:
 * Donating to ISRG / Let's Encrypt:   https://letsencrypt.org/donate
 * Donating to EFF:                    https://eff.org/donate-le
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

... which means you've successfully received a new certificate! Now you can update your NGINX reverse proxy configuration:

sudo nano /etc/nginx/sites-available/${FQDN}

to change it as follows (again, make sure you've replaced FQDN with your actual domain name!)

#    ssl_certificate /etc/ssl/certs/ssl-cert-snakeoil.pem;
#    ssl_certificate_key /etc/ssl/private/ssl-cert-snakeoil.key;
    ssl_certificate /etc/letsencrypt/live/FQDN/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/FQDN/privkey.pem;

Once you've saved the file you can test your NGINX configuration:

sudo nginx -t

and if it doesn't return any errors (nor should it return warnings), you can reload NGINX's config to make your new secure configuration live!

sudo service nginx reload

Installing Moodle

Now that we've filled in all the parts of the tool-chain, your server is ready to receive requests for your Moodle site! You should be able to go to your web browser on your local computer and enter the URL, replacing FQDN with your domain name of course:

https://FQDN

(and entering http://FQDN should automatically redirect to https://FQDN - feel free to test it!)

And if all is well, your Docker Compose stack running Moodle should respond by serving you up a page that looks like the attached "Install page" screenshot.

You'll probably notice it looks a bit rough, as, for the moment at least, the Cascading Style Sheets (aka CSS, the code that determines how web content looks) won't be loading correctly. We'll address that shortly.

We need to get through the installation of Moodle first, during which time, the Moodle system creates your config.php file in the /home/data/${FQDN}/src/ directory. Once that's created, you'll be able to edit the file and make the tweaks to fix this problem.

But first you'll need to proceed with the installation process - you'll be asked to select your language. And then you can click 'next' to continue. You'll be asked to specify your database (pick 'MariaDB' in this case) and then you'll be asked for the database details, as showing in the attached screenshot.

Your host will be db, your database name and user will both be moodle (case sensitive!) and you'll have to enter the database password you put into your docker-compose.yml file.

Then you should get a page showing you that your setup meets Moodle's requirement for installation (see attached screenshot), and finally you'll be asked to proceed with the install. Do so!

The installation will create the relevant database tables in the moodle database via the MariaDB container, and Moodle will create the aforementioned config.php file. The next thing to do is edit that file:

sudo nano /home/data/${FQDN}/src/config.php

where you'll tweak the configuration file so that the line

$CFG->wwwroot = 'http://moodletest.milll.ws';

looks like (note the shift from ''http'' to ''https''):

$CFG->wwwroot = 'https://moodletest.milll.ws';

And add this line above the final comment (the lines starting with '//')

$CFG->sslproxy=true;

which informs the Moodle system that it's sitting behind an SSL reverse proxy so it can adjust its behaviour accordingly.

So it should look something like this:

<?php  // Moodle configuration file

unset($CFG);
global $CFG;
$CFG = new stdClass();

$CFG->dbtype    = 'mariadb';
$CFG->dblibrary = 'native';
$CFG->dbhost    = 'db';
$CFG->dbname    = 'moodle';
$CFG->dbuser    = 'moodle';
$CFG->dbpass    = '[your super-secret db password]';
$CFG->prefix    = 'mdl_';
$CFG->dboptions = array (
  'dbpersist' => 0,
  'dbport' => 3306,
  'dbsocket' => '',
  'dbcollation' => 'utf8mb4_general_ci',
);

$CFG->wwwroot   = 'https://moodletest.milll.ws';
$CFG->dataroot  = '/var/www/moodledata';
$CFG->admin     = 'admin';

$CFG->directorypermissions = 0777;

$CFG->sslproxy=true;

require_once(__DIR__ . '/lib/setup.php');

// There is no php closing tag in this file,
// it is intentional because it prevents trailing whitespace problems!
?>

Save your updated config.php file, and return to the installation process - after the next button press, when the Moodle installer page loads, you should suddenly see the interface look all beautiful and tidy as the CSS suddenly loads.

You'll be asked set some properties for your site, like giving it a name, and finally, you'll have to provide the details of an 'admin' user - I recommend creating a default admin user, 'admin', with a very strong password. You'll log in with that admin user (and provide the details to your organisation so that there's a bit of succession planning, if you're not available at some point) just this once, and immediately create a new user for ''yourself'' with your preferred details, which you'll then elevate to 'administrator' privileges, and then do all your administration of the site via your own user.

Once you've successfully completed defining an admin user, you'll end up at the admin 'dashboard', where you can create the personal administrator user for yourself. And then you can log out as 'admin' and log in as yourself... at which point, you're done! Moodle is installed. Hazzah!

Backing up your data

The way we run Docker Compose, all the files we want to preserve are stored on the VPS's own filesystem. The Docker containers only store 'volatile' data. That means we can literally remove the Docker containers at any time, and it won't (under normal circumstances) affect our precious data. The only time it ''might'' is when data, e.g. in the database, is held in system memory rather than being written to disk.

That means that making backups of our data can be achieve simply by copying files on the hard disk - and if we're really interested in preserving that data, we'll send it somewhere else - on a different computer, in a different geographic region, and ideally managed by a different company - to achieve suitable diversity.

MariaDB database dumps

The only element of our stack which isn't amenable to this file copying is the data managed by MariaDB. It is ''not'' generally safe to copy the files of a running database and to expect that restarting the database with those copied files will result in an uncorrupted data set. To safely backup a database, you either have to shut it down (which is disruptive and inelegant) or you have to get it do a database 'dump' while it's running (which it can usually do very quickly, without noticable impact on the performance of the database.

A database 'dump' is a frozen snapshot of the data in a generic form, usually SQL. It allows us to restore the database tables, the data they contain, and the relationships (indices, foreign keys, etc.) between the tables. It usually even lets us take data from an older version of MariaDB and import it into a newer version (which is very handy!).

So we need a method for periodically - and automatically - creating database dumps of our MariaDB Moodle database. Luckily, it's not too hard. I've created this script to do it.

Upgrading Moodle with Git

Add new comment

Upgrading a Docker-based Discourse Forum instance

dave — Thu, 19 May 2022 02:45:12 +0000

Upgrading a Docker-based Discourse Forum instance

Blog tags

discourse

docker

dave Thu 19/05/2022 - 14:45

Discourse is the world-leading online web-based forum. It's a superb, extremely mature-and-yet-cutting edge platform. It also happens to be Free and Open Source Software, which is why we, at the OER Foundation, use it.

Like all good software, it's undergoing continuous improvement by its developer community who release fairly frequent - perhaps every couple weeks - updates. Luckily, keeping your Discourse forum up-to-date isn't particularly onerous.

Upgrading a Discourse instance deployed via Docker (as all of ours are) follows one of two possible workflows. Both usually start with either a Discourse administrator logging into the Discourse site (which she/he might not do very often) and noticing on the default admin dashboard page (see the first attached screenshot) that there is a pending upgrade, usually with a 'click here to upgrade' link. Alternatively, if an administrator is not logged in (the system is aware of this) the system sends administrators for the site an email (see the second attached screenshot for a sample) alerting them all that an upgrade is pending, with a link to the upgrade page in the Discourse administration system.

Always backup first!

As any good system administrator knows, their users' data is of utmost importance. So, before proceeding with any upgrade, it's crucial to make a backup of your system to ensure that if everything goes pear-shaped, you can recover and get back to where you were.

Normally, I consider it sufficient to take a Discourse 'backup' via the admin backup interface 'immediately before doing the upgrade' (see attached screenshot), leaving ''off'' the uploaded content, as this is unlikely to be affected or removed via the upgrade process, to save space. Doing this immediately prior to the upgrade minimises the chances that any users lose data (e.g. a recent post, in the space of time between the backup and the actual upgrade or even a post they're working on when the upgrade commences - if the latter happens, as long as they don't leave the page and wait for the forum to 'come back' after the upgrade, they should be able to proceed without losing anything... but there's a small chance they could be unlucky - be mindful of this and try to a) announce upgrades on the site prior to running them, and b) try to do so outside of your busiest hours, which should be easy to determine thanks to Discourse's superb analytics (also a feature of the default admin dashboard).

After conducting the backup, you can do the upgrade.

Upgrade Workflows

You will ''usually'' have a straightforward 'point-and-click' web browser based upgrade process, but on occasion, you will click on the 'upgrade link' and end up on a page like 'upgrade is too complicated for the web interface' message (see attached screenshot). In that latter case, things get a little more complicated. Also, such upgrades can result in somewhat more downtime for the forum, so only conduct them when you have sufficient time.

Web-based upgrade

In most cases, when you click on the "Click here to upgrade" link, you'll be taken to a page that assesses the various components (the Discourse codebase, the Docker subsystems, and any additional functional plugins) making up your Discourse install and alerts you which of those components have a pending upgrade. This can sometimes take a while (the system is having to compare what you have on your server to the official up-to-date versions of those things on the web), but when done, the page will show you 'upgrade' buttons for the components with upgrades pending as per the attached sample upgrade pages screenshot.

Once you've got those showing, you can click the 'Upgrade' button that's highlighted (if any - that one needs to be done first, prior to the others) and then the 'Start upgrade" operation for each one. Or, sometimes, you'll see an "Upgrade all" button at the top of the page, which applies all the pending upgrades in a single operation.

Usually, in practice, this renders your Discourse forum unusable (it'll be in 'maintenance mode' from a user's perspective) for a few seconds to a few minutes, depending on the upgrade.

Command line 'launcher' upgrade

When the upgrade is more fundamental, usually involving the Docker subsystem, you sometimes don't have the option of a web-based upgrade. Instead of the upgrade web pages from the previous section, you get message like that shown in the next screenshot, which talks about running the upgrade from the command line using the launcher command and doing a 'rebuild' of the system.

In that case, you need to be able to log into the host server (usually a remote cloud-based "Virtual Private Server" or VPS) - I always use SSH to do that see these instructions from a terminal on my Linux desktop - with suitable permissions (usually the ability to run 'sudo' commands, i.e. act as the administrative or 'root' user of that server).

Once you're logged in as a user with 'sudo' privileges, you need to go to the Docker configuration directory for that instance of Discourse - in the OER Foundation's case, I have the convention of having the installation in the /home/docker/ directory in a subdirectory with the same name as the as the site, e.g. forum.oeru.org. In that case, I'd enter

cd /home/docker/forum.oeru.org

Note that we don't use the default path for our Discourse installation (/var/discourse), so we're being rebels here. Doing an

ls -l

here should give a directory that looks a bit like this (you might not have the 'oeru'-related files) when listed:

-rw-rw-r-- 1 dave dave  1099 Nov 29  2019 LICENSE
-rw-r--r-- 1 dave root  8285 Jul 24  2021 README.md
-rw-rw-r-- 1 dave dave   258 Dec  2  2019 README.oeru
drwxrwxr-x 1 dave dave    16 Nov 29  2019 bin
drwxrwxr-x 1 dave dave    16 Apr 15 23:36 cids
drwxrwxr-x 1 dave dave    30 May 19 06:47 containers
-rwxrwxr-x 1 dave dave 11955 Apr 15 23:22 discourse-doctor
-rwxr-xr-x 1 dave root 27132 Oct 21  2021 discourse-setup
drwxrwxr-x 1 dave dave   192 Dec 22 04:43 image
-rwxrwxr-x 1 dave dave 23538 Apr 15 23:22 launcher
drwxrwxr-x 1 dave dave   120 Nov 16  2021 samples
drwxrwxr-x 1 dave dave    22 Nov 29  2019 scripts
drwxrwxr-x 1 dave dave    16 Nov 29  2019 shared
drwxrwxr-x 1 dave dave   866 Apr 15 23:22 templates
drwxr-xr-x 1 dave root   130 Jan 16 22:42 tests

The script we're going to use is the launcher script, and we're going to use the site's configuration which, by our convention, will reside in containers/app.yml(you can see its contents by typing less containers/app.yml assuming your user has sufficient permissions to view it directly. If not, prepend sudo to your command.

To run the upgrade, you have to complete two steps as suggested by the page you saw when you clicked on the upgrade link:

First, ensure that the 'git' repository that was used to provide this set of files to begin with is up-to-date - if your user owns these files (as my 'dave' user does in the example above) then run this to avoid changing the ownership of the files:

git pull

If you don't own the files you'll need to run

sudo git pull

which should result in a fairly quick (a few seconds) request to the git server, pulling down files that have been changed by the developers since the last time you did a git pull... Note, if the pull fails, you may find that you've altered one or more files for some reason and that the change you've made would be clobbered by the pull. The easiest thing to do in that case is - for each file - to (prepending with sudo if required):

cp [filename] [filename].bak

so you have a backup copy of your changed file, and then issue (again, with sudo prepended if necessary):

git checkout [filename]

which reverts [filename] to its original git-distributed version. The git pull should now work.

Second, simply type (again, prepending with sudo if required:

./launcher rebuild app

This will initiate what can be a rather long process (perhaps 1 to as many as, say, 30 minutes, depending on the upgrade, the amount that needs to be downloaded, your server's network bandwidth, and a variety of other variables) which will involve downloading new Docker containers with Discourse code, as well as possibly a new PostgreSQL database container among others. Your Discourse will continue running as long as it can safely do so, but then the launcher script will shut down the containers and restart them. It might also be 'recompiling' aggregated resources used by Discourse, like various Ruby On Rails gems, icon files, style sheets (CSS), or Javascript files...

Once it's done, you'll get the command prompt back. You're done! Have a look at your site and you should see you're up-to-date!

You can log out of the server vai CTRL-D or typing exit.

Problems?

I have, on a few occasions, had an upgrade fail for some reason. If that happens, you should see an indicator of has caused the failure (in a few cases for me, it was the fact that the PostgreSQL database didn't have enough time to restart before the Discourse app tried to access it to run data schema upgrades, and failed as a result. Changing the 'timeout' value from 5 seconds to 60 seconds fixed it - looking on the Meta Discourse - Discourse's own Discourse used to discuss running Discourse - is a very good resource when trying to solve system-related problems. It's worth creating an account for yourself on it if you're going to be responsible for a Discourse instance!

Blog comments

I had to use the command `…

Peter N Lewis (not verified) Thu 23/03/2023 - 22:28

I had to use the command `./launcher rebuild app` (no `.yml`) for this to work.

Well spotted, Peter, and…

dave Wed 07/06/2023 - 15:41

Well spotted, Peter, and thanks for letting me know! I've corrected the tutorial.

Add new comment

Creating Linux shell users with sudo and SSH access

dave — Mon, 09 May 2022 23:33:58 +0000

Creating Linux shell users with sudo and SSH access dave Tue 10/05/2022 - 11:33

With Linux servers, both local and in "the Cloud" (aka a Virtual Private Server or VPS), access is generally remote via a command line interface. This tutorial covers the process of creating normal shell (command line) users who have the ability to perform administrative tasks (via the 'sudo' - Super User DO - mechanism), as well as configuring that user to be able to log in from their workstation via SecureShell (SSH) without needing to enter a password (i.e. using public-private key pairs).

To begin with, we need a Linux server to which we have administrative access. This can be direct access to a local Linux server on our own infrastructure, or a remote VPS for which we have the 'root' (administrative user) access, either via SSH (to a default root or unprivileged user created by the VPS provider) or via a browser-based console (which simulates sitting down at monitor/keyboard at the actual server itself).

You'll log into your server either via your web browser (see attached images of the Digital Ocean VPS control page, with console link, and a browser-based console session, logged in as the 'root' user), or via SSH from your workstation. If you are on Linux or MacOS, you can use the in-built SSH client (you'll want to create an SSH key-pair for yourself first - on Linux, on a Mac, or on Windows. Note: I normally leave the 'passphrase' blank - it's slightly less secure especially if your SSH keys are on a computer you share with other people, but it's a lot more convenient, and a trade-off I'm willing to make!

You can address your server either by its IP address (either IPv4, e.g. 137.184.84.159, or if you're one of the rare far-sighted people with an IPv6 address, e.g. 2604:a880:4:1d0::4a2:c000, you can use that instead).

If you'd like to create your own VPS using a cloud infrastructure provider like Digital Ocean, and perhaps even configure a domain name to point to it (so you can use that domain to address your server rather than an IP address, see the first couple sections of this tutorial.

Logging in

Logging into your VPS will depend on whether you're using a console or logging in via SSH. If via a console, you'll just need to enter the username (possibly 'root') and password you set when creating the VPS. If via SSH, you'll run

ssh [your server IPv4 or IPv6]

followed by the ENTER key (that'll be true for any line of commands I provide). If your VPS is configured for passwordless logins, you shouldn't need a password. If not, you'll need the password you created when provisioning the VPS. Usually, it's either one or the other. If you're running Putty, you'll enter the same details into the Putty interface.

In most cases, if you've never logged into that VPS via SSH previously, you're likely to see a notification like this right after entering the command (with your VPS' hostname and IP address and its own ECDSA key fingerprint - it'll be unique to each server):

The authenticity of host 'sandbox.milll.ws (163.12.232.115)' can't be established.
ECDSA key fingerprint is SHA256:0doRcAm9kAZjraohX+CeCwv1Yv6bIbzAYUSTrWGc9Y4.
Are you sure you want to continue connecting (yes/no/[fingerprint])?

Type 'yes' and ENTER. You should only ever see this once for any IP or domain name you use to access this particular VPS.

Once you're logged in, you'll see a command line prompt, probably somewhat similar to that of the attached screenshot. The name of the VPS you've logged into and your username will normally (by convention) be specified in your command prompt. In the attached screenshot, you can see my prompt is dave@mesc:~$. That means I'm logged in as user dave, the VPS's short-name is mesc (it's long name is mesc.oerfoundation.org which is too long to include in a prompt) and my current location is ~, the 'tilde' character which is short-hand on a UNIX shell for the user's 'home directory'. On a Linux VPS, it's likely to be in /home/[username] or /home/dave in my user's case.

Note, on a UNIX system, directories (aka folders) are delineated with a forward slash, /. Windows is the only system that uses a backslash \ to delineate directories.

The $ at the end of the prompt tells you you're an 'unprivileged user'. If you were logged in as an admin user, usually 'root', you'd see a # in your prompt to remind you that you've got unlimited power within the VPS, and to use it carefully!

If your prompt doesn't show your username, you can find out what user you're logged in as by typing

whoami

Creating a regular user

Let's say you're creating a user with the username of 'fiafia' - by convention, usernames on Linux systems are all lower case letters, without spaces or punctuation. Assuming your user has 'sudo' privileges, you can create the user like this (you will be asked to enter your user's password the first time you use sudo, and again after not using it for a while, usually 10 minutes):

sudo adduser fiafia

That will request that you enter additional information, like the user's full name, a phone number, and even an office room number (you can leave out any unnecessary information), and it will create a home directory for the user, /home/fiafia in this case.

It will also require you to set a password for the new user. If you're not sure how to generate a good random password, you can user our short tutorial. Should you want to change it you can set it like this - your user will need to know their password in order to run sudo commands:

sudo passwd $U

which will instruct you to enter a strong password twice (to guard against typos).

To give fiafia the ability to log in via SSH and to allow her to run administrative commands, her user needs to be added to several 'groups', members of which get additional privileges:

sudo adduser fiafia admin
sudo adduser fiafia sudo

Those groups, ssh, admin, and sudo will confer those abilities on your new user.

If you're working with VPS running a version of Ubuntu prior to 22.04 (like 20.04) or perhaps a Debian variant, you might also need to add the user to the 'ssh' group (it appears that with 22.04, Ubuntu have decided the ssh group was unnecessary and have removed it).

Just to be safe, if you run this on 22.04 too, it might fail to complete (saying something like "there's no 'ssh' group"), but that's not a problem.

sudo adduser fiafia ssh

If you wanted to create several users, you could avoid typing in each user's name so many times by instead setting a variable in the shell, say U, to the username and then reference the value of U, which you do by putting a '$' in front of it, like $U. So to create another new user, say 'masina', you could do the same thing like this, and copy and paste the following into your terminal window (If pasting via 'CTRL-V' doesn't work, try 'SHIFT+CTRL-V', similarly copy via 'SHIFT+CTRL-C'):

U=masina
sudo adduser $U
sudo adduser $U admin
sudo adduser $U sudo

(Note: to repeat these and the following commands for another user, just assign a new username to U, U=[newusername] and re-paste the same commands with $U.)

Because, by following this tutorial, your user is likely to log in without using their password, it makes sense to put it somewhere safe where they can find it. I usually do it this way (replace [password] with the same password you used for the previous command):

sudo echo "[password]" > /home/$U/.password

which, in this case, will put the password in a 'hidden' file (files or directories with names starting with a '.' are called hidden files or directories - they won't appear in a normal directory listing, e.g. ls or ls -l. If you want to see them, try ls -la where the 'a' means 'all') called .password in the user masina's home directory, /home/masina.

You will also want to make sure that your new user has permission to remove that file. The right way to do that is to transfer ownership of the file to that user:

sudo chown $U /home/$U/.password

Creating SSH keys on your VPS

Now we can create an SSH key pair for this new user on this VPS. Helpfully, in the process, the .ssh (hidden) directory for the file into which to put that user's workstation public SSH key to allow them passwordless logins from that workstation:

sudo su $U

That command allows your user to "become" the user $U and execute the following commands (the 'sudo' isn't needed here as that user is working in her/his own space with her/his own files):

ssh-keygen -t rsa -b 2048

This command uses the SSH keygen utility to create a 2048bit keypair: a public and a private key which provide the means for engaging in full encrypted (and extremely secure!) interactions with a remote server via the SSH protocol. You'll have to hit "ENTER" three times to complete the process (as I said above, I normally leave the passphrase blank).

Once that is done, you can create and edit a new file:

nano ~/.ssh/authorized_keys

In that file, you can copy and paste (without spaces on either end) that user's workstation's public ssh key (never publish a private key anywhere!), save and close the file (use CTRL-X).

The quickest way to do that on a Linux computer is by running the command (back on your workstation!):

cat ~/.ssh/id_rsa.pub

which will print the contents of your public key file below it, so you can easily highlight and copy & paste it (remember, a copy from a Linux terminal is SHIFT+CTRL+C).

When you're done pasting the public key into your authorized_keys file, save and exit the editor (CTRL-X) and type

exit

The exit will return you to your own user. You can use CTRL-D (slightly faster to type) to do the same thing.

From that point, that user should be able to SSH to the VPS via ssh [username]@[domain name] without needing to enter a password.

These instructions use 'sudo' in front of commands because I assume you're using a non-root user. The instructions will still work fine even if you're logged in as 'root'.