postfix

Creating your own OER Foundation-style Libre Self-hosting Infrastructure with Docker Compose and Ubuntu LTS

dave — Thu, 12 Oct 2023 00:12:25 +0000

Creating your own OER Foundation-style Libre Self-hosting Infrastructure with Docker Compose and Ubuntu LTS

Blog tags

ubuntu linux

nginx

let's encrypt

postfix

docker compose

dave Thu 12/10/2023 - 13:12

Tips for this tutorial
Create a Virtual Private Server
- VPS Properties:
Key variables for you VPS
- Get your Domain lined up
Editing files
- Set up an unprivileged user for yourself
Configure the VPS
- Configuring your firewall
- Install the Nginx
Outgoing VPS Email (optional)
Installing the Docker Engine, Docker Compose, and Let's Encrypt
- Backwards compatibility for Docker Compose
- Docker use by non-root user
Setting up places for your Docker configurations and persistent Data
Configuring Nginx reverse proxy for your service
Let's Encrypt setup
- Requesting Let's Encrypt certificates

Tips for this tutorial

This tutorial is aimed at adventuresome would-be system administrators. I try not to assume any specialised knowledge on your part, and try to provide useful tips and explanation along the way to help you build a valid mental model of what you're doing. At the same, this is not a trivial process. Luckily, if you try it out, and decide not to follow through, so long as you delete your VPS, you should not be out-of-pocket by more than a few cents.

With this tutorial, I'm assuming you've got a computer with an Internet connection, that can run SSH (all modern systems should do that) and you can copy-and-paste stuff from this tutorial (in your browser) into either a terminal window (in which you're SSH'd into your VPS) or into a text editor. Note, if you find it difficult to paste into a terminal window, try using CTRL+SHIFT+V (CTRL+V is already used as a short-cut for something else in UNIX terminals since long before the Windows world started using CTRL+C and CTRL+V).

When I provide files you need to copy, look for the 'tokens' or placeholders with values you need to substitute (search-and-replace) in square brackets like this, [token name], with your own values. Any text editor worth using should let you do that!

Create a Virtual Private Server

The first step is to create a place to host instances of your own LibreSoftware services.

You can use a local piece of computing hardware of sufficient capacity in your own home or organisation (e.g. a redundant depreciated desktop should be sufficiently powerful), recognising that it needs to be reliable because anyone using your solution will not be able to access it if it's not running. But, doing that is no joke - making reliable systems is hard.

The more cost-effective self-hosting approach in our experience is to lease a low cost commodity Linux Virtual Private Server (aka a VPS) running Ubuntu Linux 22.04 (or the latest "Long Term Support" version - the next one, 24.04, will be released in April 2024). That's what we assume you're running for all our recent tutorials.

We have used quite a few Linux VPSs commodity providers. Known good options include Digital Ocean, Linode, Vultr, Hetzner, and TurnkeyLinux. There are many (hundreds of) other credible options. We recommend you find one hosted in the network epicentre (which isn't necessarily the same as the 'geographic' epicentre) of your audience. In the past year, we shifted almost all of our hosting to Hetzner as they've got the benefit of not being US-owned (They're German, and are therefore less likely to expose us to the egregiously over-reaching US Cloud and Patriot Acts). Shifting to Hetzner also reduced our infrastructure costs by about 50% compared to Digital Ocean. It also represents a 95% savings over Amazon AWS or Microsoft Azure hosting (can't imagine how anyone could justify hosting with either of them).

If you have trouble getting a VPS, you might find this video I created for provisioning a VPS, using Digital Ocean as an example, useful. In my experience, the process for provisioning VPSs on other platforms is very similar. You'll find this process much easier than using either Microsoft Azure or Amazon AWS, which we strongly recommend against using.

VPS Properties:

We recommend that you provision a VPS with the following spec - these will suffice capacity-wise for all the tutorials we provide, although depending on your load, you might want to beef them up, which you can do as the need arises - ou should be able to upgrade those specs in real time if required, except for your disk space. You can, however, provision a secondary storage space (you can start small and increase it as you need to). I will cover setting this up, as it'll make your life far far easier in the medium-long term.

4-8 GB RAM - system volatile memory for running applications
2-4 Virtual CPUs - processing capacity
80-160 GB Disk space (NVMe storage is faster than SSD which is faster than spinning disk space) - long term storage for data
running Ubuntu Linux 22.04 (the current Long Term Support version) - the operating system
extra storage - 20-40GB extra space (can be expanded on fairly short notice) - a long term storage option separate to your operating system drive - this is very very useful in the event that your app inadvertently fills your hard drive. You'll thank yourself for doing this (I an assure you from bitter experience). It's always a good idea to learn from others' past mistakes rather than repeat them!

You'll need to create an account for yourself on your chosen hosting provider (it's a good idea to use Two Factor Authentication, aka 2FA, on your hosting account so that no one can log in as you and, say, delete your server unexpectedly - you'll find instructions on how to set up 2FA on your hosting provider's site) and create an Ubuntu 22.04 (or the most recent 'Long Term Support' (LTS) version) in the 'zone' nearest to you (or your primary audience, if that's different).

If you don't already have an SSH key on your computer, I encourage you to create one and specify the public key in the process of creating your server - specifying the 'public key' of your SSH identity during the server creation process that should allow you to log in without needing a password!

You'll need to note the server's IPv4 address (it'll be a series of 4 numbers, 0-254, separated by full stops, e.g. 103.99.72.244), and you should also be aware that your server will have a newer IPv6 address, which will be a set of 8 four hex character values (each hex character can have one of 16 values: 0-9,A-F) separated by colons, e.g. 2604:A880:0002:00D0:0000:0000:20DE:9001. With one or the other of those IPs, you should be able to log into your new server via SSH. If you're on a UNIX command line (e.g. a Linux or MacOS desktop), do this in a terminal (On Windows, I understand people use a tool called Putty for SSH, in which case follow the app's instructions - or you'll find many tutorials on using it with a quick web search):

ssh [your server IPv4 or IPv6]

followed by the ENTER key (that'll be true for any line of commands I provide).

In some cases, depending on your hosting provider, you'll have a password to enter, or if you've specified your pre-existing public SSH key, you shouldn't need to enter a password at all, you should be logged in. To check what user you care, you can type

whoami

If it returns root (there's also a convention of using a '#' as the command prompt), you're the root or super-admin of the server. If not, you're a normal user (some hosting providers have a convention of giving you a default user called "ubuntu" or perhaps "debian") with a prompt that is, by convention, a '$'.

Now that you're logged in, it's worth doing an upgrade of your server's Ubuntu system! Do that as follows (this works regardless of whether your a root user or an unprivileged user with 'sudo' ability):

sudo apt update && sudo apt dist-upgrade

Usually the user, even if it's not the root user, will have the ability to use the sudo command modifier - that means "do this action as the root (aka the 'Super User', thus 'su' in 'sudo' for short) user" - if you're a non-root user, you'll likely be asked to enter your password as a security precaution the first time you run a command prefaced by sudo. Enter it, and it should run the command. Plus, the system shouldn't bother you for it again unless you leave your terminal unused for a while (usually 5 minutes) and come back to it.

At this point, I also like to install a cool software package called 'etckeeper' which records configuration changes on your VPS for future reference (it can be life-saving if trying to recover from an administrative mess-up!):

sudo apt install etckeeper git

which will also install some dependencies, including the very important (and relevant later on) 'git' version control system.

Key variables for you VPS

To set up your services, you'll need a few crucial bits of information related to your system's identity and external systems you'll need it to interact with. For example, as mentioned before, you'll need a domain name. For the rest of this tutorial, we'll use the convention of representing those variables as a name inside [], for example, the domain name you've picked, [domain name].

Here's a list of variables you'll need to know to complete the rest of this tutorial:

[ipv4] and [ipv6] - your VPS' IPv4 and IPv6 addresses (the latter can be ignored if your cloud provider doesn't support IPv6 addresses) as described above.
[domain name] - the fully qualified domain names or subdomains of a base [domain name] by which you want your services to be accessed. You must have full domain management ability on this domain. Example: nextcloud.oeru.org - that's the nextcloud subdomain of the oeru.org domain.
Authenticating SMTP details - this is required so your services can send emails to users - crucial things like email address validation and password recovery emails...
- [smtp server] - the domain name or IPv4 or IPv6 address of an SMTP server
- [smtp port] - the port number on the server that is listening for your connection. By convention it's likely to be 465 or 587, or possibly 25.
- [smtp reply-to-email] - a monitored email to which people can send email related to this WordPress site, e.g. notifications@[domain name]
- [smtp user] - the username (often an email address) used to authenticate against your SMTP server, provided by your email provider.
- [smtp password] - the accompanying password, provided by your email provider.
[your email] - an email address to which system-related emails can be sent to you, perhaps something like webmaster@[domain name].
[vps username] - the username you use on your server (by convention, these are one word, and all lower case).

Get your Domain lined up

You will want to have a domain to point at your server, so you don't have to remember the IP number. There're are thousands of domain "registrars" in the world who'll help you do that... You just need to "register" a name, and you pay yearly fee (usually between USD10-30 depending on the country and the "TLD" (Top Level Domain. There're national ones like .nz, .au, .uk, .tv, .sa, .za, etc., or international domains (mostly associated with the US) like .com, .org, .net, and a myriad of others. Countries decide on how much their domains wholesale for and registrars add a margin for the registration service).

Here in NZ, I use the services of Metaname (they're local to me in Christchurch, and I know them personally and trust their technical capabilities). If you're not sure who to use, ask your friends. Someone's bound to have recommendations (either positive or negative, in which case you'll know who to avoid).

Once you have selected and registered your domain, you can 'manage your Zone' to set up (usually through a web interface provided by the registrar) an A Record which associates your website's name to the IPv4 address of your server. So you should just be able to enter your server's IPv4 address, the domain name (or sub-domain) you want to use for the web service you want to set up.

Nowadays, if your Domain Name host offers it (some don't, meaning you might be better off with a different one), it's also important to define an IPv6 record, which is called an AAAA Record... you put in your IPv6 address instead of your IPv4 one.

You might be asked to set a "Time-to-live" (which has to do with the length of time Domain Name Servers are asked to "cache" the association that the A Record specifies) in which case you can put in 3600 seconds or an hour depending on the time units your registrar's interface requests... but in most cases that'll be set to a default of an hour automatically.

Editing files

In the rest of this tutorial, we're going to be editing quite a few files via the command line. If you're new to this, I recommend using the 'nano' text editor which is installed by default on Ubuntu Linux systems. It's fairly simple, and all of its options are visible in the text-based interface. I tend to use a far more powerful but far less beginner-friendly editor called 'vim'. There're other editors people might choose, too. To use your preferred editor for the rest of the tutorial, enter the following to set an environment variable EDIT, specifying your preferred editor, e.g.:

EDIT=$(which nano)

or, if you're like me

EDIT=$(which vim)

so that subsequent references to $EDIT will invoke your preferred editor. Note the command $(which nano) is a script which finds the full path to the named command, in this case 'nano'. Putting a command inside the $() means 'replace with the value the script returns', so it sets the value of EDIT to the path of the nano command in this case. On my current machine, the value it returns is /usr/bin/nano, which is pretty typical.

To test (at any time) whether you session still knows your $EDIT command, run

echo $EDIT

if it returns the path to your preferred editor, you're good to go. If not, just reassert (run again) the EDIT= line from above!

Note: if you log out and back in again, change users, or create a new terminal tab/session, you'll need to reassert the EDIT value.

Set up an unprivileged user for yourself

This will log you into your server as it did the first time, either as 'root' or the default unprivileged user. It's not considered good practice to access your server as root (it's too easy to completely screw it up by accident). It's a good idea to create your own separate 'non-root' user who has 'sudo' privileges and the ability to log in via SSH. If you are currently logged in as 'root', you can create a normal user for yourself via (replace [vps username] with your chosen username - in my case, I'd use U=dave):

U=[vps username]
adduser $U
adduser $U ssh
adduser $U admin
adduser $U sudo

You'll also want to a set a password for user [vps username] (we have a tutorial on creating good passwords):

passwd $U

then become that user temporarily (note, the root user can 'become' another user without needing to enter a password) and create an SSH key and, in the process, the .ssh directory (directories starting with a '.' are normally 'hidden' - you can show them in a directory listing via ls -a) for the file into which to put your public SSH key:

su $U

after which you need to re-run your EDIT command: EDIT=$(which nano)

and then run ssh-keygen -t rsa -b 2048
$EDIT ~/.ssh/authorized_keys

and in that file, copy and paste (without spaces on either end) your current computer's public ssh key (never publish your private key anywhere!), save and close the file.

and then leave the 'su' state, back to the superuser:

CTRL+D or type exit

From that point, you should be able to SSH to your server via ssh [vps username]@[domain name] without needing to enter a password.

These instructions use 'sudo' in front of commands because I assume you're using a non-root user. The instructions will still work fine even if you're logged in as 'root' (the 'sudo' will be ignored as it's unnecessary).

Configure the VPS

First things first. Let's make sure you've got the time zone set appropriately for your instance. It'll probably default to 'UTC' (Greenwich Mean Time). For our servers, I tend to pick 'Pacific/Auckland' which is our time zone. Run this

sudo dpkg-reconfigure tzdata

and pick the appropriate timezone. You can just leave it running UTC, but you might find it tricky down the track if, for example, you're looking at logs and having to constantly convert the times into your timezone.

Configuring your firewall

In the name of safety from the get-go, let's configure our firewall. We work on the basis of explicitly allowing in only what we want to let in (i.e. a 'default deny' policy).

First we'll enable the use of SSH through the firewall (not doing this could lock us out of your machine!)

sudo ufw allow ssh

while we're here, we'll also enable data transfer from the internal (to the VPS) Docker virtual network and the IP range it uses for Docker containers:

sudo ufw allow in on docker0
sudo ufw allow from 172.0.0.0/8 to any

Then we'll enable forwarding from internal network interfaces as required for Docker containers to be able to talk to the outside world:

sudo $EDIT /etc/default/ufw

and copy the line DEFAULT_FORWARD_POLICY="DROP" tweak it to look like this (commenting out the default, but leaving it there for future reference!):

#DEFAULT_FORWARD_POLICY="DROP"
DEFAULT_FORWARD_POLICY="ACCEPT"

and then save and exit the file (CTRL-X and then 'Y' if your editor is nano).

You also have to edit /etc/ufw/sysctl.conf

sudo $EDIT /etc/ufw/sysctl.conf

and remove the "#" at the start of the following lines, so they look like this:

# Uncomment this to allow this host to route packets between interfaces
net/ipv4/ip_forward=1
net/ipv6/conf/default/forwarding=1
net/ipv6/conf/all/forwarding=1

Then we need to restart the network stack to apply that configuration change

sudo systemctl restart systemd-networkd

(on older Ubuntu systems this would have been done via sudo service networking restart...)

Next we have to enable the (default on Ubuntu) UFW firewall to start at boot time to keep your machine relatively safe.

sudo $EDIT /etc/ufw/ufw.conf

And set the ENABLED variable near the top:

ENABLED=yes

Now you can formally start UFW now:

sudo ufw enable

Install the Nginx

Next we need to install the Nginx web server and reverse-proxy, as well as the Let's Encrypt SSL certificate generator, both of which are crucial for any secure web services you might want to host. Nginx is a more efficient and flexible alternative to the older Apache web server you might've seen elsewhere (Nginx recently surpassed Apache as the most widely used web server on the Internet).

sudo apt install nginx-full letsencrypt ssl-cert

You'll get a couple pop-up windows in your terminal, just hit ENTER to accept the defaults. Having installed it, we need to create firewall rules to allow external services to see it:

sudo ufw allow 'Nginx Full'

You can check if the firewall rules you requested have been enabled:

sudo ufw status

Outgoing VPS Email (optional)

Although it's not absolutely necessary (you can do this section later if you're in a big hurry), it's very useful for your server to be able to send out emails, like status emails to administrators (perhaps you) about things requiring their attention, e.g. the status of backups, pending security updates, expiring SSL certificates, etc.

To do this, we'll set up the industrial strength Postfix SMTP server, which is pretty quick and easy. First we install Postfix and a command line mail client for testing purposes.

sudo apt install postfix bsd-mailx

During the install, you'll be asked to select a bunch of configuration parameters. Select the defaults except:

Select "Internet Site with Smarthost",
fill in the domain name for your server [domain name],
the [smtp server] name and [smtp port] (in the form [smtp server]:[smtp port], e.g. smtp.oeru.org:587 ) of your "smarthost" who'll be doing the authenticating SMTP for you, and
the email address to which you want to receive system-related messages, [your email].

After that's done, we set a default address for the server to mail to, to [your email] selected above. First

sudo $EDIT /etc/aliases

We need to make sure the "root" user points to a real email address. Add a line at the bottom which says (replacing [your email] with your email :) )

root: [your email]

After which you'll need to convert the aliases file into a form that postfix can process, simply by running this:

sudo newaliases

Then we have to define the authentication credentials required to convince your mail server that you're you!

sudo $EDIT /etc/postfix/relay_password

and enter a single line in this format:

[smtp server] [smtp user]:[smtp password]

as an example, this is more or less what I've got for my system. Note that the [smtp user] in my case is an email address (this is common with many smtp system - the user is the same as the email address):

smtp.oeru.org smtp-work@fossdle.org:YourObscurePassw0rd

then save the file and, like the aliases file, run the conversion process (which uses a slightly different mechanism):

sudo postmap /etc/postfix/relay_password

Finally, we'll edit the main configuration file for Postfix to tell it about all this stuff:

sudo $EDIT /etc/postfix/main.cf

If your SMTP server uses port 25 (the default for unencrypted SMTP) you don't have to change anything, although most people nowadays prefer to use StartTLS or otherwise encrypted transport to at least ensure that your SMTP authentication details (at least) are transferred encrypted. That means using port 587 or 465. If you're using either of those ports, find the "relayhost = [your server name]" line... and add your port number after a colon, like this

relayhost = [smtp server]:[smtp port]

or, for example:

relayhost = smtp.oerfoundation.org:465

Then we have to update the configuration for Postfix to ensure that it knows about the details we've just defined (this command will automatically back up the original default configuration so you can start from scratch with the template below):

sudo mv /etc/postfix/main.cf /etc/postfix/main.cf.orig && sudo $EDIT /etc/postfix/main.cf

You can just copy-and-paste the following into it, substituting your specific values for the [tokens]. Note: the IPv6 designations in the line mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128 are not tokens - you can leave those unchanged.

# See /usr/share/postfix/main.cf.dist for a commented, more complete version
 
# Debian specific:  Specifying a file name will cause the first
# line of that file to be used as the name.  The Debian default
# is /etc/mailname.
#myorigin = /etc/mailname
 
smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu)
biff = no
 
# appending .domain is the MUA's job.
append_dot_mydomain = no
 
# Uncomment the next line to generate "delayed mail" warnings
#delay_warning_time = 4h
readme_directory = no
 
# See http://www.postfix.org/COMPATIBILITY_README.html -- default to 3.6 on
# fresh installs.
compatibility_level = 3.6
 
# TLS parameters
smtpd_tls_cert_file=/etc/ssl/certs/ssl-cert-snakeoil.pem
smtpd_tls_key_file=/etc/ssl/private/ssl-cert-snakeoil.key
smtpd_tls_security_level=may
 
smtp_tls_CApath=/etc/ssl/certs
#smtp_tls_security_level=may
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache
 
smtpd_relay_restrictions = permit_mynetworks permit_sasl_authenticated defer_unauth_destination
myhostname = [domain name]
alias_maps = hash:/etc/aliases
alias_database = hash:/etc/aliases
myorigin = /etc/mailname
mydestination = $myhostname, localhost
relayhost = [smtp server]:[smtp port]
mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128
mailbox_size_limit = 0
recipient_delimiter = +
inet_interfaces = all
inet_protocols = all
 
# added to configure accessing the relay host via authenticating SMTP
smtp_sasl_auth_enable = yes
smtp_sasl_password_maps = hash:/etc/postfix/relay_password
smtp_sasl_security_options = noanonymous
smtp_tls_security_level = encrypt
 
# if you're using Ubuntu prior to 20.04, uncomment (remove the #) the
# earlier line smtp_tls_security_level = may to save errors in 'postfix check'
# and comment this line (by adding a # at the start)
smtp_tls_wrappermode = yes

Once you've created that main.cf file, you can double check that your config is valid:

sudo postfix check

and if it's all ok, you can get Postfix to re-read its configuration:

sudo postfix reload

You can then try sending an email so see if it works!

By default, a command line application called "mail" is installed as part of the bsd-mailx package we installed alongside postfix. You can use it to send test email from the command line on your host to verify you've got things working correctly! The stuff in <> are the keys to hit at the end of the line...

mail you@email.domain<ENTER>

Subject: Testing from your.relay.server.domain<ENTER>
Testing postfix remote host<ENTER>
<CTRL-D>
Cc:<ENTER>

Typing (hold down the Control or Ctrl key on your keyboard and press the "d" key) will finish your message, showing you a "CC:" field, in which you can type in other email addresses if you want to test sending to multiple addresses. When you then hit , it will attempt to send this email. It might take a few minutes to work its way through to the receiving email system (having to run the gauntlet of spam and virus filters on the way).

You can also always check the postfix system logs to see what postfix thinks about it using the command:

sudo less +G /var/log/mail.log

if your system doesn't have a /var/log/mail.log, never fear! Try this instead:

sudo less +G /var/log/syslog

In either case, hit to have the log update in real time.

Installing the Docker Engine, Docker Compose, and Let's Encrypt

First let's install the Docker Engine, which (these days) comes with Docker Compose and the Let's Encrypt scripts that let you procure no-cost Secure Sockets Layer certificates to secure access to your server. You can follow the official Docker Engine install instructions, but I've summarised them here (If the following doesn't work for you, go back to the official instructions, because something might've changed since I wrote this).

First we want to make sure no old Docker engines are installed on this server (this probably won't do anything, but no harm in running it):

for pkg in docker.io docker-doc docker-compose docker-compose-v2 podman-docker containerd runc; do sudo apt-get remove $pkg; done

Second, we want to set up Docker's 'APT' repository, so you can keep Docker up-to-date with their latest versions (usually more up-to-date than those shipped with Ubuntu):

First we Add Docker's official GPG key (copy and paste all of this at your command line):

sudo apt-get update
sudo apt-get install ca-certificates curl gnupg
sudo install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
sudo chmod a+r /etc/apt/keyrings/docker.gpg

Then we have to add the repository to Apt our system's sources and install the Docker Engine:

echo \
  "deb [arch="$(dpkg --print-architecture)" signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \
  "$(. /etc/os-release && echo "$VERSION_CODENAME")" stable" | \
  sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt-get update
sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

With this approach, any updates made by the Docker community will be installed as part of your regular server upgrades.

Having installed the most recent release of the Docker engine, you should find that you've now got Docker Compose built in. Test that by running

docker compose version

which should return something like

Docker Compose version v2.18.1

If that's the case, superb, everything's great.

Backwards compatibility for Docker Compose

Since version 2 of the Docker Engine, the 'Docker Compose' capability has been incorporated into the base system. Prior to version 2, using Docker Compose required installing a separate app, and it was run by typing docker-compose rather than docker compose... so something I do now, to accommodate my muscle memory of typing docker-compose is to create a tiny script that lets me keep using that command, but have it call, instead, the new docker compose functionality. I do this via

sudo $EDIT /usr/local/bin/docker-compose

into which I put the following:

#!/bin/bash
D=`which docker`
$D compose "$@"

After saving that, we have to make the script 'executable' via

sudo chmod a+x /usr/local/bin/docker-compose

So you should now be able to run

docker-compose version

and get the same result that you did above for docker compose version...

Docker use by non-root user

If the above docker commands didn't work for you... and if you want to run Docker commands without being the root user or using sudo, as we usually do, you need to do a few more steps...

create a 'docker' group on your system (this might already exist, but doing this again won't hurt): sudo groupadd docker
add your user to it: sudo usermod -aG docker $USER
refresh your shell so that it recognises your user's membership in the docker group: newgrp docker

You should now be to run a test as a non-root user docker run hello-world

Setting up places for your Docker configurations and persistent Data

The next step is to set up the file structure for holding your Docker configurations and the data your Docker containers will access. This is my convention, so you're welcome to do things different, but this is a 'known good' approach.

Now we create the set of directories I use for holding Docker Compose configurations (/home/docker) and the persistent data the Docker containers create (/home/data)

D=[domain]
sudo mkdir -p /home/data/$D
sudo mkdir -p /home/docker/$D

It's helpful to make sure that your non-root user can also read and write files in these directories:

U=[vps username]
sudo chown -R $U /home/docker
sudo chown -R $U /home/data

Configuring Nginx reverse proxy for your service

Above, we installed Nginx as well as the Let's Encrypt scripts. Now we'll configure them as it's useful to have them working before you set up your services.

In order for you, outside of your server, to see your specific LibreSoftware services, you will need to set up a secure external 'reverse proxy' on your host VPS which will accept requests for each of those services from the Internet and pass those requests securely to the two sets of Docker containers providing the services. These will answer to https://[domain name], noting that a given VPS can answer on behalf of more than one domain name or subdomain name, where each references a different instance of a service or altogether different services. For example, a given VPS could host multiple services, like, say, a password manager, a WordPress blog, a Mautic email automation system, an Authentik Single-Sign On service, and a Discourse forum, where each has a separate domain name (or sub domain name) and each has an Nginx reverse proxy configuration that directs requests to the appropriate domain to the appropriate set of Docker containers.

We use Let's Encrypt to provide the SSL certificates (each is a file with a specially generated, very long string) which we use to limit access to our services to encrypted (secure) connections (protecting both our users and ourselves from external enemies), usually one for each service domain name and corresponding Nginx configuration file. Some Nginx configurations will have multiple domains names for which a given service accepts connections, but it consolidates them to the 'canonical' (official) domain name. For example, the OER Foundation's WordPress website will respond to any of the following:

but all requests to any of those 4 options (note the difference between http:// and https:// where 's' refers to 'Secure' or encrypted) will all be redirected - transparently by the Nginx configuration, which will be reflected in your browser's address bar - to the canonical web address (or URL), https://oerfoundation.org, that we've chosen because it's short and secure.

Nginx will not run unless the SSL certificates you reference in your configurations are valid. Given that we need to request them with a working Nginx prior to them being created puts us in an awkward position. We use a trick to get around it: we temporarily reference the default 'self-signed' SSL certificates (sometimes called 'Snakeoil certs' because that's the placeholder name they're given) that every new Linux system generates when it's installed, that are valid certificates (and thus acceptable to Nginx) but *they won't work with our domains, as they're generic and not 'signed' by an external party, like Let's Encrypt, meaning that your browser won't like them. But that's ok, as you browser will never need to see them, and Let's Encrypt's systems won't look at them either. We'll swap the Snakeoil certs out as soon as we've successfully created the Let's Encrypt ones, and your browser will be happy, and all will be well with the world.

This should happen automatically when you install the Let's Encrypt packages, but just to be sure, run this (it won't harm anything if you run it more than once):

sudo make-ssl-cert generate-default-snakeoil

which creates your default 'Snakeoil' certificates. They are technically valid certificates, but they aren't signed by any

Let's Encrypt setup

Step one of using Let's Encrypt is to make sure the Let's Encrypt scripts are installed (note: sometimes they're referred to as 'certbot' - it's the same code):

sudo apt install letsencrypt

Let's Encrypt and Nginx need to work together. Nginx stores all of its configuration in the directory /etc/nginx. The first thing we'll do is create a place for Let's Encrypt Nginx-specific configuration details:

sudo mkdir /etc/nginx/includes

Then we create that configuration file itself:

sudo $EDIT /etc/nginx/includes/letsencrypt.conf

into which we copy-and-paste the following (no [tokens] to replace in this one!)

# Rule for legitimate ACME Challenge requests
location ^~ /.well-known/acme-challenge/ {
    default_type "text/plain";
    # this can be any directory, but this name keeps it clear
    root /var/www/letsencrypt;
}
 
# Hide /acme-challenge subdirectory and return 404 on all requests.
# It is somewhat more secure than letting Nginx return 403.
# Ending slash is important!
location = /.well-known/acme-challenge/ {
    return 404;
}

As described in the file we've just created, Let's Encrypt will look for a secret code we create to verify that we own the domain we're requesting an SSL certificate for, so we have to make sure it exists:

sudo mkdir /var/www/letsencrypt

You will need to install an Nginx reverse proxy configuration file for each web-based service you run on your VPS, but those will usually be service-specific in their make up, so I won't discuss them here.

But, for the record (to make your lives a bit easier), this is what you'll do after you've got a suitable Nginx reverse proxy configuration in place and ready to run to acquire a Let's Encrypt SSL certificate.

Requesting Let's Encrypt certificates

To request Let's Encrypt SSL certificates for your service, run the following, replacing the [domain name] reference with the address you've selected for your service. Note that 'certbot' is the script provided by the Let's Encrypt package - historically, it could also be called via 'letsencrypt', although apparently the latter is now deprecated:

sudo certbot --webroot -w /var/www/letsencrypt -d [domain name]

Note - if you want to address your instance from multiple domains, use one (or more) -d [another domain] - just make sure that

all those domains already point to your VPS, and
those domains are included in the Nginx proxy configuration above.

otherwise the Let's Encrypt certbot request will fail!

Here's what you're likely to see as output from the first run of the letsencrypt script - note that it will ask you for an email address (so it can send you warnings if your certificate is going to expire, e.g. due to a problem with renewal (like if you make a configuration change that breaks the renewal process)).

Saving debug log to /var/log/letsencrypt/letsencrypt.log
Enter email address (used for urgent renewal and security notices)
 (Enter 'c' to cancel): webmaster@fossdle.org
 
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Please read the Terms of Service at
https://letsencrypt.org/documents/LE-SA-v1.3-September-21-2022.pdf. You must
agree in order to register with the ACME server. Do you agree?
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
(Y)es/(N)o: y
 
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Would you be willing, once your first certificate is successfully issued, to
share your email address with the Electronic Frontier Foundation, a founding
partner of the Let's Encrypt project and the non-profit organization that
develops Certbot? We'd like to send you email about our work encrypting the web,
EFF news, campaigns, and ways to support digital freedom.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
(Y)es/(N)o: y
Account registered.
Requesting a certificate for [domain name]
 
Successfully received certificate.
Certificate is saved at: /etc/letsencrypt/live/[domain name]/fullchain.pem
Key is saved at:         /etc/letsencrypt/live/[domain name]/privkey.pem
This certificate expires on (some future date).
These files will be updated when the certificate renews.
Certbot has set up a scheduled task to automatically renew this certificate in the background.
 
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
If you like Certbot, please consider supporting our work by:
 * Donating to ISRG / Let's Encrypt:   https://letsencrypt.org/donate
 * Donating to EFF:                    https://eff.org/donate-le
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Ideally, you'll see a message like the above. If not, and there's an error, the error messages they provide are usually very useful and accurate. Fix the problem and try again. Note, your SSL certificate will have the name of your [domain name], even if it also provide support for [second domain name] (or third, fourth, etc.).

Once you have a Let's Encrypt certificate, you can update our Nginx configuration:

sudo $EDIT /etc/nginx/sites-available/[domain name]

and swap all occurrences of

    ssl_certificate /etc/ssl/certs/ssl-cert-snakeoil.pem;
    ssl_certificate_key /etc/ssl/private/ssl-cert-snakeoil.key;
#    ssl_certificate /etc/letsencrypt/live/[domain name]/fullchain.pem;
#    ssl_certificate_key /etc/letsencrypt/live/[domain name]/privkey.pem;

#    ssl_certificate /etc/ssl/certs/ssl-cert-snakeoil.pem;
#    ssl_certificate_key /etc/ssl/private/ssl-cert-snakeoil.key;
    ssl_certificate /etc/letsencrypt/live/[domain name]/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/[domain name]/privkey.pem;

which enables your new domain-specific SSL certificate. Check that Nginx is happy with your change:

sudo nginx -t

and if so,

sudo service nginx reload

You domain should now be enabled for https:// access. Note that going to http://[domain name] should automatically redirect you to https://[domain name] because you care about your user's security!

And that's it - your server is now ready to receive specific configurations for a myriad of Libre web services! Have a look around this site and check out some of the other tutorials - for many of them, you'll now be able to skip to near the end, as you've done most of the groundwork already!

Add new comment

Installing BigBlueButton on an OERu Docker Server

dave — Wed, 10 Nov 2021 02:42:41 +0000

Installing BigBlueButton on an OERu Docker Server

Blog tags

BigBlueButton

ubuntu linux

20.04

docker

docker-compose

postgresql

postfix

nginx

let's encrypt

dave Wed 10/11/2021 - 15:42

This tutorial was developed for the Samoan Knowledge Society Initiative, in which the OER Foundation is very pleased to be involved. (Update 2022-02-10: we have also created a video tutorial based on this written tutorial)

BigBlueButton (BBB) is a full-featured video conferencing system designed for large scale remote learning at the university level. In some ways, BBB goes beyond feature parity with widely used proprietary video conferencing systems like Zoom or Microsoft Teams, Webex, or Google Hangouts. It's main differences are that BBB

was designed from the gound up for educational use, and
is Free and Open Source Software (FOSS) and anyone can install it (as we describe in this tutorial) at no cost other than whatever cost you pay for hosting the application and a bit of your time.

You can read all about BigBlueButton and how it works using the excellent documentation provided by its development community.

There are no per-user license fees (or any fees), and your users don't need to install special software, vastly simplifying its adoption: BBB works brilliantly on any desktop computer, laptop, or mobile device (tablet or phone) with a modern web browser that implements the WebRTC open web standard (which is all modern browsers).

The source code for BBB's internal components, and its many ancillary tools is available for anyone to learn from or improve - that's the fundamental benefit of FOSS. Since the outbreak of COVID19, the team at a small Ottowa, Canada-based company, Blindside Networks, has been working tirelessly with the BBB community to improve the BBB code, which is being used all over the world by millions of people every day.

This tutorial walks you through provisioning and building your own BigBlueButton instance on a server running Ubuntu GNU/Linux - version 20.04 is the Long Term Support (LTS) version as of this writing - using Docker containers for the various software services and components, with the containers managed by Docker Compose.

Table of contents

Introduction
- References
- Copying and pasting
Information you need to know
Preferences
Set up your virtual server
Configure your Domain to point at your Server
- Best Practice: access your server as a non-root user
A few post-install updates
- Update the server's hosts file
Firewall Configuration
Install Postfix so the server can send email
- Email test
Install Nginx webserver and Let's Encrypt tools
Set up COTURN certificates
Set up Nginx Reverse Proxy Configuration
- Extra security with dhparam.pem
Install Docker
Install Docker-compose
Git-Clone BBB Docker repository
Configure your BBB
- Fix minor configuration error that blocks JODConverter
Build your BBB
Visit your new BBB
Create an admin user:
Run your first conference
Next Steps

Introduction

References

For this tutorial, we are using a Digital Ocean 'Droplet' as our cloud server. You can use an almost identical process to provision a server from any one of a thousand other commodity GNU/Linux cloud hosting providers.

To demonstrate the process of defining a sub domain and its IPv4 and IPv6 addresses (via A and AAAA records respectively) I'm using my preferred local domain registrar Metaname (based here in Christchurch, New Zealand, as am I. I know the developer/owner of the service and I trust him). You can use any domain registrar that gives you the ability to configure your 'DNS zone file'. Some of might have your own DNS server or institutional name servers, in which case you might have to request these changes be made on your behalf.

Copying and pasting

We will be providing a bunch of command lines that you can copy and paste into a terminal on the computer you're using to access your virtual server. Links like

this one

are intended for you to copy and paste into the command line. On Linux, please note that in some cases if "CTRL-V" doesn't work to paste, try using "CTRL+SHIFT+V" (that's because in some terminals, "CTRL+V" had a pre-existing purpose as terminals have been around since long before desktops and 'copy-paste" were invented and the arbitrary CTRL-C, CTRL-X, CTRL-V key combos were chosen by Microsoft without any consideration for prior art). Similarly, if you want to copy from a terminal, you might need to use "CTRL+SHIFT+C". Try it.

When we ask you edit a file (using the editor you choose to assign to the EDIT shell variable below), we expect you to complete the recommended changes, and then save the file and exit back to the command line.

We'll aim to show you what to change using a 'code' box:

Like this one
which is suitable
for multi-line
content (like what you 
find in a file)

Information you need to know

Info you need to have handy to complete this tutorial:

You need to know the IPv4 address of your server. Token: [IPv4] - it'll be in the form of nnn.nnn.nnn.nnn where nnn = 0-254, example value 143.110.228.88
If your server has one, you need to know its IPv6 address. Token: [IPv6] - it'll be in the form of hhhh:hhhh:hhhh:hhhh:hhhh:hhhh:hhhh where h = 0-9a-f, i.e. a hex value, example 2604:a880:4:1d0::402:b000
You need to have a domain name (or a sub domain name) to point at your server's IPv4 and IPv6. Token: [Domain], example domain bbbtest.milll.ws (in this case, bbbtest is the subdomain of the milll.ws domain)
The email address your server should send status or admin-related email to. Token [Admin email], example webmaster@[Domain]
You'll need a set of authenticating SMTP account settings including - these can be used both on the host and in the BBB installation:
- [SMTP server] - the domain name or IP address of an existing SMTP server, e.g. about.oerfoundation.org (our one), or smtp.googlemail.com,
- [SMTP port] - the port number for the service you're using. It'll usually be one of 587 or 465. Older, unauthenticated SMTP servers used to use port 25, but that's now blocked by most ISPs due to its abuse by spammers.
- [SMTP security] - usually this'll be something like 'SSL' (usually associated with port 465) or 'StartTLS' (usually associated with port 587).
- [SMTP username] - the username - often an email address - for the authenticating SMTP service, and lastly,
- [SMTP password] - the password accompanying the username.
You'll want an email address that you can check to send test emails to: token [Test email], e.g. you@youremailprovider.tld, and finally
You'll want a 'from' email address that users of your system will see when they get emails from you. Token: [Outgoing email], e.g. notifications@milll.ws.

Preferences

You'll need to copy and paste the following lines into any terminal you're using to complete this tutorial, so that that session is aware of your preferred values:

EDIT=$(which nano)

SITE=bbbtest.milll.ws

We're going to reference these periodically in the following process, so it's important you set these correctly. To verify them, you can run this in any terminal at any time to verify that the values are still defined and current:

echo "Our variables = $EDIT and $SITE"

Set up your virtual server

You'll need to log into your cloud service provider's dashboard - in our case, we've used DigitalOcean. That means pointing our browser to https://digitalocean.com and either using "Log In" (if we already have an account) or "Sign Up" (if we don't).

Once you're logged in, you'll want to go to 'Create' and select 'Droplets' (Digital Ocean calls each of their virtual servers a "Droplet").

We've included a screenshot of the Create Droplet page, with the relevant options selected. They are as follows:

Choose an image: we choose 'Ubuntu 20.04 LTS'
Choose a plan: we choose 'Basic'
- for CPU options: we choose 'Premium AMD with NVMe SSD'
- for the size, we choose either the '$24/mo' (choose the '$48/mo' option if you want to be able to support large groups, like 100+ simultaneous participants).
Choose a datacenter region: we choose 'San Franciso, zone 3' (which we've determined to be in the 'centre of the internet', i.e. the most central point to give the best website performance to a global audience. You might want to try a different place depending on your location in the globe and your audience).
VPC Network - we just leave the default.
Select additional options: we check 'IPv6' and 'Monitoring' but not User data.
Authentication: if we have added one or more SSH keys to our DigitalOcean profile select 'SSH keys'. If not, you can select 'Password'. It's far more efficient and secure to use SSH whereever possible.
We're ony creating 1 Droplet and we Choose a hostname of [Domain]
We don't need to add any tags
We won't Add backups.

Then we can Create Droplet...

When the Droplet is finished, we can get the [IPv4] and [IPv6] addresses.

We can also either log in via the command line from a local GNU/Linux or UNIX (e.g. MacOS) terminal or via a graphical SSH client (on Microsoft Windows, most people use Putty) via ssh root@[IPv4] or ssh root@[IPv6] (replacing those tokens with their actual IPv4 or IPv6 addresses). If we've set a key, we should get logged in without the need to enter any passwords. If not, we have to enter a password provided by DigitalOcean.

Configure your Domain to point at your Server

Once you have selected and registered your domain with a domain registrar - we are using Metaname - you can set up (usually through a web interface provided by the registrar)

an "A Record" which associates your website's name (your [Domain], e.g. milll.ws, or a subdomain of that domain, e.g. bbb.milll.ws, where 'bbb' is the subdomain part) to the IPv4 address of your server. You should just be able to enter your server's IPv4 address, the domain name (or sub-domain) you want to use for your server.
if your registrar offers it it's also important to define an IPv6 record, which is called an "AAAA Record"... you put in the same domain name or subdomain and then your IPv6 address instead of your IPv4 one.

I've attached an screenshot of the Metaname interface for configuring these DNS zone records.

You might be asked to set a "Time-to-live" (which has to do with the length of time Domain Name Servers are asked to "cache" the association that the A Record specifies) in which case you can put in 3600 seconds or an hour depending on the time units your interface requests... but in most cases that'll be set to a default of an hour automatically.

Once your domain A and AAAA records are configured, you should be able to log into your server via ssh root@[Domain], or, for example, ssh root@bbb.milll.ws. Although it should be instant, depending on your registrar, it might take as long as a few hours (even 24) for your domain name assignments to propogate through the DNS network and be available on your computer.

Best Practice: access your server as a non-root user

You should be able to test that your A and AAAA Records have been set correctly by logging into your server via SSH using your domain name rather than the IPv4 or IPv6 address you used previously. It should (after you accept the SSH warning that the server's name has a new name) work the same way your original SSH login did. On Linux, you'd SSH via a terminal and enter ssh root@[domain name]. I think you can do similar on MacOS and on Windows, I believe people typically use software called Putty...

But this will log you into your server as the 'root' user.

It's not considered good practice to access your server as root (it's too easy to completely screw it up). Best practice is to create a separate 'non-root' user who has 'sudo' privileges and the ability to log in via SSH. If you are currently logged in as 'root', you can create a normal user for yourself via (replace [username] with your chosen username):

U=[username]
adduser $U

You will be required to set a password for the user here. Then you can add the new user to several relevant groups that confer the required administrative capabilities.

adduser $U ssh
adduser $U admin
adduser $U sudo

If you want to change the password for user [username] you can run:

passwd $U

then become that user temporarily (note, the root user can 'become' another user without needing to enter a password) and create an SSH key and, in the process, the .ssh directory (directories starting with a '.' are normally 'hidden') for the file into which to put your public SSH key:

su $U
ssh-keygen -t rsa -b 2048
nano ~/.ssh/authorized_keys

and in that file, copy and paste (without spaces on either end) your current computer's public ssh key (never publish your private key anywhere!), save and close the file.

From that point, you should be able to SSH to your server via ssh [username]@[domain name] without needing to enter a password.

These instructions use 'sudo' in front of commands because I assume you're using a non-root user. The instructions will still work fine even if you're logged in as 'root'.

The rest of the tutorial can be run as your 'sudo-capable' non-root user.

A few post-install updates

Update the server's hosts file

This is required due to some quirks of the auto-detection of your domain name used by BigBlueButton - you'll need it later.

sudo $EDIT /etc/hosts

and add (or make sure it already has) the following:

127.0.1.1 [First part of domain]
127.0.0.1 localhost
[IPv4] [Domain]
[IPv6] [Domain]

For example - my test system has these details:

127.0.1.1 bbbtest
127.0.0.1 localhost
143.110.228.88 bbbtest.milll.ws
2604:a880:4:1d0::402:b000 bbbtest.milll.ws

Do an initial post-install update to the latest software versions:

sudo apt-get update && sudo apt-get dist-upgrade

and add a couple useful apps that, for example, track changes to our system configuration for future reference (etckeeper) and allow us to do network troubleshooting (net-tools):

sudo apt-get install -y etckeeper net-tools

We also want to enable console logins in the event we have trouble using SSH:

wget -qO- https://repos-droplet.digitalocean.com/install.sh | sudo bash

Firewall Configuration

We need to configure firewall for admin access via SSH and access to internet for Docker containers

sudo ufw allow OpenSSH

sudo ufw allow in on docker0

sudo ufw allow from 172.0.0.0/8 to any

Then we need to adjust the default firewall policy:

sudo $EDIT /etc/default/ufw

You'll find this:

# Set the default forward policy to ACCEPT, DROP or REJECT.  Please note that
# if you change this you will most likely want to adjust your rules
DEFAULT_FORWARD_POLICY="DROP"

which you need to change to

# Set the default forward policy to ACCEPT, DROP or REJECT.  Please note that
# if you change this you will most likely want to adjust your rules
DEFAULT_FORWARD_POLICY="ACCEPT"

Next you need to edit this file

sudo $EDIT /etc/ufw/sysctl.conf

And change (near the top of the file):

# Uncomment this to allow this host to route packets between interfaces
#net/ipv4/ip_forward=1
#net/ipv6/conf/default/forwarding=1
#net/ipv6/conf/all/forwarding=1

# Uncomment this to allow this host to route packets between interfaces
net/ipv4/ip_forward=1
net/ipv6/conf/default/forwarding=1
net/ipv6/conf/all/forwarding=1

(removing the "#" uncomments those lines)

Next, you have to restart the server's networking stack to apply the changes you've made:

sudo systemctl restart systemd-networkd

Luckily, this is generally instantaneous, so your connection to your server shouldn't be interrupted.

Then you either start or (if it's already running for some reason) restart the firewall

sudo ufw enable

sudo service ufw restart

(running both won't do any harm)

And we also update the firewall configuration to tell it to automatically start at boot time.

sudo $EDIT /etc/ufw/ufw.conf

Change ENABLED=no to ENABLED=yes.

Next we need to set up some specialised rules for the "COTURN" functionality of the BigBlueButton system. COTURN is a FOSS application that provides both TURN and STUN (video) (alternative non-video reference) functionality which are core to the WebRTC protocol on which modern video conferencing applications are built, and are necessary for your users to connect their audio and video to a session, particularly if they're behind an institutional firewall.

Note: in some cases, institutions will have implemented a rather over-the-top-paranoid 'default deny' for video conference hosts which will block access to your BBB instance unless you can get them to add your site's domain name to their institutional whitelist. Sadly, there's no easy way around this.

sudo ufw allow 7443/tcp

sudo ufw allow 16384:32768/udp

sudo ufw allow 3478/udp

sudo ufw allow 5349/tcp

sudo ufw allow from 10.7.7.0/24

Ok, that's it for the firewall shenanigans.

Install Postfix so the server can send email

It's generally a good idea to make sure any server you build can send email, and has a sensible default email address to send stuff to: stuff like administrative messages alerting you to failed tasks, or other system problems. We recommend that you designate an email address for whoever's responsible for this server. We've got a more comprehsensive howto for setting this up if you're wanting extra details.

We achieve the outgoing mail functionality using the industrial strength Postfix SMTP server (it's widely used by ISPs for large-scale email services) along with a command line SMTP client that allows us to test our solution from the command line:

sudo apt install postfix bsd-mailx

During this installation, you'll be asked by the installer to make some decisions to do preliminary configuration of the Postfix installation. Select the default asnwers except as follows:

Select "Internet Site with Smarthost",
fill in the [Domain] name you've chosen for your server,
the domain name or IP address and port (in the form [SMTP server]:[port], examples might be smtp.oeru.org:465, or 10.11.143.22:465) of your "smarthost" who'll be doing the authenticating SMTP for you - note: our configuration works with port 465, and
the email address to which you want to receive system-related messages, namely [Admin email].

Once that's installed, we need to set up our default email address for this server:

sudo $EDIT /etc/aliases

This file will containerd

# See man 5 aliases for format
postmaster:    root

update it to include an email address to send stuff intended for the system admin (aka 'root'):

# See man 5 aliases for format
postmaster:    root
 
root: [Admin email]

after writing that, we have to convert that file into a form understood by Postix:

sudo newaliases

Next, we need to create a new file with the SMTP 'relay host' aka 'smart host' details:

sudo $EDIT /etc/postfix/relay_password

In it you need to put the following:

[SMTP Server] [SMTP username]:[SMTP password]

Here's an example:

about.oerfoundation.org demosmtp@milll.ws:7TLM6qGoqZXHfkDmkh6

Once you've set that up, we have to again prepare that file for use by Postfix:

sudo postmap /etc/postfix/relay_password

Finally, we need to update Postfix's main configuration to tell it to use our authenticating SMTP details:

sudo $EDIT /etc/postfix/main.cf

You'll need to make main.cf look like this - specifically commenting out smtp_tls_security_level=may by adding a '#' at the start of the line, and then adding the details below # added to configure accessing the relay host via authenticating SMTP. You should also take this opportunity to confirm that your [Domain], [SMTP server] and [SMTP port] are set correctly.

smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu)
biff = no
 
# appending .domain is the MUA's job.
append_dot_mydomain = no
 
# Uncomment the next line to generate "delayed mail" warnings
#delay_warning_time = 4h
 
readme_directory = no
 
# See http://www.postfix.org/COMPATIBILITY_README.html -- default to 2 on
# fresh installs.
compatibility_level = 2
 
# TLS parameters
smtpd_tls_cert_file=/etc/ssl/certs/ssl-cert-snakeoil.pem
smtpd_tls_key_file=/etc/ssl/private/ssl-cert-snakeoil.key
smtpd_tls_security_level=may
 
smtp_tls_CApath=/etc/ssl/certs
## Commented out for SmartHost configuration
#smtp_tls_security_level=may
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache
 
 
smtpd_relay_restrictions = permit_mynetworks permit_sasl_authenticated defer_unauth_destination
myhostname = [Domain]
alias_maps = hash:/etc/aliases
alias_database = hash:/etc/aliases
myorigin = /etc/mailname
mydestination = $myhostname, bbbtest.milll.ws, localhost.milll.ws, localhost
relayhost = [SMTP server]:465
mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128
mailbox_size_limit = 0
recipient_delimiter = +
inet_interfaces = all
inet_protocols = all
 
# added to configure accessing the relay host via authenticating SMTP
smtp_sasl_auth_enable = yes
smtp_sasl_password_maps = hash:/etc/postfix/relay_password
smtp_sasl_security_options =
smtp_tls_security_level = encrypt
 
# add this if you're using Ubuntu 20.04, and comment out (with a "#") the
# earlier line smtp_tls_security_level = may to save errors in 'postfix check'
# and uncomment this line (by removing the #)
smtp_tls_wrappermode = yes

Here's an example of what the final code could look like (don't use these exact values as they won't work for you):

First, I commented this out:

#smtp_tls_security_level=may

Made sure this was my [Domain]

myhostname = bbbtest.milll.ws

Made sure this had my [SMTP server] and [SMTP port] details:

relayhost = about.oerfoundation.org:465

And I added this at the end:

# added to configure accessing the relay host via authenticating SMTP
smtp_sasl_auth_enable = yes
smtp_sasl_password_maps = hash:/etc/postfix/relay_password
smtp_sasl_security_options =
smtp_tls_security_level = encrypt
 
# add this if you're using Ubuntu 20.04, and comment out (with a "#") the
# earlier line smtp_tls_security_level = may to save errors in 'postfix check'
# and uncomment this line (by removing the #)
smtp_tls_wrappermode = yes

After this, the Postfix configuration is done. We can check that we haven't got any typos via

sudo postfix check

If not, we can apply our configuration changes:

sudo postfix reload

and we can confirm it all worked correctly by checking the log file for Postfix:

sudo less +G /var/log/mail.log

Note: you might see a warning like this: postfix/postfix-script: warning: symlink leaves directory: /etc/postfix/./makedefs.out - it's spurious and you don't need to worry about it.

Email test

Next we can test our outgoing SMTP by sending a test message via the command line - use a test email that you can check!:

mail [Test email]

for example:

mail demo@oerfoundation.org

After you hit Enter, you'll be shown

Subject: {Enter an email subject, e.g. "Test email from [Domain]"}

After that, hit Enter, and you'll be shown a blank line. On that line:

{Enter your email text, e.g. "This is just a test."}

Then, to finish your email, type CTRL-D which will show you another line

CC: {Enter any CC email addresses, e.g. mybackupemail@anotherprovider.tld }

After you hit Enter, your email should be sent if all your configuration options are accepted by your remote host (the SMTP server at the address [SMTP server] with all the other details you entered).

You can check if it worked by looking at the Postfix log again:

sudo less +G /var/log/mail.log

If it sent the email, you'll see something like (but with different IP addresses, serial numbers, and addresses) - the key bit is the status=sent bit and the 250 code from the SMTP server:

08FBD2B501C: to=<demo@oerfoundation.org>, relay=dovecot[172.22.1.250]:24, delay=1.3, delays=1.3/0.02/0/0.04, dsn=2.0.0, status=sent (250 2.0.0 <demo@oerfoundation.org> MMokCa2TpWEffAAAPgmdMA Saved)

You can also check for receipt of email and verify receipt (note, if you don't get it quickily, check your email spam folder).

Install Nginx webserver and Let's Encrypt tools

sudo apt install -y nginx-full ca-certificates letsencrypt ssl-cert

You need to tell your firewall to open the ports that the Nginx webserver will use:

sudo ufw allow "Nginx Full"

And then we need to create a special configuration for Let's Encrypt and then an identify verification directory:

sudo mkdir /etc/nginx/includes

sudo mkdir /var/www/letsencrypt

To create the specific configuration, we create this file:

sudo $EDIT /etc/nginx/includes/letsencrypt.conf

And fill it with this information:

# Rule for legitimate ACME Challenge requests
location ^~ /.well-known/acme-challenge/ {
    default_type "text/plain";
    # this can be any directory, but this name keeps it clear
    root /var/www/letsencrypt;
}
 
# Hide /acme-challenge subdirectory and return 404 on all requests.
# It is somewhat more secure than letting Nginx return 403.
# Ending slash is important!
location = /.well-known/acme-challenge/ {
    return 404;
}

We'll reference this file in our Nginx configuration file for the reverse proxy functionality.

Set up COTURN certificates

In preparation for creating our Let's Encrypt SSL certificate for our [Domain], we're going to set up a 'hook' which is triggered when our SSL certificate is created, and it copies our current certificate to a place where the COTURN server can find it.

sudo $EDIT /etc/letsencrypt/renewal-hooks/deploy/coturn.sh

Copy and past the following into your file (replacing [Domain] with your domain name):

#!/bin/bash
 
DOMAIN=[Domain]
DEST=/etc/coturn-ssl
 
#if [[ 1 == 1 ]]; then
if [[ $RENEWED_DOMAINS == *"$DOMAIN"* ]]; then
  cp -L /etc/letsencrypt/live/${DOMAIN}/fullchain.pem $DEST
  cp -L /etc/letsencrypt/live/${DOMAIN}/privkey.pem $DEST
  echo "updated $DOMAIN certificates in $DEST"
fi

Next, make that script executable:

sudo chmod a+x /etc/letsencrypt/renewal-hooks/deploy/coturn.sh

so that it is run automatically anytime the relevant certificate is created or renewed.

After that, we have to create a place for the COTURN-specific certificates to go:

sudo mkdir /etc/coturn-ssl

Set up Nginx Reverse Proxy Configuration

Next we have to set up our Nginx webserver on our virtual server, which will be the Docker container host for our BigBlueButton system. The reverse proxy configuration will allow the virtual server to receive web requests from BBB users and pass them securely through to the right Docker container.

Create a suitable configuration file - at the OERF we use the convention of calling our configuration

sudo $EDIT /etc/nginx/sites-available/$SITE

In this file, you'll past in the following, replacing [Domain] as appropriate - now might be a good idea to try out search and replace in your text editor!

server {
  listen 80;
  # if your host doens't support IPv6, comment out the following line
  listen [::]:80;
  server_name [Domain];
 
  access_log  /var/log/nginx/[Domain].access.log;
  error_log /var/log/nginx/[Domain].error.log;
 
  include includes/letsencrypt.conf;
 
  # redirect all HTTP traffic to HTTPS.
  location / {
    return  302 https://$server_name$request_uri;
  }
}
 
server {
  listen 443 ssl http2 default_server;
  # if your host doens't support IPv6, comment out the following line
  listen [::]:443 ssl http2 default_server;
  server_name [Domain];
 
  # We comment these out *after* we have successfully geneated our Let's Encrypt certificate for [Domain].
  ssl_certificate /etc/ssl/certs/ssl-cert-snakeoil.pem;
  ssl_certificate_key /etc/ssl/private/ssl-cert-snakeoil.key;   
  # we start with these commented out until after we can generate our Let's Encrypt certificate for [Domain]!
  #ssl_certificate /etc/letsencrypt/live/[Domain]/fullchain.pem;
  #ssl_certificate_key /etc/letsencrypt/live/[Domain]/privkey.pem;
  ssl_dhparam /etc/ssl/certs/dhparam.pem;
 
  access_log  /var/log/nginx/[Domain].access.log;
  error_log /var/log/nginx/[Domain].error.log;
 
  location / {
    proxy_http_version 1.1;
    # for BBB 2.4, the port used is 48087. For earlier versions of BBB, it's 8080
    proxy_pass http://127.0.0.1:48087;
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_set_header Upgrade $http_upgrade;
    #proxy_set_header Connection $connection_upgrade;
    proxy_set_header Connection "Upgrade";
    proxy_cache_bypass $http_upgrade;
  }
}

Update 2021-12-30 With the release of BBB 2.4 a few weeks ago, the Docker configuration now has one 'breaking' change: the port has been changed to 48087! So, if you are installing an older version of BBB, the previous default was port 8080. If you get a 502 error after setting up your containers, check to make sure you haven't been bitten by this little detail!

Right - back to the process...

After creating it, we have to 'enable' the configuration:

sudo ln -sf /etc/nginx/sites-available/$SITE /etc/nginx/sites-enabled/

It's also not a bad idea to disable the Nginx default configuration, as it can sometimes interfere with things:

sudo rm /etc/nginx/sites-enabled/default

Extra security with dhparam.pem

We're going to create a 'dhparam' certificate for your configuration. These take a long time to generate - between 10-30 minutes, depending not on the speed of your computer, but on the rate at which it creates 'random events' that allow it to create a suitablely complex random prime number.

sudo openssl dhparam -out /etc/ssl/certs/dhparam.pem 2048

If your time isn't limited, you can increase the size of your dhparam from 2048 to 4096 - it'll take quite a lot longer to create.

Ok - now that we've created our dhparam.pem, referenced in our Nginx configuration for [Domain], we should have everything in place. We can now test Nginx's configuration:

sudo nginx -t

If you don't get any errors or warnings, you can activate the new configuration:

sudo service nginx reload

You server should now successfully respond to your [Domain], but it'll redirect to HTTPS using the default (self-signed) certificates that are valid as far as Nginx is concerned, but your browser won't let you access the page due to those inappropriate certificates. You can test it by putting http://[Domain] into your browser and seeing if it redirects you to https://[Domain] and a 'bad certificate` (or similar) page.

To fix that, we can now generate a Let's Encrypt certificate which will be valid.

sudo letsencrypt certonly --webroot -w /var/www/letsencrypt -d ${SITE}

Since this is your first time running the letsencrypt script, you'll be asked for a contact email (so the Let's Encrypt system can warn you if your certificates are going to be expiring soon!) - use your [Admin email] for this. You can also opt in to allowing them to get anonymous statistics from your site.

Once you've done that, the Let's Encrypt system will verify that you (the person requesting the certificate) also controls the server that's requesting it (using the details specified in the /etc/nginx/includes/letsencrypt.conf file, along with data that the letsencrypt script writes into a special file in the /var/www/letsencrypt directory) and, all going well, you'll see a "Congratulations!" message telling you that you have new certificates for your [Domain].

Now is a good time to check if your renew-hook worked properly - there should now be two files in /etc/coturn-ssl, namely fullchain.pem and privkey.pem. If that's not the case, something might have gone wrong.

Then you can re-edit your Nginx confguration file:

sudo $EDIT /etc/nginx/sites-available/$SITE

and comment out the default certificates and uncomment the [Domain]-specific certificates, like this (we'll assume you've already got your [Domain] substituted!):

 
  ...
 
  # We comment these out *after* we have successfully geneated our Let's Encrypt certificate for [Domain].
  #ssl_certificate /etc/ssl/certs/ssl-cert-snakeoil.pem;
  #ssl_certificate_key /etc/ssl/private/ssl-cert-snakeoil.key;   
  # we start with these commented out until after we can generate our Let's Encrypt certificate for [Domain]!
  ssl_certificate /etc/letsencrypt/live/[Domain]/fullchain.pem;
  ssl_certificate_key /etc/letsencrypt/live/[Domain]/privkey.pem;
  ssl_dhparam /etc/ssl/certs/dhparam.pem;
 
  ...

Once you've got that going, you again check to make sure your Nginx config is valid:

sudo nginx -t

and apply your configruation changes:

sudo service nginx reload

If you now go to http://[Domain] in your browser, you should be redirected to https://[Domain] and you shouldn't get any certificate errors, although you might get a "502" error because the service that reverse proxy configuration is trying to send you to doesn't yet exist! That's what we're expecting.

One more thing - we need to make sure that the coturn.sh renewal hook script has run. We can check to make sure that there are two files, fullchain.pem and privkey.pem in the /etc/coturn-ssl directory. If we start the Docker containers before these files exist, the Docker daemon creates them as directories by default which can lead to all sorts of trickiness. Run

sudo ls -l /etc/coturn-ssl/

and make sure you get a result like this:

-rw-r--r-- 1 root root 5588 Dec 13 08:42 fullchain.pem
-rw------- 1 root root 1704 Dec 13 08:42 privkey.pem

Install Docker

Now we have to set up the Docker container support by first registering a new package source...

sudo apt-get install -y apt-transport-https curl gnupg lsb-release

sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg

sudo echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

updating our list of available packages to include the Docker-related packages...

sudo apt-get update

and then install them:

sudo apt-get install -y docker-ce docker-ce-cli containerd.io

Which installs a bunch of dependencies as well.

To make sure that our non-root user can talk to the Docker server, we need to jump through a couple more hoops:

sudo addgroup docker

sudo adduser dave docker

After doing this, the easiest thing is to log out and log back in again to make sure my user can access the new permissions. To confirm that my user has docker group privileges, I can run

id

which should give a result like

uid=1000(dave) gid=1000(dave) groups=1000(dave),4(adm),27(sudo),1001(docker)

with the latter being the confirmation.

Install Docker-compose

Now we're almost to the BigBlueButton part - we just need to install the Docker Compose framework:

sudo apt install python3-pip

sudo pip install -U pip

sudo pip install -U docker-compose

Once you've done that, you should be able to run docker-compose at your command prompt and it should give you the docker-compose help page. If so, great work! Almost there.

Git-Clone BBB Docker repository

Now it's time to install the set of Docker containers that make up the BigBlueButton stack of coordinated services.

We make a place for the docker configuration and data to live:

sudo mkdir -p /home/docker/

go into it

cd /home/docker

and then we use the magic of 'git' (if you don't know it, this is one of the single most powerful tools programmers use - everyone would benefit from knowing how 'version control' works - git is by far the most widely used version control aka 'source code control' system in the world. It's open source.) to 'clone' the BigBlueButton developers' Docker definitions code:

sudo git clone -b main --recurse-submodules https://github.com/bigbluebutton/docker.git bbb-docker

That command puts all the code into a directory called bbb-docker so let's go there:

cd bbb-docker

and in there, we run this command to gram a second layer of code that is referenced by the first layer we already downloaded in the previous step:

sudo git submodule update --init

Finally, we run this handy script provided by the BBB developers to set up a working Docker Compose configuration:

sudo ./scripts/setup

This will script will ask you some questions about your system. These are the questions and the answers we'll use - note, where I've written [IPv4] and [IPv6] below, you should see the actual IPv4 and IPv6 addresses for your server:

Should greenlight be included? (y/n): y
Should an automatic HTTPS Proxy be included? (y/n): n
Should a coturn be included? (y/n): y
Coturn needs TLS to function properly.
   Since automatic HTTPS Proxy is disabled,
   you must provide a relative or absolute path
   to your certificates.
Please enter path to cert.pem: /etc/coturn-ssl/fullchain.pem
Please enter path to key.pem: /etc/coturn-ssl/privkey.pem
Should a Prometheus exporter be included? (y/n): y
Please enter the domain name: bbbtest.milll.ws
Should the recording feature be included?
   IMPORTANT: this is currently a big privacy issues, because it will 
   record everything which happens in the conference, even when the button
   suggests, that it does not.
   make sure that you always get people's consent, before they join a room!
   https://github.com/bigbluebutton/bigbluebutton/issues/9202
Choice (y/n): y
Is [IPv4] your external IPv4 address? (y/n): y
Is [IPv6] your external IPv6 address? (y/n): y

Once you finish answering these questions, the script creates a file called .env (the leading '.' means it's a 'hidden' file that won't show up in normal directory listings - you have to know it's there, as it holds important system values and shouldn't be deleted.

Configure your BBB

But now, we're going to tweak it, because it holds all the important customised values we need to configure our BigBlueButton service.

sudo $EDIT .env

When you're editing the file, scroll down through it and adjust the values you find as follows.

Uncomment this one

ALLOW_MAIL_NOTIFICATIONS=true

Set the following

SMTP_SERVER=[SMTP server]
SMTP_PORT=[SMTP port]
SMTP_DOMAIN=[Domain]
SMTP_USERNAME=[SMTP username]
SMTP_PASSWORD=[SMTP password]
SMTP_AUTH=plain
SMTP_STARTTLS_AUTO=true
#
SMTP_SENDER=[Outgoing email]

and finally set

DEFAULT_REGISTRATION=invite

This last one makes it possible for you to invite external people to make use of your BigBlueButton instance - they can create a log in and create their own rooms over which they'll have some control. These can be people in your organisation or community for whom you want your BBB to be available as a resource.

Fix minor configuration error that blocks JODConverter

One of the containers used in the BBB stack is called JODConverter which, in standard Free and Open Source Software tradition, makes use of an 'upstream' development by another developer, EugenMayer, which in turn makes use of an upstream development...

The issue is that a recent update (in early December 2021) has broken the process of launching this container. This is a big problem because the JODConverter provides the services of converting uploaded presentations and downloaded documents (like BBB's Public Chat or Shared Notes) in various useful file formats.

Turns out the fix is easy. While still in bbb-docker, Just run

sudo $EDIT mod/jodconverter/Dockerfile

and add the following line to the very bottom of the file:

CMD ["--spring.config.additional-location=optional:/etc/app/"]

Save it, and you're done. I have submitted an issue with the fix I've found to the bigbluebutton/docker project.

Once that configuration is done, it's finally time to...

Build your BBB

The first time it's run, this command will trigger the building of a set of no less than 22 separate Docker containers, each running its own crucial service as part of the BigBlueButton stack.

sudo docker-compose up -d

Note: this process can take a LONG time, like an hour or more depending on your server's internet connection speed.

If you want to see how long it takes the first time, run this instead:

sudo time docker-compose up -d

which will give you a readout of the time the command takes to complete.

Visit your new BBB

Once you next see the command prompt ($ or #), it means that your BBB system is starting up. You can see the status of the containers by running

sudo docker-compose ps

which should give you something that looks like this, once everything is running (it might take a few minutes!):

              Name                            Command                  State                Ports          
-----------------------------------------------------------------------------------------------------------
bbb-docker_apps-akka_1             /bin/sh -c dockerize     - ...   Up                                     
bbb-docker_bbb-web_1               /entrypoint.sh                   Up (healthy)                           
bbb-docker_coturn_1                docker-entrypoint.sh --ext ...   Up                                     
bbb-docker_etherpad_1              /entrypoint.sh                   Up             9001/tcp                
bbb-docker_freeswitch_1            /bin/sh -c /entrypoint.sh        Up                                     
bbb-docker_fsesl-akka_1            /bin/sh -c dockerize     - ...   Up                                     
bbb-docker_greenlight_1            bin/start                        Up             10.7.7.1:5000->80/tcp   
bbb-docker_html5-backend-1_1       /entrypoint.sh                   Up                                     
bbb-docker_html5-backend-2_1       /entrypoint.sh                   Up                                     
bbb-docker_html5-frontend-1_1      /entrypoint.sh                   Up                                     
bbb-docker_html5-frontend-2_1      /entrypoint.sh                   Up                                     
bbb-docker_jodconverter_1          /docker-entrypoint.sh --sp ...   Up                                     
bbb-docker_kurento_1               /entrypoint.sh                   Up (healthy)                           
bbb-docker_mongodb_1               docker-entrypoint.sh mongo ...   Up (healthy)   27017/tcp               
bbb-docker_nginx_1                 /docker-entrypoint.sh ngin ...   Up                                     
bbb-docker_periodic_1              /entrypoint.sh                   Up                                     
bbb-docker_postgres_1              docker-entrypoint.sh postgres    Up (healthy)   5432/tcp                
bbb-docker_prometheus-exporter_1   python server.py                 Up             9688/tcp                
bbb-docker_recordings_1            /bin/sh -c /entrypoint.sh        Up                                     
bbb-docker_redis_1                 docker-entrypoint.sh redis ...   Up (healthy)   6379/tcp                
bbb-docker_webhooks_1              /bin/sh -c /entrypoint.sh        Up                                     
bbb-docker_webrtc-sfu_1            ./docker-entrypoint.sh npm ...   Up             127.0.0.1:3008->3008/tcp

At that point, you can visit https://[Domain] and instead of a 502 error, you should see the BigBlueButton 'Greenlight' front page.

Now the final step...

Create an admin user:

To create an admin user, you run this:

sudo docker-compose exec greenlight bundle exec rake admin:create

It will give you a login email address and a randomly generated password. Use these to log in. In the profile (top right Admin menu dropdown), you can alter the admin user's email to a real email (perhaps [Admin email]?), and change the password if you like.

Then you can go back to the Admin menu and select "Organisation" which should put you on the Manage Users page. You then invite yourself to join as a user by sending yourself an email (which, if your SMTP settings are correct, will work). If you don't receive it in a minute or two, check your spam folder.

Either log out of Greenlight before clicking the link in the email or open the link (by copying and pasting it) in a different browser where you're not logged in to this Greenlight instance, and create a user account for yourself.

Then log back into Greenlight as the Admin user (if you've previously logged out) and refresh the "Manage Users" page. You should find your newly created user. Select 'Edit' from the vertical 3 dotted menu. On the "Update your Account Info" page, set the User Role for your user to 'Admin" and click the "Update" button. You should now be able to log out as Admin, and back in as your own user, but with administrative privileges.

Run your first conference

Now it's time to try running a video conference. Click on the "Home" link (top right menu) and you should see that you have a default room called "Home Room" with a URL specified in the form [Domain]/b/[first 3 letters of your name]-[3 random hex digits]-[3 more random hex digits]. For me, it might look something like this: https://[Domain]/b/dav-a5b-73z

Click "Start" to initiate a conference in that Home Room.

In the conference room, you should select 'Microphone' so you can test speaking and listening, and also test your video camera (assming you have access to both on the computing device you're using to access BBB - note, a modern cellphone normally offers all of these.

We always encourage people participating in video conferences (regardless of the technology) to employ headphones to separate the audio from the conference form their input. It greatly improves the audio quality for all involved!.

If you select 'Microphone', there will be a brief delay whil your browser negotiates with the COTURN server in your BBB stack to link your data and audio channels. Once it's done that, it will give you an "echo test" window (see included screen shots). You should speak into your microphone when you see that screen and check whether you can hear yourself. This will confirm that your audio settings are right (or not) and will help the BBB system adjust its echo cancellation algorithms.

You can then also clidk the 'camera' icon (bottom middle of the page) to activate your webcam if you have one (or choose one if you have multiple cameras) as well as the video quality.

You can also try to 'Start recording' if you want to test your ability to record a session. Note that there is a delay after the end of a session in which you've recorded before the recording is displayed on the 'room' page in Greenlight. It might take minutes or even hourse to generate depending on the power of your server and the length of the session.

To see other administrative functionality including moderation and breakout rooms, click on the "gear" icon next to the "Users" heading in the left hand column. You can also experiment with the "Public Chat" and the "Shared Notes". Both can be saved at any time (via the top right 3 dot menu in that section) and will be included in any recordings you make.

Note the contents of Public Chat and Shared Notes will be wiped at the end of a session unless you explicitly save them or record the session (at least briefly at the end).

You can provide that "room address" to anyone and they can join your room (via an modern browser) when you have a session running. Alternatively, you can click on the 3 dotted menu associated with your room below the 'Search for room" form, and change the default properties of your room, including configuring it to let anyone who knows the room's address start a session. You can also create additional rooms (as an Admin user, you can set the limits on many of these properties via Organisation in the top right menu under your user name.

Have fun with your new, world class, cost-effective, large-scale BigBlueButton video conferencing application!

Next Steps

In a future tutorial, we'll provide information on how to troubleshoot BBB issues, how to upgrade it as new versions are made available by developers, and how to ensure that your recordings, user database, and configuration are backed up incrementally, encrypted, in remote storage.

Blog comments

It says in the nginx-config…

LPup (not verified) Tue 04/01/2022 - 20:32

It says in the nginx-config to change the port for the proxy and not forget to change it in the .env file. Where do I need to put what into the .env file for using a different port than 8080?

Hmm - thanks for pointing…

dave Tue 04/01/2022 - 20:44

Hmm - thanks for pointing that out. Yes, I think I made an error with that instruction (I've updated the Nginx config). If you implement BBB today via this set of instructions, you'll be installing BBB 2.4 - port 8080 was used in 2.3 and earlier versions. For 2.4, use port 48087. To be honest, I'm not sure of the right way to alter it if you have to due to other services running on the same system. Writing it manually in the docker-compose.yml file will be overwritten the next time you do an update... It might be necessary to change the port specified in the relevant container's Dockerfile under mods...

Add new comment

Configuring a Linux server to send email via the Postfix SMTP server using an external authenticating SMTP host

dave — Mon, 02 Aug 2021 02:08:28 +0000

Configuring a Linux server to send email via the Postfix SMTP server using an external authenticating SMTP host

Blog tags

ubuntu linux

postfix

smtp

18.04

free & open source

foss

20.04

22.04

dave Mon 02/08/2021 - 14:08

Just about any and every server needs to be able to send email - whether it's end-user-email, like password recovery services for a website to emails to system administrators reporting on the status of system backups and errors. The problem is that it's non trivial (understatement) to set up a mail server properly.

This howto assumes you have a Linux server (these instructions are for Ubuntu 22.04 and 20.04, although it should work on earlier versions of Ubuntu server and Debian Linux with minor changes, and the concepts will be very similar on other Linuxen) with a static IP address, with one or more fully-qualified-domain-names (fdqn) pointing at that address, and you have SSH-based access to it. I've previously provided tips on how to get to this stage.

Authenticating SMTP

To send email, you need access to a server, somewhere on the Internet, that provides the Simple Mail Transfer Protocol (SMTP) service. It's an open standard, and for most of the history of the Internet, email services have been mostly provided by Free and Open Source Software (FOSS) tools - the first SMTP was called "Sendmail" and it was fully FOSS, and it's still in use today (although it has mostly been superseded by faster, more secure systems, the best of which are also FOSS).

At the OERu, we use the Docker-based installation of the amazing, completely FOSS MailCow project to provide our organisational email services. I might cover that set up in a future tutorial here, because MailCow makes an otherwise almost intractable problem - hosting your own email service - much more tractable. Having a MailCow set up means we can offer "full service" email for any number of domains and users and aliases with all the bells and whistles including incoming and outgoing mail with all the virus scanning (we don't really need it because we use Linux desktops, but for other folks it's useful) and dynamic spam filtering services you'd expect from a much larger operation: Team MailCow have done an amazing job in pulling together a comprehensive set of FOSS applications to provide all the conceivable requirements of a full-fledged, multi-domain email system, including shared calendaring, contacts, and webmail. A great companion to your organisation's MailCow server would be a BitWarden password safe server (also FOSS)... just sayin'.

So, now, assuming that we have a MailCow server or some other functionally equivalent SMTP service available (apparently you can do this with Gmail, if you're a paying using although because of Google's terms of use, we recommend finding a more trustworthy solution), we have the option of "authenticated SMTP" for outgoing email using credentials we can set up. For example, in MailCow, we can specify a domain we host, like say oeru.org (and for which we've defined an MX record and a few other relevant records as guided by MailCow administrative web interface). On top of that, we can specify a mailbox for a dedicated "send stuff from remote relay hosts" email address using that domain, like smtp@oeru.org, with a strong password. With that, we can securely send email using that email address as the username and that password from anywhere we have access to the Internet.

The only tricky part is that we have to ensure that whatever "reply to" email address we specify from our applications, say notifications@tech.oeru.org, is using a domain we also host on the same server, and that there's an email alias of that email address defined and set as "allow to send from smtp@oeru.org" in the MailCow interface. If we haven't made sure of that, our mail server is likely to reject sending emails with that "mismatching" email address. This is a basic spam deterrence measure, which is for the best, despite sometimes making a email system administrator's life harder.

Once we've got that (and it's easy once you've done it once or twice - I'm mostly writing this down now so I don't have to try to re-remember every time I need to set up a new server - and I hope it helps others, too), we can set up any server we control to send secure (and spam-filter-resilient) email. For what it's worth, too, MailCow uses Postfix as its SMTP server component (there're a bunch of other components, too).

Postfix SMTP with SmartHost

The first thing you need to do to create a postfix smarthost is to install the postfix application on a new server (this assumes you're logged in with a user who has "sudo" - aka admin - permissions):

sudo apt update && sudo apt install postfix bsd-mailx

During the install, you'll be asked to select a bunch of configuration parameters. Select the defaults except:

Select "Internet Site with Smarthost",
fill in the domain name for your server,
the domain name and port (in the form [smtp server domain]:[port], e.g. smtp.oeru.org:587 ) of your "smarthost" who'll be doing the authenticating SMTP for you, and
the email address to which you want to receive system-related messages.

After that's done, you can proceed.

Next Steps

For the rest of this tutorial, you'll need to do the following. First, select your text editor. I use vim, but if you're new to the command line, I recommend using nano - it's more straightforward:

EDIT=`which nano` or EDIT=`which vim`

sudo $EDIT /etc/aliases

We need to make sure the "root" user points to a real email address. Add a line at the bottom which says (replacing [your email] with your email :) )

root: [your email]

After which you'll need to convert the aliases file into a form that postfix can process, simply by running this:

sudo newaliases

Then we have to define the authentication credentials required to convince your mail server that you're you!

sudo $EDIT /etc/postfix/relay_password

The resulting file only needs one line with three bits of information:

[smtp server domain] [user name]:[password]

for example:

smtp.oeru.org smtp@oeru.org:SomeObscurePassw0rd

Then save the file and, like the aliases file, run the conversion process (which uses a slightly different mechanism):

sudo postmap /etc/postfix/relay_password

Finally, we'll edit the main configuration file for Postfix to tell it about all this stuff:

sudo $EDIT /etc/postfix/main.cf

If your SMTP server uses port 25 (the default for unencrypted SMTP) you don't have to change anything, although most people nowadays prefer to use StartTLS or otherwise encrypted transport to at least ensure that your SMTP authentication details (at least) are transferred encrypted. That means using port 587 or 465. If you're using either of those ports, find the "relayhost = [your server name]" line... and add your port number after a colon, like this

relayhost = [your server name]:[server port]

or, for example:

relayhost = smtp.oeru.org:465

Next, add the following lines at the bottom of the file:

# added to configure accessing the relay host via authenticating SMTP smtp_sasl_auth_enable = yes smtp_sasl_password_maps = hash:/etc/postfix/relay_password smtp_sasl_security_options = noanonymous
smtp_tls_security_level = encrypt

# if you're using Ubuntu prior to 20.04, uncomment (remove the #) the
# earlier line smtp_tls_security_level = may to save errors in 'postfix check' # and comment this line (by adding a # at the start) smtp_tls_wrappermode = yes

And, finally, comment out the line smtp_tls_security_level = may higher in the file - careful not to confuse it with the very similar smtpd_tls_security_level variable (note the extra '''d''' in `smtpd...`) line.

Save the file, and then check that your syntax is correct:

sudo postfix check

If it is (running the command returns no errors, and it might not return anything at all - that's a good thing!), then you can run

sudo postfix reload

to get postfix to reload its configurations and you can test out your new smarthost-configured SMTP server!

If not, the output of the check command will usually give you a helpful insight into what is wrong with your configuration... you'll also find that looking at the mail log is very helpful and offers great insights:

sudo less +G /var/log/mail.log

and if you're not able to fix it based on those, you'll find postfix is widely documented and has rich set of easily discoverable resources out there on the web - a search engine is your best resource!

Testing your outgoing email

$ mail you@email.domain<ENTER>

Subject: Testing from your.relay.server.domain<ENTER> Testing postfix remote host<ENTER> <CTRL-D> Cc:<ENTER>

Typing <CTRL-D> (hold down the Control or Ctrl key on your keyboard and press the "d" key) will finish your message, showing you a "CC:" field, in which you can type in other email addresses if you want to test sending to multiple addresses. When you then hit <ENTER>, it will attempt to send this email. It might take a few minutes to work its way through to the receiving email system (having to run the gauntlet of spam and virus filters on the way).

You can also always check the postfix system logs to see what postfix thinks about it using the command above. Hit <SHIFT-F> to have the log update in real time.

Done

Now you've got working outgoing email from your server. That means many higher-level web applications you might install on your infrastructure will work out-of-the-box, because what you've set up, for example, enables the default PHP email service and that used by other stacks.

Sending from Docker Containers

You can configure your server so you can reference it from services you run from Docker containers on your host. You do this by referencing the host, like via an ad hoc SMTP server on your container like msmtp, and you can just reference it as 172.17.0.1, which is the default base IP for Docker hosts from the perspective of Docker containers. You might find it's different on your particular install. In that case, you have to make your Postfix SmartHost accept email for sending from the Docker containers on that server. There're quite a few examples of that among my Docker recipes on the OERu's git repository.