wordpress

Building a Course Site with WordPress on Ubuntu 20.04 using Docker Compose

dave — Tue, 24 Aug 2021 04:00:22 +0000

Building a Course Site with WordPress on Ubuntu 20.04 using Docker Compose

Blog tags

wordpress

docker

docker-compose

ubuntu linux

20.04

multisite

redis

nginx

php

let's encrypt

dave Tue 24/08/2021 - 16:00

The OERu offers accredited tertiary (University level) courses to learners anywhere on the Internet, whether they are using desktop computers or mobile devices.

Instead of using a Learning Management System to frame the courses, the OERu Courses are hosted in a WordPress MultiSite (originally called a "Network" of sites within a single WordPress installation) implementation, with each course a 'subsite' represented by sub-directory below the main site. For example, the first Learning in a Digital Age micro-course, "Digital literacies for online learning" with the course code LiDA 101, can be found at https://course.oeru.org/lida101. This post explains how you can replicate our fully Free and Open Source Software large-scale Open Educational Resource (OER) course delivery platform at negligible cost.

Step one - a suitable host
Step two - prepare the host
- Set up Docker and Docker-Compose
Step three - configure your domain
- Set up Nginx reverse proxy
- Set up Let's Encrypt for the domain
Step four - get the code
Step five - set up Docker Compose for the site
- Create the NGINX container's configuration
- Launch containers
Step Six - set up your site
Step Seven - celebrate!
Snapshotting your first OER course!
Enabling OERu's Register Enrol functionality
Data Backups
Staying up-to-date
Acknowledgements

To host our WordPress Multisite, we use a quartet of Docker containers - a Redis caching server, an Nginx webserver, and two servers running the PHP script interpreter in FPM mode, with the first being the main workhorse responding to queries sent to it by the Nginx server... The other PHP container is responsible for running cron (scheduled) tasks in the background. It's all driven by Docker Compose (which coordinates the individual Docker containers) on an Ubuntu Linux host. The host also runs an Nginx instance to act as a 'reverse proxy', and the endpoint of our SSL (we use Let's Encrypt certificates). This is all consistent with our FOSS Docker-based hosting conventions.

Although our process to get one of the these sites up and running is made up of simple steps, there are a bunch of them, so get comfortable and buckle yourself in and prepare for a fun ride! Just beware - this is a pretty audacious tutorial describing an advanced production-ready system with lots of moving parts doing a serious amount of stuff behind the scenes, so this process is likely to take a few hours from start to finish and isn't for the faint of heart.

Step one - a suitable host

The first step is to get yourself an entry-level virtual server or compute instance somewhere.

I generally use DigitalOcean (I have no affiliation with the company), but there are many other commodity hosting services (check out Vultr or Linode, for example) around the world which offer comparably (or better) spec'd servers for USD5.00/month, or USD60.00/year. For that you get a Gigabyte (GB) of RAM, a processor, and 40GB of SSD (Static Storage Device = faster) storage.

A server (a "Droplet" in Digital Ocean parlance) with a GB of RAM and 20+ GB of disk space will be sufficient for this sort of service. If you expect it to have heavy traffic, or you might want to add more services, you might want to invest in a higher-spec server up-front (because, among other things, it'll offer you more disk space). Most of our servers are USD40/month instances (USD480/year) which buys 8GB of RAM, 4 virtual processors, and 160GB of disk space.

I suggest you create an account for yourself on your chosen hosting provider (and I encourage you to use Two Factor Authentication, aka 2FA, on your hosting account so that no one can log in as you and, say, delete your server unexpectedly - you'll find instructions on how to set up 2FA on your hosting provider's site) and create an Ubuntu 20.04 (or the most recent 'Long Term Support' (LTS) version - the next will be 22.04, in April 2022) in the zone nearest to you.

If you don't already have an SSH key on your computer, I encourage you to create one and specify the public key in the process of creating your server - that should allow you to log in without needing a password!

You'll need to note the server's IPv4 address (it'll be a series of 4 numbers, 0-254, separated by full stops, e.g. 103.99.72.244), and you should also be aware that your server will have a newer IPv6 address, which will be a set of 8 four hex character values [each hex character can have one of 16 values: 0-9,A-F] separated by colons, e.g. 2604:A880:0002:00D0:0000:0000:20DE:9001. With one or the other of those IPs, you can log into it via SSH.

Once you get logged in, it's worth doing an upgrade of your server's Ubuntu system! Do that as follows:

sudo apt-get update && sudo apt-get dist-upgrade

Get your Domain lined up

You will want to have a domain to point at your server, so you don't have to remember the IP number. There're are thousands of domain "registrars" in the world who'll help you do that... You just need to "register" a name, and you pay yearly fee (usually between USD10-30 depending on the country and the "TLD" (Top Level Domain. There're national ones like .nz, .au, .uk, .tv, .sa, .za, etc., or international domains (mostly associated with the US) like .com, .org, .net, and a myriad of others. Countries decide on how much their domains wholesale for and registrars add a margin for the registration service).

Here in NZ, I use the services of Metaname (they're local to me in Christchurch, and I know them personally and trust their technical capabilities). If you're not sure who to use, ask your friends. Someone's bound to have recommendations (either positive or negative, in which case you'll know who to avoid).

If you want to use your domain for other things besides your WordPress instance, I'd encourage you to use a subdomain, like (my usual choice) is "course.domainname", namely the subdomain "course" of "domainname".

Once you have selected and registered your domain, you can set up (usually through a web interface provided by the registrar) an "A Record" which associates your website's name to the IPv4 address of your server. So you should just be able to enter your server's IPv4 address, the domain name (or sub-domain) you want to use for your WordPress service. Nowadays, if your Domain Name host offers it (some don't, meaning they're way behind the times), it's also important to define an IPv6 record, which is called an "AAAA Record"... you put in your IPv6 address instead of your IPv4 one.

You might be asked to set a "Time-to-live" (which has to do with the length of time Domain Name Servers are asked to "cache" the association that the A Record specifies) in which case you can put in 3600 seconds or an hour depending on the time units your interface requests... but in most cases that'll be set to a default of an hour automatically.

Log into your server

You should be able to test that your A and AAAA Records have been set correctly by logging into your server via SSH using your domain name rather than the IPv4 or IPv6 address you used previously. It should (after you accept the SSH warning that the server's name has a new name) work the same way your original SSH login did. On Linux, you'd SSH via a terminal and enter ssh root@[domain name]. I think you can do similar on MacOS and on Windows, I believe people typically use software called Putty...

This will log you into your server as the 'root' user. It's not considered good practice to access your server as root (it's too easy to completely screw it up). Best practice is to create a separate 'non-root' user who has 'sudo' privileges and the ability to log in via SSH. If you are currently logged in as 'root', you can create a normal user for yourself via (replace [username] with your chosen username):

U=[username]
adduser $U
adduser $U ssh
adduser $U sudoers

You'll also want to a set a password for user [username]:

passwd $U

then become that user temporarily (note, the root user can 'become' another user without needing to enter a password) and create an SSH key and, in the process, the .ssh directory (directories starting with a '.' are normally 'hidden') for the file into which to put your public SSH key:

su $U ssh-keygen -t rsa -b 2048 nano ~/.ssh/authorized_keys

and in that file, copy and paste (without spaces on either end) your current computer's public ssh key (never publish your private key anywhere!), save and close the file.

From that point, you should be able to SSH to your server via ssh [username]@[domain name] without needing to enter a password.

These instructions use 'sudo' in front of commands because I assume you're using a non-root user. The instructions will still work fine even if you're logged in as 'root'.

Sort out your details

You'll replace the similarly [named] variables in the configuration files and command lines below. These are the values you need to find or create.

[domain name] - the fully qualified domain name or subdomain by which you want your WPMS to be accessed. You must have full domain management ability on this domain. Example: course.oeru.org
[port] - this is an unused port, i.e. not used by any other service, that will be used by the Nginx reverse proxy to talk to your Docker Nginx webserver container. A conventional option would be 8080... If that's already in use, try 8081, etc. You can check what ports are in use with the command sudo netstat -punta - the ports are listed after the ':' in each case
MariaDB/MySQL database details
- [database root password] - the administrative user (root) password for this server - see our tutorial on creating strong random passwords
- [your database password] - if you, optionally, want to set up an admin user for yourself on this server. Paired with [your username] on the server.
- [database name] - the name of the database for this specific WordPress site - example wordpress
- [database user] - a separate username for this WordPress database - example wordpress (but it can be different too)- note: you'll be creating two database users with this same username and password, but that's intended.
- [database password] a separate password for this WordPress database user
Authenticating SMTP details - this is required so your WordPress site can send emails to users - crucial things like email address validation and password recovery emails...
- [smtp host] - the domain name or IPv4 or IPv6 address of an SMTP server
- [smtp reply-to-email address] - a monitored email to which people can send email related to this WordPress site, e.g. webmaster@[domain name]
- [smtp user] - the username (often an email address) used to authenticate against your SMTP server, provided by your email provider.
- [smtp password] - the accompanying password, provided by your email provider.
Wordpress configuration values
- [redis password] - another random password, this time for the caching service we'll set up to make your WordPress site faster than billy-o.
- [wordpress keys and salts] - a series of random numbers to make your WordPress site far more secure than it would otherwise be - these can be generated automatically using the approach described below!.
For those incorporating our WEnotes system, you'll need the following (these values are optional and can be ignored!)
- [couchdb host] - the domain name of the couchdb host - in the case of the OERu, our host is couch.oerfoundation.org. Yours might be different.
- [couchdb mention database] - the name of the designated 'mention' database on the couchdb host.
- [couchdb user] - a couchdb username, provided by whoever manages your couchdb host
- [couchdb password] - a couchdb password, also provided by whoever manages your couchdb host
Docker.com login details (you'll need a username and password).

Note: not all values in all files surrounded by [] need to be replaced! If they're not included in the list above, leave them as you find them!

Step two - prepare the host

Preparing the host involves ensuring your firewall, UFW, is configured properly, installing the Nginx webserver to act as your host's reverse proxy, and installing MariaDB (MySQL compatible but better, for a variety of reasons including that it's not controlled by Oracle) and configuring it properly. Then we can create the MariaDB database specifically for this WordPress installation. We also suggest you install Postfix so your server can send out email to you, and finally, we'll ensure that your server knows how to launch Docker containers and manage them with Docker Compose.

Before we do anything else, let's make sure your Ubuntu package repository is up-to-date.

sudo apt-get update

If you pause this build process for more than a few hours, it pays to run it again before you continue on.

Firewall with UFW

No computer system is ever full secure - there're always exploits waiting to be found, so security is a process of maintaining vigilance. Part of that is reducing exposure - minimising your "attack surface". Use a firewall - ufw is installed on Ubuntu by default and is easy to set up and maintain. Make sure you've got exceptions for SSH (without them, you could lock yourself out of your machine! Doh!).

Run the following commands to allow your Docker containers to talk to other services on your host.

sudo ufw allow in on docker0
sudo ufw allow from 172.0.0.0/8 to any

Specifically for Docker's benefit, you need to tweak the default Forwarding rule (I use vim as my editor. An alternative, also installed by default on Ubuntu, nano, is probably easier to use for simple edits like this, so I'll use nano here):

sudo nano /etc/default/ufw

and copy the line DEFAULT_FORWARD_POLICY="DROP" tweak it to look like this (commenting out the default, but leaving it there for future reference!):

#DEFAULT_FORWARD_POLICY="DROP"
DEFAULT_FORWARD_POLICY="ACCEPT"

and then save and exit the file (CTRL-X and then 'Y').

You also have to edit /etc/ufw/sysctl.conf and remove the "#" at the start of the following lines, so they look like this:

sudo nano /etc/ufw/sysctl.conf

# Uncomment this to allow this host to route packets between interfaces
net/ipv4/ip_forward=1
net/ipv6/conf/default/forwarding=1
net/ipv6/conf/all/forwarding=1

and finally restart the network stack and ufw on your server

sudo systemctl restart systemd-networkd
sudo service ufw restart

Installing the Nginx webserver/reverse proxy

In the configuration I'm describing here, you'll need a webserver running on the server - it'll be acting as a reverse proxy for the Docker-based Nginx instance described below. I prefer the efficiency of Nginx and clarity of Nginx configurations over those of Apache and other open source web servers. Here's how you install it.

sudo apt-get install nginx-full

To allow nginx to be visible via ports 80 and 443, run

sudo ufw allow "Nginx Full"

To check that all worked, you can put http://[domain name] into your browser's address bar, and you should see a default "NGINX" page...

Note: make sure your hosting service is not blocking these ports at some outer layer (depending on who's providing that hosting service you may have to set up port forwarding).

Installing MariaDB

MariaDB is effectively a drop-in alternative to MySQL and we prefer it because it's not controlled by Oracle and has a more active developer community. On Ubuntu, MariaDB pretends to be MySQL for compatibility purposes, so don't be weirded out by the interchangeable names below. Install the latest versions of the server and client like this.

sudo apt-get install mariadb-server mariadb-client

You need to set a root (admin) user password - you might want to create a /root/.my.cnf file containing the following (replacing [mysql root password]) to let you access MariaDB without a password from the commandline:

sudo nano /root/.my.cnf

and put the following info into it

[client]
user=root
password=[database root password]

You should now be able to type mysql at the command prompt (note, the name mysql is used for backward compatibility with many implementations where MariaDB is being used to replace MySQL).

Optional MySQL non-root user

If you're happy to use your root user to access MySQL, e.g. sudo mysql (which uses 'sudo' to access it via the root user), then you can safely ignore the rest of this section.

If you're accessing the server via a non-root user (which is a good idea, and is the reason we use sudo in this howto), you might want to create a similar ~/.my.cnf file in your directory , with your username in place of root, and a different password. That will allow you to work with the MariaDB client without needing to enter the root credentials each time.

To make it work, you'll need to run the following as the MySQL admin user - this should be the default on this new install - remember to replace your [tokens]!). This creates two users with the same credentials that will allow you to log in either from the same server (i.e. 'localhost') or from any of your Docker containers (often useful for debugging!), namely the wildcard '%'. Remember: if you change the user's details, you'll have to do it for both the localhost and '%' users.

CREATE USER "[your username]"@"localhost" IDENTIFIED BY "[your database password]";
CREATE USER "[your username]"@"%" IDENTIFIED BY "[your database password]";
GRANT ALL ON *.* to "[your username]"@"localhost" WITH GRANT OPTION;
GRANT ALL ON *.* to "[your username]"@"%" WITH GRANT OPTION;
FLUSH PRIVILEGES;

Don't be alarmed if MySQL tells you "0 rows affected" when you create a user - unless you see a specific 'error', it's still creating them.

End optional MySQL non-root user section

Tweak the configuration so that it's listening on the right internal network device.

sudo nano /etc/mysql/mariadb.conf.d/50-server.cnf

and copy the bind-address line and adjust so it looks like this - we want MariaDB to be listening on all interfaces, not just localhost (127.0.0.1)...

# Instead of skip-networking the default is now to listen only on
# localhost which is more compatible and is not less secure.
#bind-address           = 127.0.0.1
bind-address            = 0.0.0.0

Then restart MariaDB:

sudo service mysql restart

It should now be listening on MySQL/MariaDB's default port 3306 on all interfaces, i.e. 0.0.0.0. For safety's sake, external access to the MariaDB server is blocked by your UFW firewall.

Now set up the database which will hold WordPress' data. Log into the MariaDB client on the host (if you've created a .my.cnf file in your home directory as describe above, you won't need to enter your username and password):

mysql -u root -p

Enter your root password when prompted and then replace the following [database-related tokens] to create the database with the right language encoding, along with access to the right separate user:

CREATE DATABASE [database name] CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
CREATE USER "[database user]"@"%" IDENTIFIED BY "[database password]";
GRANT ALL ON [database name].* to "[database user]"@"%";
FLUSH PRIVILEGES;

The last line ensures that MariaDB has updated its internal permissions to recognise your new user. To exit the SQL client, just type \q and ENTER.

Sending emails

Because a server like this one is set up to perform lots of rather complex jobs to perform, it's vital that your server has the ability to send you emails to alert you of problems, like failed updates or backups. We encourage you to follow our instructions on how to configure your server to use the Postfix SMTP server to send out email, using your Authenticating SMTP details.

Regular automatic database backups

Finally, it's a good idea (but optional - if you're in a hurry, you can do this later) to make sure that your server is maintaining backups of your database - in this case, we'll use the automysqlbackup script to automatically maintain a set of dated daily database backups. It's easy to install, and the database backups will be in /var/lib/automysqlbackup in dated folders and files. If you haven't set up Postfix in the previous step, just beware you will be asked to set it up when installing automysqlbackup.

sudo apt-get install automysqlbackup

That's all there is to it. It should run automatically every night and store a set of historical SQL snapshots that may well save your bacon sometime down the track!

Set up Docker and Docker-Compose

First, you need to set up Docker support on your server - use the 'repository method' for Ubuntu 20.04 and choose the 'x86_64 / amd64' tab!

Also, if you're using a non-root user, follow the complete instructions including setting up Docker for your non-root user.

The way I implement this set of containers is to use Docker Compose) which depends on the Python script interpreter (version 3+). I suggest using the latest installation instructions provided by the Docker community. Of the options provided, I use the 'alternative instructions', employing the 'pip' approach. This is what I usually do (to summarise the pip instructions):

The firrst step is to Install Ubuntu's Python3 pip which is a bit outdated...

sudo apt install python3-pip

use the Ubuntu instance, called pip3 to install the latest Python 3 pip

sudo pip install -U pip

and (finally) install the docker-compose script:

sudo pip install -U docker-compose

Set up our conventional directories

To set up your server, I recommend setting up a place for your Docker containers as per our Docker-related conventions:

sudo mkdir -p /home/data/[domain name]
sudo mkdir -p /home/docker/[domain name]

Step three - configure your domain

Set up Nginx reverse proxy

A reverse proxy is a website intermediary. It accepts requests from the Internet, like a normal webserver would, but instead of having direct access to the data being requested, it, in turn, makes a request from another webserver either on the same host, or on a different one, and passes the results of that request back to the requester. In this case, the Nginx instance on the host is making a request of a different host that happens to reside on the same host instance as a Docker 'container'.

Our convention is to create an Nginx reverse proxy configuration file with the same name as our [domain name], so in the case of, say, our Course WordPress Multisite, the file would be /etc/nginx/sites-available/course.oeru.org. Create a file in your /etc/nginx/sites-available with the following (again, replacing the [values] with your own values.

sudo nano /etc/nginx/sites-available/[domain name]

and copy and paste this in (and remember to replace the [tokens] with your relevant variables!):

#
# Set [domain name] and [port] below to make this work
#
# HTTP does *soft* redirect to HTTPS
#
server {
    # add [IP-Address:]80 in the next line if you want to limit this to a single interface
    listen 80;
    listen [::]:80;
    server_name [domain name];
    root /home/data/[domain name]/scr;
    index index.php;
 
    # change the file name of these logs to include your server name
    # if hosting many services...
    access_log /var/log/nginx/[domain name]_access.log;
    error_log /var/log/nginx/[domain name]_error.log;
 
    # for let's encrypt renewals!
    include includes/letsencrypt.conf;
 
    # redirect all HTTP traffic to HTTPS.
    location / {
        return  302 https://[domain name]$request_uri;
    }
}
#
# HTTPS
#
# This assumes you're using Let's Encrypt for your SSL certs (and why wouldn't
# you!?)... https://letsencrypt.org
server {
    # add [IP-Address:]443 ssl in the next line if you want to limit this to a single interface
    listen 443 ssl;
    listen [::]:443 ssl;
    # Note: these are *temporary* certificates, created when your host was set up
    # they are only in use to get Nginx to start up properly and let you create your let's encrypt certificates!
    ssl_certificate /etc/ssl/certs/ssl-cert-snakeoil.pem;
    ssl_certificate_key /etc/ssl/private/ssl-cert-snakeoil.key;
    # these will be used after we finish the Let's Encrypt process
    #ssl_certificate /etc/letsencrypt/live/[domain name]/fullchain.pem;
    #ssl_certificate_key /etc/letsencrypt/live/[domain name]/privkey.pem;
    ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
    # to create this, see https://raymii.org/s/tutorials/Strong_SSL_Security_On_nginx.html
    ssl_dhparam /etc/ssl/certs/dhparam.pem;
    keepalive_timeout 20s;
 
    server_name [domain name];
    root /home/data/[domain name]/src;
    index index.php;
 
    # change the file name of these logs to include your server name
    # if hosting many services...
    access_log /var/log/nginx/[domain name]_access.log;
    error_log /var/log/nginx/[domain name]_error.log;
 
    location / {
        # a good value for [port] is 8080, unless it's already in use by another service on your server...
        proxy_pass http://127.0.0.1:[port];
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_set_header Host $http_host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Host $server_name;
        proxy_set_header X-Forwarded-Proto https;
        proxy_connect_timeout 900;
        proxy_send_timeout 900;
        proxy_read_timeout 900;
        send_timeout 900;
    }
    #
    # These "harden" your security
    add_header 'Access-Control-Allow-Origin' "*";
    # from https://gist.github.com/Stanback/7145487
    add_header 'Access-Control-Allow-Credentials' 'true' always;
    add_header 'Access-Control-Allow-Methods' 'GET, POST, PUT, DELETE, OPTIONS' always;
    add_header 'Access-Control-Allow-Headers' 'Accept,Authorization,Cache-Control,Content-Type,DNT,If-Modified-Since,Keep-Alive,Origin,User-Agent,X-Requested-With' always;
    #
    # for H5P embedding
    #add_header 'X-Frame-Options' 'ALLOW-FROM https://h5p.oeru.org';
    # required to be able to read Authorization header in frontend
    add_header 'Access-Control-Expose-Headers' 'Authorization' always;
    # tested at https://csp-evaluator.withgoogle.com/
    # works, but only B+ on MozOBs https://observatory.mozilla.org/analyze.html
    add_header X-XSS-Protection "1; mode=block";
}

Having created that file, we now have to create the ssl_dhparam file we referenced, staring by installing OpenSSL tools:

sudo apt-get install openssl

by running (warning - this can take quite a long time - like 5-15 minutes in my experience, depending on a lot of factors - the system needs to generate sufficient entropy to achieve acceptable randomness):

sudo openssl dhparam -out /etc/ssl/certs/dhparam.pem 4096

if you're short on time, you can create a key half the size far more quickly (a few seconds, typically):

sudo openssl dhparam -out /etc/ssl/certs/dhparam.pem 2048

When that's done, you should see there's a file here: ls -l /etc/ssl/certs/dhparam.pem

Set up Let's Encrypt for the domain

We've already written a guide to setting up and securing your domain with Let's Encrypt but here're the relevant details:

sudo mkdir /etc/nginx/includes

and edit the following file

sudo nano /etc/nginx/includes/letsencrypt.conf

to make sure it has the following content:

# Rule for legitimate ACME Challenge requests
location ^~ /.well-known/acme-challenge/ {
    default_type "text/plain";
    # this can be any directory, but this name keeps it clear
    root /var/www/letsencrypt;
}
 
# Hide /acme-challenge subdirectory and return 404 on all requests.
# It is somewhat more secure than letting Nginx return 403.
# Ending slash is important!
location = /.well-known/acme-challenge/ {
    return 404;
}

Next, make sure your designated Let's Encrypt directory exists (note - you only need to do this once on a given host):

sudo mkdir /var/www/letsencrypt

Now, we'll make sure Nginx is aware of your configuration, which you do like this (substituting [domain name] with your domain!):

sudo ln -sf /etc/nginx/sites-available/[domain name] /etc/nginx/sites-enabled

then make sure Nginx is happy with your configuration syntax:

sudo nginx -t

and fix any typos which might've crept in. If it says your configurations are okay, then make your configuration live:

sudo service nginx reload

Once this is done, you can check to see if things are working properly by entering http://[domain name] in your browser's address bar. It should redirect you to http**s**://[domain name] and give you an error that there's a 'mismatch' in your certificates... which there is: my configuration approach means the server and your reverse proxy configuration are temporarily using the default "SnakeOil" SSL certificate pair (required to get Nginx to start in SSL mode), but your Nginx configuration is working to the extent required for us to request our Let's Encrypt certificate!

And here's how we actually generate your SSL certificat via Let's Encrypt (replacing [domain name] as appropriate):

sudo letsencrypt certonly --webroot -w /var/www/letsencrypt -d [domain name]

If it works, it gratifyingly results in a message that starts with "Congratulations"! Well done if you got that! Note, if you get an error, make sure your domain is properly configured to point to your server! Also, there could be a delay in that configuration change taking effect due to the vagueries of the DNS system. If it worked, you should have a set of certificates in the directories, currently commented out with leading "#"s, in the Nginx configuration file above. You'll now need to re-edit it

sudo nano /etc/nginx/sites-available/[domain name]

to change it so the relevant part looks like this (again, ensuring your [domain name] is in place in the relevant locations!):

    # Note: these are *temporary* certificates, created when your host was set up
    # they are only in use to get Nginx to start up properly and let you create your let's encrypt certificates!
    #ssl_certificate /etc/ssl/certs/ssl-cert-snakeoil.pem;
    #ssl_certificate_key /etc/ssl/private/ssl-cert-snakeoil.key;
    # these will be used after we finish the Let's Encrypt process
    ssl_certificate /etc/letsencrypt/live/[domain name]/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/[domain name]/privkey.pem;

and then we need to do our obligatory configuration test:

sudo nginx -t

and if Nginx is happy, reload the configuration to put it into effect:

sudo service nginx reload

Now, if you point your browser at http://[domain name], it should automatically redirect to https://[domain name] without any errors, except that it won't yet have any content to show you, so you might get a 404 error, which is expected!

Step four - get the code

Next we have to get all the relevant code for the WordPress site and its dependencies.

WordPress source code

The latest source code for WordPress is always available from https://wordpress.org/latest.tar.gz - we'll get a copy of it now and put it in the right place:

cd /home/data/[domain name]
sudo wget https://wordpress.org/latest.tar.gz
sudo tar xvfz latest.tar.gz && sudo mv wordpress src

After this, you should find a directory, src in your [domain name] directory. That contains all the default source code for WordPress including default themes and plugins.

Next, we have to make sure the default theme is in place, and then set up the WordPress multisite configuration process.

Later we'll get the assortment of third party plugins and custom OERu plugins needed to flesh out the WordPress functionality we need.

OERu Theme

First, we'll get the OERu theme, which is specially designed to provide an accessible desktop or mobile experience, given that many - even a majority - of our learners are in the developing world, where mobile computing is dominant!

We'll go to the theme directory - you should already be in /home/data/[domain name]. From there we go into the theme directory:

cd src/wp-content/themes

and we issue a git clone command to retrieve the latest version of the OERu theme from our git repository:

sudo git clone https://git.oeru.org/oeru/oeru_course.git oeru-course

Then it's time to customise our WordPress configuration.

Customise wp-config.php

First we need to create your wp-config.php file in your src directory:

sudo nano wp-config.php

Note: you'll need your password for Redis and a set of [wordpress keys and salts], both of which are essentially just random numbers that are used to make you site far more secure. You can use the this link to generate a set of random values suitable for copying-and-pasting into you wp-config.php file in your src directory - here's an example of the output I just requested (don't use these, generate your own):

define('AUTH_KEY',         '?4^Huc~R1=WW+T_p~.0dH$XJ`>U*MoreMmZ{@tORSFG3aX37#ZS+0ou{j^DS3{f<');
define('SECURE_AUTH_KEY',  '7.k4htjPnrA/?6JJlogA4Wp*o|,&&>;20ppqeqHq#gI <%gDz[o( hpRRB|!jws%');
define('LOGGED_IN_KEY',    'fTX,WkI=doAUE%?{zHp5.?fN%WWtBuy~`Scntr<]I1WvlF6i=7J kjO0Z%%~Z-`N');
define('NONCE_KEY',        '?p&]/*(G-+W!0#[&Y6KKj)j Ok5QI(SUc@@rv,ivtF> AR;Yv+Yu#>$B$<P9Ld|j');
define('AUTH_SALT',        'H6s2H~KP]Z7YXTFt|8[Lgz[1~5wF+PJzxR^KW$|he+9|RF/vi@}/|<8bkC:w)qW%');
define('SECURE_AUTH_SALT', '@- j6Kn CjP/mdbmLXtkC+>1>+H-8pXETlJ4+]b-9x_/t*.D}VA1w<^A?0 R<f+1');
define('LOGGED_IN_SALT',   ' y9:oh)]nA$}%9N-xk1MAQN1bH 8z{UD/e~K|G5{(9y|,n2E*,KwYPIf~HwhHT J');
define('NONCE_SALT',       '|B?p#Q4|.=[VL8)2AX;zy-R2;x#dqIo=!C3,;OACT%-uaQ7Li5KSVSSnLahwlZ+o');

This is what your wp-config.php file should look like - copy and paste the following into the file, replacing the relevant [tokens] with your versions (in particular note that the above wordpress keys and salts need to go where I've got the token [wordpress keys and salts] below):

<?php
/**
 * The base configuration for WordPress
 *
 * The wp-config.php creation script uses this file during the
 * installation. You don't have to use the web site, you can
 * copy this file to "wp-config.php" and fill in the values.
 *
 * This file contains the following configurations:
 *
 * * MySQL settings
 * * Secret keys
 * * Database table prefix
 * * ABSPATH
 *
 * @link https://codex.wordpress.org/Editing_wp-config.php
 *
 * @package WordPress
 */
 
// ** MySQL settings - You can get this info from your web host ** //
/** The name of the database for WordPress */
define('DB_NAME', '[database name');
 
/** MySQL database username */
define('DB_USER', '[database user]');
 
/** MySQL database password */
define('DB_PASSWORD', '[database password]');
 
/** MySQL hostname */
//define('DB_HOST', '10.10.10.1');
define('DB_HOST', '172.17.0.1');
 
/** Database Charset to use in creating database tables. */
define('DB_CHARSET', 'utf8mb4');
//define('DB_CHARSET', 'utf8');
 
/** The Database Collate type. Don't change this if in doubt. */
define('DB_COLLATE', '');
 
/**#@+
 * Authentication Unique Keys and Salts.
 *
 * Change these to different unique phrases!
 * You can generate these using the {@link https://api.wordpress.org/secret-key/1.1/salt/ WordPress.org secret-key service}
 * You can change these at any point in time to invalidate all existing cookies. This will force all users to have to log in again.
 *
 * @since 2.6.0
 */
[wordpress keys and salts]
 
/**
 * WordPress Database Table prefix.
 *
 * You can have multiple installations in one database if you give each
 * a unique prefix. Only numbers, letters, and underscores please!
 */
$table_prefix  = 'wp_';
 
/**
 * For developers: WordPress debugging mode.
 *
 * Change this to true to enable the display of notices during development.
 * It is strongly recommended that plugin and theme developers use WP_DEBUG
 * in their development environments.
 *
 * For information on other constants that can be used for debugging,
 * visit the Codex.
 *
 * @link https://codex.wordpress.org/Debugging_in_WordPress
 */
define('WP_DEBUG', false);
//define('WP_DEBUG', true);
define('CONCATENATE_SCRIPTS', false);
define('SCRIPT_DEBUG', true);
define('WP_DEBUG', true);
define('WP_DISABLE_FATAL_ERROR_HANDLER', true );   // 5.2 and later
 
/* We are behind a reverse proxy */
if (isset($_SERVER['HTTP_X_FORWARDED_FOR'])) {
    $forwarded_address = explode(',', $_SERVER['HTTP_X_FORWARDED_FOR']);
    $_SERVER['REMOTE_ADDR'] = $forwarded_address[0];
}
if (isset($_SERVER['HTTP_X_FORWARDED_PROTO']) && $_SERVER['HTTP_X_FORWARDED_PROTO'] == 'https') {
    $_SERVER['HTTPS'] = 'on';
}
 
/* Disable the default 'ad hoc' cron mechanism.
   We'll use actual cron instead. */
define('DISABLE_WP_CRON', true);
 
/* That's all, stop editing! Happy blogging. */
 
/** Absolute path to the WordPress directory. */
if ( !defined('ABSPATH') )
        define('ABSPATH', dirname(__FILE__) . '/');
 
/* Multisite */
// see https://wordpress.org/support/article/create-a-network/ 
define('WP_ALLOW_MULTISITE', true);
/*define('MULTISITE', true);
define('SUBDOMAIN_INSTALL', false);
define('DOMAIN_CURRENT_SITE', '[domain name]');
define('PATH_CURRENT_SITE', '/');
define('SITE_ID_CURRENT_SITE', 1);
define('BLOG_ID_CURRENT_SITE', 1);*/
 
/* disable trash, immediately permanently delete */
define('EMPTY_TRASH_DAYS', 0);
/* set the default theme for the network */
define('WP_DEFAULT_THEME', 'oeru_course');
 
/** Caching-related configuration */
/** Redis */
define('WP_REDIS_HOST', 'redis');
define('WP_REDIS_PASSWORD', '[redis password]');
define('WP_REDIS_PATH', '/tmp/cache');
 
/* WEnotes plugin configuration, commented out by default */
/* this is optional(!!) - only use if you're deploying the OERu WEnotes stack - contact us if you want help! */
/*define('WENOTES_HOST', '[couchdb host]');
define('WENOTES_PORT', '80');
define('WENOTES_DB',   '[couchdb mention database]');
define('WENOTES_USER', '[couchdb user]');
define('WENOTES_PASS', '[couchdb password]');*/
 
// go to /wp-admin/maint/repair.php to see if a repair is needed...
//define( 'WP_ALLOW_REPAIR', true );
 
 
/** Sets up WordPress vars and included files. */
require_once(ABSPATH . 'wp-settings.php');

Step five - set up Docker Compose for the site

The recipe for the four Docker containers that together provide the WordPress multisite service is pretty straightforward. All you need to do is go into /home/docker/[domain name] and

sudo nano docker-compose.yml

and enter the following, replacing the [tokens] as usual - note, if you don't have the same value for [port] specified below as you do in your 'reverse proxy' configuration above, nothing will work:

version: "3"
 
services:
    redis:
        image: redis:alpine
        command: redis-server --requirepass [redis password]
        networks:
            default:
                aliases:
                   - redis.[domain name]
    php:
        image: oeru/php74-fpm-wpms
        links:
            - redis
        volumes:
            - /home/data/[domain name]/src:/var/www/html
        environment:
            - SMTP_HOST=[smtp host]
            - SMTP_PORT=587
            - SMTP_REPLYTO_EMAIL=[smtp reply-to email address]
            - SMTP_AUTH_USER=[smtp user]
            - SMTP_AUTH_PASSWORD=[smtp password]
        restart: unless-stopped
        networks:
            default:
                aliases:
                   - [domain name]
    nginx:
        image: oeru/nginx-buster-wp
        links:
            - php
            - redis
        ports:
            - "127.0.0.1:[port]:80"
        volumes:
            - /home/data/[domain name]/nginx/conf.d:/etc/nginx/conf.d
            - /home/data/[domain name]/nginx/cache:/var/cache/nginx
            - /home/data/[domain name]/src:/var/www/html
        restart: unless-stopped
        networks:
            default:
                aliases:
                   - nginx.[domain name]
    cron:
        image: oeru/php74-fpm-wpms-cron
        links:
            - php
            - nginx
        volumes:
            - /home/data/[domain name]/src:/var/www/html
        environment:
            - SMTP_HOST=[smtp host]
            - SMTP_PORT=587
            - SMTP_REPLYTO_EMAIL=[smtp reply-to email address]
            - SMTP_AUTH_USER=[smtp user]
            - SMTP_AUTH_PASSWORD=[smtp password]
        restart: unless-stopped
        networks:
            default:
                aliases:
                   - cron.[domain name]

Create the NGINX container's configuration

In our Docker Compose file, we've specified that our NGINX container will have its configuration in /home/data/[domain name]/nginx/conf.d, so let's put it there.

Run the following:

sudo mkdir -p /home/data/[domain name]/nginx/conf.d

to create the relevant directories, and then create the default configuration for the container:

sudo nano /home/data/[domain name]/nginx/conf.d/default.conf

Copy and paste this

# Caching configuration
# https://easyengine.io/wordpress-nginx/tutorials/multisite/subdirectories/fastcgi-cache-with-purging/
#
fastcgi_cache_path /var/cache/nginx levels=1:2 keys_zone=WORDPRESS:500m inactive=60m;
fastcgi_cache_key "$scheme$request_method$host$request_uri";
fastcgi_cache_use_stale error timeout invalid_header http_500;
 
# from https://www.nginx.com/resources/wiki/start/topics/recipes/wordpress/
#
# using subdir approach, not subdomain...
map $uri $blogname{
    ~^(?P<blogpath>/[^/]+/)files/(.*)  $blogpath ;
}
 
map $blogname $blogid{
    default -999;
    #Ref: http://wordpress.org/extend/plugins/nginx-helper/
    include /var/www/html/wp-content/uploads/nginx-helper/map.conf;
}
 
# statements for each of your virtual hosts to this file
server {
    listen 80;
    root /var/www/html;
    index index.php index.html index.htm;
 
    #
    # Caching configuration
    #
    #fastcgi_cache start
    set $skip_cache 1;
 
    # POST requests and urls with a query string should always go to PHP
    if ($request_method = POST) { set $skip_cache 1; }
    if ($query_string != "") { set $skip_cache 1; }
 
    # Don't cache uris containing the following segments
    if ($request_uri ~* "(/wp-admin/|/xmlrpc.php|/wp-(app|cron|login|register|mail).php|wp-.*.php|/feed/|index.php|wp-comments-popup.php|wp-links-opml.php|wp-locations.php|sitemap(_index)?.xml|[a-z0-9_-]+-sitemap([0-9]+)?.xml)") {
        set $skip_cache 1;
    }
    # Don't use the cache for logged in users or recent commenters
    if ($http_cookie ~* "comment_author|wordpress_[a-f0-9]+|wp-postpass|wordpress_no_cache|wordpress_logged_in") {
        set $skip_cache 1;
    }
    # end main Caching functionality
 
    #
    # Flow of serving pages
    #
    # from https://www.nginx.com/resources/wiki/start/topics/recipes/wordpress/
 
    #avoid php readfile()
    location ^~ /blogs.dir {
        internal;
        alias /var/www/html/wp-content/blogs.dir;
        access_log off; log_not_found off; expires max;
    }
 
    if (!-e $request_filename) {
        rewrite /wp-admin$ $scheme://$host$uri/ permanent;
        rewrite ^(/[^/]+)?(/wp-.*) $2 last;
        rewrite ^(/[^/]+)?(/.*\.php) $2 last;
    }
 
    # from https://www.digitalocean.com/community/tutorials/how-to-install-wordpress-with-nginx-on-ubuntu-14-04
    # with other bits from https://premium.wpmudev.org/blog/wordpress-multisite-wordpress-nginx/
    location / {
        try_files $uri $uri/ /index.php?$args;
    }
    error_page 404 /404.html;
 
    # Directives to send expires headers and turn off 404 error logging.
    location ~* ^.+\.(xml|ogg|ogv|svg|svgz|eot|otf|woff|mp4|ttf|css|rss|atom|js|jpg|jpeg|gif|png|ico|zip|tgz|gz|rar|bz2|doc|xls|exe|ppt|tar|mid|midi|wav|bmp|rtf)$ {
        access_log off; log_not_found off; expires max;
    }
 
    location ~ \.php$ {
        try_files $uri =404;
        fastcgi_pass php:9000;
        fastcgi_split_path_info ^(.+\.php)(/.+)$;
        fastcgi_keep_conn on;
        fastcgi_index index.php;
        include fastcgi_params;
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
        fastcgi_param PATH_INFO $fastcgi_path_info;
        fastcgi_intercept_errors on;
        fastcgi_buffer_size 128k;
        fastcgi_buffers 256 16k;
        fastcgi_busy_buffers_size 256k;
        fastcgi_temp_file_write_size 256k;
        fastcgi_read_timeout 500;
        #
        # caching functionality
       fastcgi_cache_bypass $skip_cache;
        fastcgi_no_cache $skip_cache;
       fastcgi_cache WORDPRESS;
       fastcgi_cache_valid 60m;
    }
    #
    # caching functionality
    location ~ /purge(/.*) { fastcgi_cache_purge WORDPRESS "$scheme$request_method$host$1";    }
    location = /favicon.ico { log_not_found off; access_log off; }
    location = /robots.txt { allow all; log_not_found off; access_log off; }
    location ~ ^(/[^/]+/)?files/(.+) {
        try_files /wp-content/blogs.dir/$blogid/files/$2 /wp-includes/ms-files.php?file=$2 ;
        access_log off; log_not_found off; expires max;
    }
}

and save and exit the file.

And now we have to create the following directory and file:

sudo mkdir -p /home/data/[domain name]/src/wp-content/uploads/nginx-helper

sudo touch /home/data/[domain name]/src/wp-content/uploads/nginx-helper/map.conf

Once that's done, we're ready to launch our containers... Phew.

Launch containers

Once that's done you can download the relevant containers (you may need to set up an account on https://hub.docker.com first - do that and then run docker login before doing the following):

docker-compose pull && docker-compose up -d && docker-compose logs -f

which will show you the combined logs of the four containers and should give you some insights if something goes wrong. If all is well, you can type CTRL-C to exit it. The Docker containers (to see what's running, you can run docker-compose ps) will run until you explicitly stop them (docker-compose stop). Unless you stop them prior to lock down, they will automatically restart anytime your server reboots.

Step Six - set up your site

Install WordPress

You should now be able to point your browser at https://[your domain] and you should automatically be redirected to the WordPress site install script, with all the database details already entered. You'll need to fill in the configuration fields and create an admin user.

Enable the OERu Course theme

You should already be in the /home/docker/[domain name] directory, but if not,

cd /home/docker/[domain name]

and then you can enable the OERu Course theme, oeru-course, using WordPress's command line utility, wp, via your PHP container:

docker-compose exec -u www-data php wp theme enable oeru-course

You should get the message Success: Enabled the 'OERu Course' theme. if all went well.

Create an admin user

Your admin user should probably be an account not normally used by a person. I usually create an admin user with a username of 'admin', a random password (using pwgen as above) and a role-based email like webmaster@[domain name] (if your domain name has email services) or some similarly useful generic email (using that protects against the 'tyranny of the individual', e.g. if you leave the organisation in whose interest you're setting up this site, and those left behind will have to make sense of this site and keep it running!

Later one, if you want to log into the site, you can always get a login prompt by going to https://[domain name]/admin/ ...

Enable multisite

The WordPress configuration we've set up in the wp-config.php file should have enabled the inbuilt functionality for enabling 'multisite' mode (or 'network', as it used to be called, i.e. for a "Network of Blogs", back when WordPress was more exclusively used for blogging). WordPress developer documentation on 'Creating a network' might be a useful reference if you run into any trouble.

Logged in as your admin user, in the admin menu structure go to Administration >Tools > Network Setup

You will be asked ot provide your Network Title and Network Admin Email (which can be the same as your current admin email).

You will also be asked whether you want to use the 'Sub-domains' or 'Sub-directories' structure for your network. Select the Sub-directories option (you will need to make changes your Nginx configurations and the wp-config.php configuration to use Sub-domains, which are beyond the scope of this howto), which means that each of your subsites will have a separate directory under your main site [domain name]. For example, on the OERu Course site, course.oeru.org, the Learning in a Digital Age 101 course sub-site is course.oeru.org/lida101. If you chose to use the Sub-domain option, you would instead reference that course as lida101.course.oeru.org.

Once you've enabled multisite, you need to update your wp-config.php -

sudo nano wp-config.php

and make the following change, to comment out the WP_ALLOW_MULTISITE variable, and uncomment the various MULTISITE-related settings:

/* Multisite */
// see https://wordpress.org/support/article/create-a-network/ 
//define('WP_ALLOW_MULTISITE', true);
define('MULTISITE', true);
define('SUBDOMAIN_INSTALL', false);
define('DOMAIN_CURRENT_SITE', '[domain name]');
define('PATH_CURRENT_SITE', '/');
define('SITE_ID_CURRENT_SITE', 1);
define('BLOG_ID_CURRENT_SITE', 1);

After you've saved that, and refreshed the page on your WordPress site, you should be looking at a fully functioning multisite!

Enable the relevant plugins

Third Party plugins

Now we need to get the specific OERu plugins that are not (yet) available through the WordPress plugin site.

The ones we use by default are

advanced-responsive-video-embedder, check-email, disable-comments, h5p, hypothesis, redis-cache, safe-redirect-manager, unconfirmed, wp-security-audit-log

As we did with the OERu Course theme, we'll install and active these plugins using WordPress's command line utility, wp, via your PHP container:

cd /home/docker/[domain name]

you can see the names of all your running containers by running this:

docker-compose ps

and you can download and activate all these plugins in 'network' (aka multisite) mode like this:

docker-compose exec -u www-data php wp plugin install advanced-responsive-video-embedder --activate-network
docker-compose exec -u www-data php wp plugin install check-email --activate-network
docker-compose exec -u www-data php wp plugin install disable-comments --activate-network
docker-compose exec -u www-data php wp plugin install h5p --activate-network
docker-compose exec -u www-data php wp plugin install hypothesis --activate-network
docker-compose exec -u www-data php wp plugin install redis-cache --activate-network
docker-compose exec -u www-data php wp plugin install safe-redirect-manager --activate-network
docker-compose exec -u www-data php wp plugin install unconfirmed --activate-network
docker-compose exec -u www-data php wp plugin install wp-security-audit-log --activate-network

These plugins should work as required without any specific configuration, but you're welcome to have a look at what they're doing and tweak them as you see fit.

OERu plugins

The final step is to install and enable the custom-developed plugins required to make this WordPress installation meet the requirements of an OERu Course site.

At the OERu (and the OER Foundation, who coordinates the OERu and employs me) we make extensive use of Git to manage our source code and to deploy it on our various servers. We'll use it here.

You'll need to go back to the plugin directory:

cd /home/data/[domain name]/src/wp-content/plugins

and run the following:

sudo git clone https://git.oeru.org/oeru/blog-feed-finder.git blog-feed-finder
sudo git clone https://git.oeru.org/oeru/oeru-h5p-tools.git oeru-h5p-tools
sudo git clone https://git.oeru.org/oeru/register-enrol.git register-enrol
sudo git clone https://git.oeru.org/oeru/wenotes.git wenotes
sudo git clone https://git.oeru.org/oeru/wpms-activity-register.git wpms-activity-register
sudo git clone https://git.oeru.org/oeru/wpms-mautic.git wpms-mautic

and the WEnotes plugin has a further dependency, our WEnotes Aggregator code.

cd wenotes
sudo git clone https://git.oeru.org/oeru/wenotes-aggregator.git wenotes-aggregator

We need ot make sure the plugins are all readable by the webserver, so

cd .. && sudo chown -R www-data ../plugins

And we have to make a couple tweaks to configurations of a couple plugins.

WEnotes tweaks

You'll want to set your WENOTES_SOURCE_NAME and WENOTES_SOURCE_URL so that your WEnotes are properly attributed.

[geshifilter-]cd /home/data/[domain name]/src/wp-content/plugins sudo nano wenotes/wenotes.php [/geshifilter-]

and edit the following values (between the ' ') to be suitable for your organisation! Put in your preferred URL as well.

define('WENOTES_SOURCE_NAME', 'OERu Course Site');
define('WENOTES_SOURCE_URL', 'course.oeru.org');

Register Enrol tweaks

Similarly, you are likely to want to change the following settings in the Register Enrol plugin configuration to reflect your site and organisation. First, edit register-enrol.php

sudo nano register-enrol/register-enrol.php

and alter the following values (between the ' ') to suit (we don't recommend change other settings unless you know what you're doing!):

// support link for users of this plugin...
define('ORE_SUPPORT_FORUM', 'https://forums.oeru.org/t/register-enrol');
define('ORE_SUPPORT_BLOG', 'https://course.oeru.org/support/studying-courses/register-enrol/');
define('ORE_SUPPORT_CONTACT', 'https://oeru.org/contact-us/');
define('ORE_SUPPORT_PASSWORD_MANAGER', 'https://course.oeru.org/lida102/learning-pathways/digital-environments/online-hygiene/#Password_managers');

and

define('ORE_DEFAULT_FROM_EMAIL', 'webmaster@oerfoundation.org');

Mautic Integration tweaks

You might also want to set up an OERu Partner record for your organisation (if you're not already a partner!) in our Mautic integration plugin in the file wpms-mautic/includes/mautic-sync.php in the $partner_names array and then and set your MAUTIC_DEFAULT_PARTNER to have your partner name in wpms-mautic/mautic-app.php.

Reasserting ownership

And a final step - this is crucial, as it will allow you to keep your WordPress instance up-to-date using the various WordPress standard approaches - is to ensure that the webserver user, www-data is the owner of the source code in your site:

sudo chown -R www-data /home/data/[domain name]/src

That's it. Done. Phew.

Step Seven - celebrate!

You've done it! You've got yourself a new Course WordPress multisite! Which, if I do say so myself, is an impressive accomplishment in it's own right.

Of course, it might not be as exciting as actually having a real live course hosted on your Course WordPress multisites... so there's one more step (maybe take a quick break to celebrate getting to this milestone before proceeding).

Snapshotting your first OER course!

There are quite a few courses (fully accredited!) available to push to your Course multisite on WikiEducator where we work with educators around the world to assemble OER-based courses, usually split into discrete 'micro-courses' that can be assembled to meet the credit requirement conventions in different parts of the world.

Here is an example course you can add to verify the process: the first micro-course in our (award winning) Learning in a digital age, called Digital literacies for online learning. That link points to the Outline for the course, which in turn shows all the resources making up the course materials in the hierarchy that will become the navigation of the resulting Course site on your WordPress Multisite. The way e get it there is by using our "Course Snapshot" process which converts that WikiEducator content in the outline into a WordPress content archive that it then pushes onto your designated Course subsite.

Before we can run a snapshot, you'll need to

request a WikiEducator account - unfortunately this is a moderated process (i.e. a person at the OER Foundation has to review your request) because we have lots of problems with would-be spammers. So this could take a fair while depending on when you request it (we're on NZ time). So much for instant gratification. Sorry about that.
you'll need to create a subsite (you don't want to replace your multisite's base site).

To do the latter, when you're logged into your WordPress multisite as the admin user, use the top menu to go to 'My Sites' -> 'Network Admin' -> 'Sites' and click the 'Add New' button. You'll need to specify a "Site Address (URL)" which is just the path following [domain name] in referencing your site. For example, the LiDA 101 course on the OERu's Course site is https://course.oeru.org/lida101 where lida101 is that path. You could use that same path here if you want (it's appropriate for this course).

You'll also need a title - you could use "Digital literacies for online learning" - and pick a Site Language (the default is probably what you want). For the 'Admin Email', use the email of your admin user, as WordPress will then make the user with that email, namely your admin user, the administrator of that lida1010 subsite, which is what we want.

Next, we go back to the WikiEducator page for the LiDA 101 outline above and find the 'Request snapshot' button near the top of the page. This is the important bit - Note: doing this will remove any existing content in the subsite you specify! Make sure that's what you're intending - clicking it will give you a dialog box into which you'll enter the full location of your subsite, which will look like https://[domain name]/[subsite name] (for example, in the OERu case it is https://course.oeru.org/lida101) , making suitable [token] substitutions, of course, and enter as your WordPress admin user and password. Clicking "Push snapshot to WordPress" sets things in motion, and, all going well, you should receive an email in a few minutes (5-20, usually) when the system has got to your request (it does them on a first-come, first-served basis) and processed it successfully. If you get that email, have another look at your site! It should look very similar to what you see on https://course.oeru.org/lida101 with the main difference being (possibly) the colour scheme and the lack of a Login/Register prompt (top right). That can be fixed as I explain next.

Enabling OERu's Register Enrol functionality

The interface the OER Foundation has developed to help learners register for the site and enrol in specific courses is not enabled by default and needs to be turned on for any given Course sub-site (this is useful for courses that are visible on the site, but not yet ready to accept enrolments). To do so, go to the relevant course site dashboard in the WordPress menu, and select Appearance -> Customize. In the resulting theme customisation side panel, selelct 'Site Navigation', and then set 'Show the login option?' to 'Yes' and 'Publish' to save the setting. Your login prompt should show near the top right corner of all the pages in that course site at that point.

Also note, you can change the pre-set colour palette for your course sites on a per-site basis using the menu combination OERu Theme -> Colour Scheme...

Data Backups

Your MariaDB will already be getting backed up daily via the automysqlbackup script installed towards the start of this process, but you also want to have backups of files and configurations on your server, and you want to have backups on a different server, i.e. remote to your server, as a matter of disaster-recovery prudence.

We will be describing how we do remote, encrypted, incremental backups in a separate how-to and will link to it here as soon as it's available.

Staying up-to-date

You should log into your site as your admin user or as your personal user (granted Administrator permissions by your admin user!) periodically to see if there are any updates available for your site - if there are, you can update them as per the instructions provided by the site.

The OERu plugins and themes undergo periodic improvements or bug fixes. To find out about them, you'll need to keep track of the various Git repositories on our Gitlab instance.

These are the relevant repositories:

We're always delighted to hear from folks using any of these components and invite people to collaborate with us on improving any and all of these software projects, including providing access to our Gitlab!

Also, from time to time, we might update the Docker containers we use for this service, and you're always welcome to make use of our updated containers. If you have any questions, feel free to get in touch or leave a comment below!!

Acknowledgements

Many thanks to the good folks at the Samoan Ministry for Education, Sport, and Culture and UNESCO in Apia for motivating me to write this up, and to educational technologist luminary Stephen Downes for unexpectedly finding this tutorial and then (even more unexpectedly) heroically and comprehensively going through it and providing extensive editorial input (part 1 and part 2), most (if not all) of which I've since incorporated! Thanks Stephen - see people were listening (just not in real-time)!

Add new comment

WikiEducator Notes: OERu's course feed aggregation and messaging system

dave — Wed, 23 Aug 2017 21:21:04 +0000

WikiEducator Notes: OERu's course feed aggregation and messaging system

Blog tags

wenotes

couchdb

faye

javascript

node.js

docker

docker compose

mastodon

hypothesis

oeru

lida101

mediawiki

wikieducator

wordpress

dave Thu 24/08/2017 - 09:21

Here at the OERu, we provide a service, attached to all of our online courses (and available to all of our partners - or anyone else for that matter) which allows anyone involved in those courses to communicate with their peers from any one of a dazzling array of online "places" with WikiEducator Notes (aka WEnotes). The entire system is free and open source software (FOSS).

The magic glue is "tags" - most social media and online systems support "tagging" in one way or another. Tools like Twitter, Mastodon, and G+ support "hashtags" like "#OERu" as shorthand. Other tools like blog engines (Medium, WordPress, and others), forums (like Discourse), content annotation services (like Hypothesis), social bookmarking services (like Semantic Scuttle), and instant messaging services (like Rocket.Chat, Riot + Synapse, Zulip, and Mattermost, among others) support adding "tags", "labels" or "categories" to any given post. WEnotes is sensitive to those tags, and thanks to the modern trend towards services providing feeds, it can happily "harvest" relevant content from all over the web, in real time (or nearly, depending on the technology).

How WEnotes components fit together

WEnotes is a collection of a significant number of separate FOSS components. See the graphical diagram of WEnotes alongside this article for an overview of the WEnotes component architecture. To provide further explanation, WEnotes is made up of several functional components:

A set of feed harvesting scripts that are part of the WEnotes-tools project (written in Javascript and Python, for those interested in that sort of detail, the former ideal for those sites producing JSON feeds and real time updates, the latter our choice for periodic scans - every 10 minutes by default at the moment - of RSS and ATOM feeds)
A central CouchDB instance, which stores all posts harvested, along with metadata like their source, a unique ID (to ensure we don't re-harvest it) and some useful info to ensure we can push the messages to subscribers effectively.
The WEnotes client for MediaWiki or WordPress, part of the WEnotes project, which provides a real-time feed of posts, either a full feed (for example this one) or specific to a tag (for example a specific OERu Course). The client also offers a posting interface, allowing logged in users to post new posts (up to 300 characters) automatically tagged in the relevant context. See two screenshots attached.
A "publish-subscribe" or "Pub-Sub" service, WEnotes-server project, a Node.JS based Faye implementation of the Bayeux protocol. It uses websockets to push out real-time updates to subscribed WEnotes clients.
The "couchwatch.js" script, also part of the WEnotes-tools project, watches the CouchDB and alerts Faye whenever a new post is received, which in turn pushes it out to subscribed WENote clients.

Whence WEnotes posts come

Here are three scenarios which outline the process by which a WEnote post is created.

Handy WEnotes post form

The most obvious way to create a WEnotes post is to go to a WEnote feed page - for instance - and log into the system (assuming you have authentication credentials or can create some). If you're logged in, you should see a simple form as illustrated in the two attached screen shots. The system then knows who you are so that your post can be properly attributed, and it knows the "context" for the feed you're looking at (namely the associated tag). You can enter a message of up to 300 characters, and hit the "Post a WEnote" button, and this will send - direct from your browser - a suitably formatted post (a simple JSON object - example below) into the WEnotes CouchDB where it is then available for inclusion in relevant feed displays.

Personal blog post, appropriately tagged

If you register for our Course site, or any course on that site, you have the option of adding a "Blog URL" - that's the web address of your blog (the address of this blog is https://tech.oeru.org, for instance). Because most widely used blogging platforms, like the open source WordPress platform (which is the most widely use web platform in the world, the basis for 25% of all websites!! For the record, this website uses Drupal, which is the 3rd biggest web platform, and is also open source), and proprietary cloud blogging platforms like Blogger and Medium, provide both "tagging" of posts and a built-in RSS feed. When you know what to look for, you'll start to see them all over the place (this blog has one - the logo is the little grey icon on the bottom left corner of this page).

When you create a blog post on your registered blog and tag it with a course-specific tag (each course we run has one, it's usually the short code immediately after the main course website URL: https://course.oeru.org/[course code here]/...) the WEnotes feed harvesters will periodically visit your blog's RSS feed and look for new posts tagged with a course tag. If it finds one, it will create a new post in CouchDB with all the necessary info to provide an informative post in the WEnotes feed, proper author attribution (and an "avatar" picture of you where available!), and a way for interested readers to get to your original post.

Tagged/Hashtagged social media reference

Because the attention of many people is heavily focused on social media - dominated by "network effect" companies like Twitter and Facebook, for better or worse, but also by an array of more open and egalitarian services (which we, on principle, prefer, but recognise that they're far less well marketed due mostly to not taking anyone's money) - we also provide a variety of social media specific "feed harvesters". Some of these periodically scan specific social media technologies for new relevant posts, others perpetually monitor their sources providing instant updates.

If you

Toot including a relevant hashtag (e.g. #oeru or #lida101 for the "Learning in a Digital Age 101" course),
use Hypothesis to annotate a web page, tagging it appropriately (e.g. oeru or lida101),
add a Semantic Scuttle "Bookmark" to a web reference of interest, tagging it appropriately,
add a topic or reply on our Community or Forums Discourse instances also suitably tagged, or
create a Moodle post on a Moodle site running a suitable "student bot",

the WEnotes feed harvesters will find it and create a suitable post in the CouchDB.

How WEnotes feeds are made

Once you have a bunch of WEnotes posts stored in CouchDB, how do we create a feed? At present, we have "WEnotes client" scripts that run either on MediaWiki instances, like Wikieducator, or on WordPress sites, like our Course multisite, for which content is usually provisioned per-course using our "Snapshot" toolchain. In both cases, the client is invoked via the inclusion of a "Widget" in the MediaWiki content that will either be viewed on the MediaWiki instance itself, or on a target WordPress course subsite.

After you log into the Wikieducator site, and create or find a page on which you want a feed, you "Edit source" and you can add both a WEnotes Post template and WEnotes Feed template two simple one line recipes like this:

{{#widget:WEnotesPost|tag=lida101}} {{#widget:WEnotes|tag=lida101|count=20}}

This invocation will create a page showing a WEnotes Post Widget, where any submitted post will automatically be tagged by "lida101", and directly below that, a feed showing up to 20 posts from the CouchDB tagged with "lida101". Note, any new suitably tagged posts will pop into the feed in realtime as they're added to CouchDB thanks to couchwatch.js and Faye - invoking a WEnotes Feed Widget automatically subscribes you to realtime updates for that tag.

You can also create a cross-tag feed using the magic "_" designator. WEnotes Posts require a tag (the default, if none is specified, is "wikieducator"):

{{#widget:WEnotes|tag=_|count=20}}

WEnotes technical details

This is a tech-blog, so I'm not going to shy away from the technical details. Here're some useful bits and pieces for folks wanting to get a better understanding of this system. If you're scared of technical details, you can stop reading here.

WEnotes technology infrastructure

We implement the services making up the WEnotes stack using a set of three Docker containers as well as our main MediaWiki implementation, WikiEducator, and our main WordPress multisite implementation, the OERu Course multisite (we also maintain development instance of each of these). The Docker containers are managed with Docker Compose and the roles are divided We've created a separate code repository on GitHub for our evolving WEnotes stack to facilitate others making use of some or all of it! The three containers currently being deployed do the following:

run CouchDB version 2.0 or later, as well as its "Fauxton" interactive web front-end.
run the Faye Pub-Sub websocket destination for realtime feed updates, implemented as a Node.JS service running under pm2.
run the Javascript (using pm2) and Python-based (using cron to run periodically) feed harvesters and the Javascript "couchwatch.js" script which alerts Faye to the addition of new posts.

Once configured (we're working on making that a straight forward "out-of-the-box" experience, although it's not quite there yet), the three containers can be deployed together with a single invocation of Docker Compose - at the moment, though, this system is quite dependent on a properly configured "options.json" file, the format of which is still in a state of flux.

WENotes JSON object

A WEnotes post is stored in our CouchDB as a JSON object (JSON = JavaScript Object Notation). This is an example of a Post as it's stored in our CouchDB:

{
"_id": "3e1c16eb1c99ae59899761a680006f4a",
"_rev": "1-7462f14a0515d98b5985b9f4b8c39dd5",
"media_attachments": [
 {
 "url": "https://mastodon.oeru.org/system/media_attachments/files/000/000/017/or…",
 "text_url": "https://mastodon.oeru.org/media/jvUoIpj6OAt-e5t-H8k",
 "preview_url": "https://mastodon.oeru.org/system/media_attachments/files/000/000/017/sm…",
 "meta": {
 "small": {
 "width": 400,
 "size": "400x321",
 "aspect": 1.2461059190031152,
 "height": 321
 },
 "original": {
 "width": 1148,
 "size": "1148x921",
 "aspect": 1.246471226927253,
 "height": 921
 }
 },
 "type": "image",
 "id": 17,
 "remote_url": ""
 }
],
"truncated": true,
"reblog": null,
"id": 21,
"in_reply_to_id": null,
"content": "Productive weekend fixing hashtags and instructions for <a href=\"https://mastodon.oeru.org/tags/lida101\" class=\"mention hashtag\" rel=\"tag\">#LiDA101</a> photo challenges . Super cool to see mastodon.oeru,org toots in the course feed. <a href=\"https://mastodon.oeru.org/tags/lida101photo\" class=\"mention hashtag\" rel=\"tag\">#lida101photo</a> <a href=\"https://mastodon.oeru.org/media/jvUoIpj6OAt-e5t-H8k\" rel=\"nofollow noopener\" target=\"_blank\">https://mastodon.oeru.org/media/jvUoIpj6OAt-e5t-H8k</a>",
"application": null,
"text": "Productive weekend fixing hashtags and instructions for #LiDA101 photo challenges . Super cool to see mastodon.oeru,org toots in the course feed. #lida101photo...",
"profile_url": "https://mastodon.oeru.org/@mackiwg",
"tags": [
 {
 "url": "https://mastodon.oeru.org/tags/lida101",
 "name": "lida101"
 },
 {
 "url": "https://mastodon.oeru.org/tags/lida101photo",
 "name": "lida101photo"
 }
],
"visibility": "public",
"we_tags": [
 "lida101"
],
"user": {
 "screen_name": "mackiwg",
 "profile_image_url": "https://mastodon.oeru.org/system/accounts/avatars/000/000/002/original/…",
 "name": "Wayne Mackintosh"
},
"spoiler_text": "",
"we_timestamp": "2017-09-10T04:09:05.000Z",
"account": {
 "username": "mackiwg",
 "display_name": "Wayne Mackintosh",
 "statuses_count": 10,
 "following_count": 1,
 "url": "https://mastodon.oeru.org/@mackiwg",
 "locked": false,
 "created_at": "2017-04-30T01:51:59.778Z",
 "avatar_static": "https://mastodon.oeru.org/system/accounts/avatars/000/000/002/original/…",
 "note": "Open sourcing education at OERu",
 "header": "https://mastodon.oeru.org/headers/original/missing.png",
 "followers_count": 1,
 "avatar": "https://mastodon.oeru.org/system/accounts/avatars/000/000/002/original/…",
 "header_static": "https://mastodon.oeru.org/headers/original/missing.png",
 "acct": "mackiwg",
 "id": 2
},
"favourites_count": 0,
"language": "en",
"url": "https://mastodon.oeru.org/@mackiwg/21",
"we_source": "mastodon",
"in_reply_to_account_id": null,
"uri": "tag:mastodon.oeru.org,2017-09-10:objectId=21:objectType=Status",
"reblogs_count": 0,
"sensitive": false,
"mentions": [],
"created_at": "2017-09-10T04:09:58.197Z"
}

The most crucial things for WEnotes are the values that start with 'we_'. They are used by the system which filters and displays WEnotes posts.

The OERu Tag List

The list of tags which OERu's WEnotes looks for changes from time-to-time, usually growing as we add new course tags or other tags of business. Currently, the list is being updated manually, but it is available as a web feed in JSON format. We intend to provide both a simple editing interface for this list and to provide an API to automate adding new tags. Eventually, when we create a new course on our Course multisite, we expect to be able to automatically register the new course "tag" with the tag list.

Credits

The entire underlying mechanics and most of the cleverness of WikiEducator Notes are thanks to the ingenuity and hard work of Jim Tittsler, my predecessor as OER Foundation Technical Lead.

Add new comment

OERu MediaWiki to WordPress Snapshot Toolchain

dave — Thu, 15 Jun 2017 23:19:06 +0000

OERu MediaWiki to WordPress Snapshot Toolchain

Blog tags

wikieducator

wordpress

multisite

snapshot

mediawiki

node.js

php

dave Fri 16/06/2017 - 11:19

One of the OERu's most compelling technological capabilities is the set of internally developed open source tools that allow us to automatically transform and inject a collection of MediaWiki content on WikiEducator (usually in form of micro-courses) making up a course into a mobile-friendly, easy-to-navigate WordPress site, usually situated on OERu's Course WordPress Multisite implementation.

How it works

Educators, usually working with one of our OERu partner institutions, can assemble the materials (either written by them or recruited from existing OERs on Wikieducator) into an "outline" which references all the content in one place. If they want to make the course available (either a cohort-based group or a "rolling" enrolment, either affiliated with their institution or under the OERu banner) they can do so either on a subsite on OERu's WordPress instance or some other WordPress, for example one hosted by their institution.

MediaWiki Widget

The process of linking an outline to a WordPress site and pushing a "snapshot" of the course content outlined to the WordPress site is remarkably simple. First, the author of the outline page includes a one line MediaWiki template or "widget" - which is managed by the open source OERu MediaWiki plugin (written in PHP, like the rest of MediaWiki) developed by Jim Tittsler - that will look something like this (usually in one line, expanded here for clarity):

{{ #widget:Snapshot | url=http://course.oeru.org/shortcode/ | logo=https://wikieducator.org/images/f/f1/MyInstitutionLogo.png | link=http://oeru.org/oeru-partners/my-institution-profile/}}

This defines

the URL of the WordPress site, using the "shortcode" the educator has defined for that course (like the institution's course code),
a logo image to display at the bottom of each page of the WordPress course site to identify the relevant institution, and
the link to which a user will be sent if they click on the logo.

There is a handy instruction page for the widget if you want a bit more detail.

Course Snapshot Script

When the page containing this template is saved, the resulting page will have a "Request snapshot" button at the top of it (see attached screenshot). Clicking this will popup a modal dialog box into which the user will be asked to enter their login and password for an admin user on the listed WordPress site (either multisite or stand-alone implementation).

If the WordPress site in question accepts these credentials, a new Snapshot task is added to WikiEducator's task list, handing over control to Jim's rather clever open source Course Snapshot script which runs in the background, and checks the task list ever minute or two. When it identifies as snapshot request, it uses the outline page and WikiEducator's API to grab all the relevant content automatically, and it builds a structured data archive (in a variant of XML) which, when completed, is imported directly into a WordPress site via WordPress' XMLRPC capabilities, replacing any existing content.

Note: the default export format used by the Course Snapshot script is a WordPress-importable data object, however for historical reasons, it also has the ability to generate (more manually at this stage) a static, internally linked HTML version of the content.

WordPress Mobile-Friendly Course Site

The resulting WordPress course site uses the OERu-created open source mobile-friendly Course WordPress theme - see the sample page screen shot. Each page includes a link back to the original page in WikiEducator from which it was generated - see sample page screen shot with the link identified. Also see the sample image of the same page as it would look on a mobile device with a narrower (lower pixel resolution) screen width.

Note: anyone can copy an outline page written by someone else and use it as the starting point for assembling their own version of the course - they will, however, have to have admin rights on the target WordPress site (and log in to it to initiate the process) protecting any existing snapshot sites from being overwritten by someone else's course content.

More examples

Here's a list of some of the currently available course outlines on WikiEducator with corresponding links to the "rendered" WordPress courses so you can see what it looks like.