14.04 Protecting your users with Let's Encrypt SSL Certs

dave — Thu, 08 Jul 2021 02:23:06 +0000

Protecting your users with Let's Encrypt SSL Certs

Blog tags

let's encrypt

install

ubuntu linux

14.04

16.04

18.04

20.04

dave Thu 08/07/2021 - 14:23

SkFor any website that requires anyone (users or even just a few admins) to transfer secrets to and from it, you want to ensure the data is end-to-end encrypted. Today various browsers (like Firefox) give warnings when you're sending secret data (like passwords) "in the clear", namely unencrypted. In early 2017, Google added further urgency to doing the right thing for your users.

In the past, getting an SSL certificate to achieve encryption for your domain (the little "lock" icon in browser address bar indicating that your communication with the site is encrypted), was a complicated, expensive proposition, requiring a lot of annoying and time consuming "identity verification" (sometimes via post in the dark old days) and a fee of, in some cases, a couple hundred dollars per year paid to your "SSL Cert Provider" to pay for those administrative services along with the software needed to gin up a long prime number to act as your encryption key (the long string of characters making up your SSL certificate).

Thankfully, thanks to the efforts of the Let's Encrypt community, the process is both far far easier, and free of cost. Now there really isn't an excuse for not having an SSL certificate on your site.

Members of the Let's Encrypt community have provided a range of useful open source tools you can use to create and maintain certificates on your hosting infrastructure (e.g. the Virtual Machine (VM) on which you're installing web services detailed in the howtos on this site!). In this case we're going to use a tool, "certbot" provided by the good folks at the Electronic Frontier Foundation.

For VMs running Ubuntu 18.04 or more recent Long Term Support (LTS) versions of the Ubuntu Linux platform, you should be able to just run:

sudo apt-get install letsencrypt

For VMs running Ubuntu 14.04 or 16.04 which are what we use, the install is easy - at your VM command line, run:

sudo add-apt-repository ppa:certbot/certbot
sudo apt-get update
sudo apt-get install certbot

We run Nginx on our VMs, which makes one or more hosted services (normally running in Docker containers) available on the Internet. Strictly speaking, we don't use full "end-to-end" encryption - in our case, on the server-end the encryption terminates at the Nginx server. We, perhaps cavalierly, assume that transfer between the host machine and a Docker container running on that host will be implicitly secure... The only way it could be compromised is if the VM itself is compromised, in which case, the Docker containers running on it could be, too. Avoiding having secure transfer between Nginx on the VM host and the various Docker containers also substantially simplifies setting up each application.

Thanks to a service which Nginx provides SNI (or Server Name Indication) - Apache and a few other web servers also provide this - which removes the historical limitation that meant you could only have one SSL certificate per IP address on a web server. The only downside of SNI is that some older browsers (and platforms) don't support it. Since those older technologies are rapidly dying out and it's quite expensive and difficult to have a single IP address for each SSL service on a server, we accept this compromise.

The Let's Encrypt Cert Process

Here's (roughly) how the process works:

Point your domain (via A or CNAME record) to point to the/an external IP address on your VM.
Set up a domain (or domains) for non-secure hosting (on port 80) via your Nginx instance.
The domain's configuration must include a special directory reserved for Let's Encrypt verification content.
You request that certbot (on the VM) acquires a certificate for that domain (or domains) at which point
1. the certbot writes a file with a hard-to-guess name to that special directory and requests that the Let's Encrypt infrastructure checks the domain name from outside
2. Let's Encrypt checks that domain name and special directory to see that the expected number appears there, thus verifying that the requesting party actually has the ability to set content at this hard-to-guess filename, and therefore has legitimate claim to being the party controlling the domain name and server.
3. Let's Encrypt registers the certificate request in the name of the party running the certbot (so it can, for example, send emails to the administrator warning them that the certificate needs to be renewed - which happens every 8 weeks or so), and
4. Let's Encrypt sends verification back to your VM's certbot telling it to complete the certificate generation, which it then (digitally) signs in the name of the Let's Encrypt Certificate Authority (which, in turn, is recognised by almost all web browsers out-of-the-box - no mean feat, I can tell you).
You get an alert telling you that you have created a valid SSL certificate.
You alter your Nginx domain configuration to
1. redirect connections to port 80 (un-encrypted) to port 443 (encrypted), and
2. you set up the 443 configuration including your new certificates.
You reload your Nginx configuration, and your site will now be end-to-end encrypted.

The Let's Encrypt Snippet

To make it easy to include the relevant directory info, I recommend that you create a new file in your Nginx configuration (substitute your preferred text editor for "vim" in the following - "nano" is a good choice if you haven't already got a preference):

sudo mkdir /etc/nginx/includes sudo vim /etc/nginx/includes/letsencrypt.conf

and make sure it has the following content (note, I learned this thanks to someone else's howto on the global Internet knowledge commons :))

# Rule for legitimate ACME Challenge requests location ^~ /.well-known/acme-challenge/ { default_type "text/plain"; # this can be any directory, but this name keeps it clear root /var/www/letsencrypt; }

# Hide /acme-challenge subdirectory and return 404 on all requests. # It is somewhat more secure than letting Nginx return 403. # Ending slash is important! location = /.well-known/acme-challenge/ { return 404; }

Next, make sure your directory exists (note - you only need to do this once per VM) - it shouldn't need an special permissions - it'll be written by the "root" user, and needs to be readable by the Nginx user, usually "www-data" on a Debian or Ubuntu Linux instance.

mkdir /var/www/letsencrypt

Example Nginx Domain Configuration - unencrypted

Here's an example of a pre-cert Nginx domain configuration for example.org and www.example.org (I usually name the configuration file after the main domain it concerns, so my file would be /etc/nginx/sites-available/example.org) - this should also let you do initial test of your app to make sure it works, before adding the additional complexity of SSL. (Replace example.com (and www.example.com) with your own domain!):

server {

listen 80; # this is one of our external IPs on the server. #listen [::]:80 default ipv6only=on; ## listen for ipv6 # this root directory isn't really relevant in a proxy situation
# so I usually set it to the system default root /usr/share/nginx/www; index index.html index.htm; server_name example.org www.example.org; access_log /var/log/nginx/example.org_access.log; error_log /var/log/nginx/example.org_error.log;

# this is where we include the snippet include /etc/nginx/includes/letsencrypt.conf;

# this is just an example of a "proxy" configuration # for, say, a Docker-based service, exposed on the VM's # local port 8081 location / { proxy_read_timeout 300; proxy_connect_timeout 300; proxy_redirect off; proxy_set_header Host $http_host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; proxy_pass http://127.0.0.1:8081; }
}

You can make sure that the configuration is visible to Nginx by adding it into the "sites-enabled" directory via a file link

sudo ln -sf /etc/nginx/sites-available/example.com /etc/nginx/sites-enabled

Test things to make sure the new configuration doesn't have typos or bad references:

sudo nginx -t

and if not, make it live:

sudo service nginx reload

You should now be able to go to http://example.com (or your domain) and you'll hopefully get your proxied application (if it's set up) or an Nginx error (see you nginx error file for more info!).

Now it's time to request the certificate!

Example Certbot invocation

Once cerbot is installed, and a domain is configured, it's pretty straightforward to get a certificate.

On the first invocation of certbot, you might get a coloured interface that requests your user details (e.g. name and email address) so that Let's Encrypt can register them for the purposes of future emails. They email if one of your certificates is on the verge of expiring, or if there's been a change to Let's Encrypt policy or process. It's worth being on the list.

You can request your certificate with the following:

sudo certbot certonly --webroot -w /var/www/letsencrypt -d example.org -d www.example.org

If it works, it gratifyingly results in a message that starts with "Congratulations"!

Example Nginx Domain Configuration - unencrypted

Once you've got your certificate, you can reference it in your configuration. We normally set up a redirect from the unencrypted version of the site to the encrypted on (except for the Let's Encrypt verification directory!):

server {     listen 80; # listen on port 80 on all IPv4 addresses
    listen [::]:80; # listen on port 80 on all IPv6 addresses
    root /var/www/html;     index index.html index.htm;     server_name example.org www.example.org;     access_log /var/log/nginx/example.org_access.log;     error_log /var/log/nginx/example.org_error.log;

include /etc/nginx/includes/letsencrypt.conf;

    # a 302 is a "soft" redirect. A 301 can never be reversed.     location / {         return 302 https://example.org$request_uri;     }      } server {
    listen 443 ssl; # listen on port 443 on all IPv4 addresses
    listen [::]:443 ssl; # listen on port 443 on all IPv6 addresses

    # this is a deprecated rule, not required in Ubuntu 20.04's nginx     # ssl on;     ssl_certificate /etc/letsencrypt/live/example.org/fullchain.pem;     ssl_certificate_key /etc/letsencrypt/live/example.org/privkey.pem;
    ssl_protocols TLSv1 TLSv1.1 TLSv1.2; ssl_dhparam /etc/ssl/certs/dhparam.pem;     keepalive_timeout 20s;

access_log /var/log/nginx/example.org_access.log; error_log /var/log/nginx/example.org_error.log;

root /var/www/html; index index.html index.htm;

server_name example.org www.example.org; # this is just an example of a "proxy" configuration # for, say, a Docker-based service, exposed on the VM's # local port 8081 location / { proxy_read_timeout 300; proxy_connect_timeout 300; proxy_redirect off; proxy_set_header Host $http_host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; proxy_pass http://127.0.0.1:8081; }
}

Note - you also need to set up the ssl_dhparam file for this configuration to work. You can do this based on these instructions after installing OpenSSL tools:

sudo apt-get install openssl

by running (warning - this can take quite a long time, 5-15 minutes in my experience - the system needs to generate sufficient entropy to achieve acceptable randomness):

openssl dhparam -out /etc/ssl/certs/dhparam.pem 4096

When you've set up the file, you can enable it:

sudo ln -sf /etc/nginx/sites-available/example.com /etc/nginx/sites-enabled

Test the file to ensure there aren't any syntax errors before reloading nginx:

sudo nginx -t

If this shows an error, you'll need to fix the file. If all's well, reload nginx to enable the new configuration:

sudo service nginx reload

You should now be able to point your browser at your domain name, and it should automatically redirect you to https://example.org - and (based on the above configuration, https://www.example.org should work too. You might want to redirect www.example.org to example.org or visa versa).

A word to the wise - if it doesn't work, check your firewall settings!

Update: discovered this very well done how-to on Let's Encrypt that offers additional background to this one.

Ongoing Certificate Maintenance

One of the nice things about EFF's certbot is that when it's installed, it also installs a nightly cron task (see /etc/cron.d/certbot) which checks all domains registered on the server for renewals. Assuming your domains have been configured in Nginx as described above, renewals should occur automatically, and you'll just receive a periodic email to let you know that they've happened.

If you want to check your renewal status, you can run this:

sudo certbot renew --dry-run

Good on you for securing your users and your site!

Add new comment

Installing Rocket.Chat with Docker on Ubuntu Linux 14.04

dave — Wed, 23 Nov 2016 21:59:47 +0000

Installing Rocket.Chat with Docker on Ubuntu Linux 14.04

Blog tags

install

ubuntu linux

14.04

docker

rocket.chat

let's encrypt

mongodb

dave Thu 24/11/2016 - 10:59

Rocket.Chat is a modern, open source messaging application which is functionally similar to a popular (and heavily marketed) proprietary tool called Slack. Rocket.Chat is built on a powerful open source real-time collaboration platform called Meteor (which, in turn, is built on the Node.JS open source framework), which supports real-time collaborative applications. The real-time collaboration means that if multiple people are using an app at the same time, even if they're spread across the world, they'll see the changes others are making in real-time.

The OERu has a Rocket.Chat instance that has been getting very positive feedback from our Open Education Resource designers and collaborators, who use it to communicate with us at the OER Foundation, and with their fellow collaborators. There are currently about 20 channels to which users can subscribe and in which they can participate, dedicated to different topics of discussion.

Rocket.Chat instances store their data in another open source tool, a key-value storage engine called MongoDB - these instructions assume that you have already got a running MongoDB installed, and to facilitate that, we've provided this handy MongoDB install guide as a companion.

This guide will cover both configuring and launching a Docker container with a working instance of Rocket.Chat. It will, in turn, be linked to another Docker container running MongoDB, and both will be capable of sending email via external authenticating email server. External user access to Rocket.Chat is provided by the Nginx web server (as a forward proxy). User interaction with Rocket.Chat is (and should always be) encrypted via recognised SSL certificates using the brilliant (and gratis) Let's Encrypt service.

This instruction set assumes that you have command-line access (via SSH, in most cases) to your server, running Ubuntu Linux 14.04, probably a Virtual Machine hosted in a data centre somewhere - a very inexpensive way to do this is, for example, via DigitalOcean (for the record, we have no relationship with DigitalOcean, I simply have substantial experience with their services), but there are many many options worldwide. These instructions will need to be modified slightly for other versions of Linux (e.g. Debian 8 or Ubuntu 16.04 or CentOS or others), but should be mostly valid. We'd be grateful to hear about anyone else's experiences in the comments below!

Installing Docker

See our instructions in the MongoDB blog post.

Installing Rocket.Chat

First, we would normally create a rocketchat user to create a place for any persistent data required for the app:

sudo adduser rocketchat

Although Rocket.Chat stores almost everything in the MongoDB you've already set up, it can store uploaded files in a designated directory so they survive updates to the Docker image:

sudo -u rocketchat mkdir /home/rocketchat/uploads

Assuming you've got Docker installed properly, to install the official Rocket.Chat Docker image, you can just run:

docker pull rocketchat/rocket.chat

Then you can launch it by running this by first launching your MongoDB container, and then running this (assumes you've named your MongoDB instance "mongodb" as per our instructions):

docker run -d --name rocketchat -p 8051:3000 \ -h [your_RocketChat_domain] \ -v /home/rocketchat/uploads:/var/www/rocket.chat/uploads \ -e "MAIL_URL=[outgoing_mail_server]" \ --link mongodb:mongo -e "MONGO_URL=mongodb://mongo/chat" \ -e "ROOT_URL=http://[your_RocketChat_domain]" \ --restart unless-stopped rocketchat/rocket.chat

Note: Please copy and paste the exact text of your "docker run" command into a reference file (I usually have a "README.oeru" reference file in the home directory of each Docker-based app I run) as you will want to refer to it when doing upgrades!

You'll need to make sure you set appropriate values for [your_RocketChat_domain] and details for an SMTP (outgoing mail) server, because Rocket.Chat needs to send emails related to things like account registration, forgotten passwords, and configurable notifications, for example to alert you that you've been mentioned in discussions.

You can use either a local mail server on the Docker host, in which case you'd put the local IP address of your server, as it would be seen by the Docker container, so 172.17.42.1 is the default IP of a Docker host. You could also use an authenticating SMTP server, and specify the details like this: smtp://[username]:[password]@[IP-or-domainname]:[port, usually 25, 465, or 587]. Here's a what a made-up example might look like:

-e "MAIL_URL=smtp://smtpmail:blahdiblah88@mail.oeru.org:25"

Note, the --restart unless-stopped will ensure that this container is restarted on a reboot unless it's explicitly stopped, like via a docker stop rocketchat, like you might to update the Docker container.

Setting up Nginx as a proxy server

Having set up the Docker container, which is listening on port 8051 on the Docker container's host, you'll need to set up a reverse proxy on that host to make the site visible to the broader internet on your public IP address (you'll want to make sure the domain you specified above points to that IP address or is a CNAME to a domain that does).

Once you've got that, you can proceed. In /etc/nginx/sites-available, I create a file called plan (as we use plan.oeru.org as our domain). Note, I use "vim" as my text editor. If you don't know it, perhaps use "nano" instead. It's much less powerful, but easier to use...

sudo vim /etc/nginx/sites-available/chat

Here're the contents - you'll want to change the domain name to suit your own choices. Similarly the names of SSL files and logs. The following file has a chunk in the middle commented out with "#s". More on that below:

server {

        listen 80; # this is one of our external IPs on the server.         #listen   [::]:80 default ipv6only=on; ## listen for ipv6         root /usr/share/nginx/www;         index index.html index.htm;         server_name chat.oeru.org;         access_log /var/log/nginx/chat.oeru.org_access.log;         error_log /var/log/nginx/chat.oeru.org_error.log;         location /.well-known {                 root /var/www/html;                 default_type text/plain;         } #        location / { #                return 301 https://chat.oeru.org$request_uri; #        }      #} # #server { #        listen 443 ssl; #        ssl on; #        ssl_certificate /etc/letsencrypt/live/chat.oeru.org/fullchain.pem; #        ssl_certificate_key /etc/letsencrypt/live/chat.oeru.org/privkey.pem;
#        ssl_protocols TLSv1 TLSv1.1 TLSv1.2; #        ssl_dhparam /etc/ssl/certs/dhparam.pem; #        keepalive_timeout 20s; # #        access_log /var/log/nginx/chat.oeru.org_access.log; #        error_log /var/log/nginx/chat.oeru.org_error.log;
#        root /var/www/html; #        index index.html index.htm;
#        server_name chat.oeru.org; # #        location /.well-known { #                root /var/www/html; #                default_type text/plain; #        }

location / { proxy_read_timeout 300; proxy_connect_timeout 300; proxy_redirect off; proxy_set_header Host $http_host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; proxy_pass http://127.0.0.1:8081; }
}

When you've set up the file, you can enable it:

sudo ln -sf /etc/nginx/sites-available/chat /etc/nginx/sites-enabled

Test the file to ensure there aren't any syntax errors before reloading nginx:

sudo nginx -t

If this shows an error, you'll need to fix the file. If all's well, reload nginx to include the new configuration:

sudo service nginx reload

You should now be able to point your browser at your domain name, and you should get your Rocket.Chat site via the HTTP (not encrypted) protocol.

A word to the wise - if it doesn't work, check your firewall settings!

In the next step, we'll sort out your SSL certificate from Let's Encrypt.

Protecting your users

Have a look at our Let's Encrypt howto.

Upgrading Rocket.Chat

You should periodically upgrade Rocket.Chat, say every couple months, to benefit from improved features... The upgrade process usually means a few minutes of down time for the site, so pick a time when it isn't likely to be heavily used... You'll normally want to upgrade any dependent Docker containers at the same time (like your MongoDB and Wekan) so we have a dedicated Upgrade article for that.

Add new comment

Installing Wekan with Docker on Ubuntu Linux 14.04

dave — Thu, 27 Oct 2016 01:12:48 +0000

Installing Wekan with Docker on Ubuntu Linux 14.04

Blog tags

install

ubuntu linux

14.04

mongodb

wekan

docker

let's encrypt

dave Thu 27/10/2016 - 14:12

Wekan is an excellent, easy-to-use "kanban board" project management support tool, suitable for all manner of projects. For those who have used the highly marketed Trello kanban service, Wekan is functionally similar open source alternative that organisations can host and control for themselves. They can also enhance it in whatever ways they are moved to do so. We encourage our partner institutions to consider this path as a way of reducing costs as well as increasing freedom and privacy. To make migrating a win-win, we have also found that Wekan is able to import entire Trello boards, preserving your data. (Update: 2017-05-24 we've just published an easier way to run Wekan and MongoDB)

The OERu provides a Wekan instance that has proven very popular with our Open Education Resource designers and collaborators.

Wekan instances store their data in another open source tool, a key-value storage engine called MongoDB - these instructions assume that you have already got a running MongoDB installed, and to facilitate that, we've provided this handy MongoDB install guide as a companion.

This guide will cover both configuring and launching a Docker container with a working instance of Wekan. It will, in turn, be linked to another Docker container running MongoDB, and both will be capable of sending email via external authenticating email server. External user access to Wekan is provided by the Nginx web server (as a forward proxy). User interaction with Wekan is (and should always be) encrypted via recognised SSL certificates using the brilliant (and gratis) Let's Encrypt service.

Installing Docker

See our instructions in the MongoDB blog post.

Installing Wekan

First, we would normally create a wekan user:

sudo adduser wekan

and then create a directory in that user's space to store Wekan-related data (so it survives updates to the Docker image):

sudo -u wekan mkdir /home/wekan/public

Assuming you've got Docker installed properly, to install the official Wekan Docker image, you can just run:

docker pull mquandalle/wekan

Then you can launch it by running this by first launching your MongoDB container, and then running this (assumes you've named your MongoDB instance "mongodb" as per our instructions):

docker run -d --name wekan -p 127.0.0.1:5555:80 \ -h [your_Wekan_domain] -e "VIRTUAL_HOST=[your_Wekan_domain]" \ -v /home/wekan/public:/built_app/programs/web.browser/app \ -e "MAIL_URL=[outgoing_mail_server]" \ --link mongodb:mongo -e "MONGO_URL=mongodb://mongo/plan" \ -e "ROOT_URL=http://[your_Wekan_domain]" \ --restart unless-stopped mquandalle/wekan

You'll need to make sure you set appropriate values for [your_Wekan_domain] and details for an SMTP (outgoing mail) server, because Wekan needs to send emails. You can use either a local mail server on the Docker host, in which case you'd put the local IP address of your server, as it would be seen by the Docker container, so 172.17.42.1 is the default IP of a Docker host. You could also use an authenticating SMTP server, and specify the details like this: smtp://[username]:[password]@[IP-or-domainname]:[port, usually 25, 465, or 587]. Here's a what a made-up example might look like:

-e "MAIL_URL=smtp://smtpmail:blahdiblah88@mail.oeru.org:25"

Note, the --restart unless-stopped will ensure that this container is restarted on a reboot unless it's explicitly stopped, like via a docker stop wekan, like you might to update the Docker container.

Setting up Nginx as a proxy server

Having set up the Docker container, which is listening on port 5555 on the Docker container's host, you'll need to set up a reverse proxy on that host to make the site visible to the broader internet on your public IP address (you'll want to make sure the domain you specified above points to that IP address or is a CNAME to a domain that does).

sudo vim /etc/nginx/sites-available/plan

# from https://github.com/wekan/wekan/wiki/Install-Wekan-Docker-in-production upstream websocket { server 127.0.0.1:5555; }

map $http_upgrade $connection_upgrade { default upgrade; '' close; }

server {         listen 80; # this is one of our external IPs on the server.         #listen   [::]:80 default ipv6only=on; ## listen for ipv6         root /usr/share/nginx/www;         index index.html index.htm;         server_name plan.oeru.org;         access_log /var/log/nginx/plan.oeru.org_access.log;         error_log /var/log/nginx/plan.oeru.org_error.log;         location /.well-known {                 root /var/www/html;                 default_type text/plain;         } #        location / { #                return 301 https://plan.oeru.org$request_uri; #        }      #} # #server { #        listen 443 ssl; #        ssl on; #        ssl_certificate /etc/letsencrypt/live/plan.oeru.org/fullchain.pem; #        ssl_certificate_key /etc/letsencrypt/live/plan.oeru.org/privkey.pem;
#        ssl_protocols TLSv1 TLSv1.1 TLSv1.2; #        ssl_dhparam /etc/ssl/certs/dhparam.pem; #        keepalive_timeout 20s; # #        access_log /var/log/nginx/plan.oeru.org_access.log; #        error_log /var/log/nginx/plan.oeru.org_error.log;
#        root /var/www/html; #        index index.html index.htm;
#        server_name plan.oeru.org; # #        location /.well-known { #                root /var/www/html; #                default_type text/plain; #        }

location / { proxy_read_timeout 300; proxy_connect_timeout 300; proxy_redirect off; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; proxy_pass http://127.0.0.1:5555; proxy_set_header Host $host; }

location ~ websocket$ { proxy_pass http://websocket; proxy_http_version 1.1; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection $connection_upgrade; } }

When you've set up the file, you can enable it:

sudo ln -sf /etc/nginx/sites-available/plan /etc/nginx/sites-enabled

Test the file to ensure there aren't any syntax errors before reloading nginx:

sudo nginx -t

If this shows an error, you'll need to fix the file. If all's well, reload nginx to include the new configuration:

sudo service nginx reload

You should now be able to point your browser at your domain name, and you should get your Wekan site via the HTTP (not encrypted) protocol.

A word to the wise - if it doesn't work, check your firewall settings!

In the next step, we'll sort out your SSL certificate from Let's Encrypt.

Protecting your users

Have a look at our Let's Encrypt howto.

Upgrading Wekan

You should periodically upgrade Wekan, say every couple months, to benefit from improved features... The upgrade process usually means a few minutes of down time for the site, so pick a time when it isn't likely to be heavily used... You'll normally want to upgrade any dependent Docker containers at the same time (like your MongoDB and Rocket.Chat) so we have a dedicated Upgrade article for that.

Add new comment

Installing MongoDB with Docker on Ubuntu Linux 14.04

dave — Thu, 27 Oct 2016 01:06:55 +0000

Installing MongoDB with Docker on Ubuntu Linux 14.04

Blog tags

install

ubuntu linux

14.04

docker

mongodb

dave Thu 27/10/2016 - 14:06

This post describes installing MongoDB (and backing it up and restoring from backup) in a Docker container on an Ubuntu Linux 14.04 virtual machine. (Update: 2017-05-24 we've just published an easier way to run MongoDB and Rocket.Chat and Wekan which depend on it)

The motivation for installing MongoDB is that it is the key-value based data container engine used by two compelling web technologies, Rocket.Chat and Wekan, that the OER Foundation has deployed for the OERu as https://chat.oeru.org and https://plan.oeru.org.

Installing Docker

Rather than trying to write a comprehensive guide to installing Docker, I'll instead recommend that you have a look at this one provided by the Docker community. It also covers installing on other versions of Ubuntu which might be helpful for some. Once you've got a working Docker container install going on your server, you can continue to the next step...

Note: the idea with Docker is that nothing important is stored in the container itself. That means that you can blow it away at any time without losing any important data. You can do that to quickly upgrade it, or to change its properties. You simply re-launch the container making sure it points properly to the persistent stuff you want it to use.

Installing MongoDB

Once you've got Docker working on your virtual machine, getting the MongoDB docker container (where it's preinstalled and ready to go) is simple:

docker pull mongo

This will install the latest version of the MongoDB container from docker.io.

Launching MongoDB

I recommend setting up a per-container user to help keep things well segmented. You can create a "mongo" user:

sudo adduser mongo

This creates a directory /home/mongo and that's where we'll get the Mongo container to store the data we want to keep persistent.

We can create "data" and "backups" directories in which it should store its data and save its backups respectively:

sudo mkdir /home/mongo/data sudo mkdir /home/mongo/backups

Then you can launch the container:

docker run --name mongodb --restart unless-stopped \ -v /home/mongo/data:/data/db -v /home/mongo/backups:/backups -d mongo --smallfiles

You should be able to check it's running by typing

docker ps

and looking for the reference to "mongodb" (that's the "name" we've assigned to this instance of the "mongo" container).

You can even log into the container and poke around the filesystem if you like, using the container's "id":

ID=`docker ps | grep mongo | cut --characters=1-12` docker exec -it $ID bash

which will drop you at the command line in the container. You can exit with "exit" or CTRL-D.

If things go wrong, you can run this:

ID=`docker ps | grep mongo | cut --characters=1-12` docker inspect $ID | grep log

Which will give you a result like this (your filenames will be different):

"LogPath": "/var/lib/docker/containers/539072b8fb976b5048bd13e90074ea4b43166b81c3d69d1720b4746785f7d917/539072b8fb976b5048bd13e90074ea4b43166b81c3d69d1720b4746785f7d917-json.log",

In which case, you can look at the log file to see what's gone wrong:

less +G /var/lib/docker/containers/539072b8fb976b5048bd13e90074ea4b43166b81c3d69d1720b4746785f7d917/539072b8fb976b5048bd13e90074ea4b43166b81c3d69d1720b4746785f7d917-json.log

You can exit less by hitting "q" or CTRL-C.

Backing up MongoDB

Backing up MongoDB requires a somewhat circuitous process. Here's what I do (after quite a bit of head scratching and web searching - Note: these backups are not directly related to the "backups" directory created above. More on that in the script below.):

On the host VM, I create a directory for backups - I use /home/backup

sudo mkdir /home/backup

and I create a bin directory in /home/mongo

sudo mkdir /home/mongo/bin

and I create a script file with my preferred editor (I use vim, you could use nano instead if vim scares you):

sudo vim /home/mongo/bin/mongo_backup.sh

In it, I put the following:

#!/bin/bash
#
# backup all the mongo DBs hosted on this server, in a docker container
#
# useful reference: https://docs.mongodb.org/manual/core/backups/
#
# in event of a disaster, this could be useful:
# https://docs.mongodb.org/manual/tutorial/recover-data-following-unexpec…
#
# script copyright dave@davelane.nz - available under terms of GPL v2 or later,
# see https://www.gnu.org/licenses/old-licenses/gpl-2.0.html
#
# Note: I've commented out some "echo" statements below that might be useful if you're
# debugging this script...
DATE=`date '+%Y%m%d-%H%M%S'`
# this is the local backup directory - gets cleared out after every backup...
DIR=/home/mongo/backups
# the archive dir, with a past archive as well.
ARCHDIR=/home/backup/mongodb
PREV=$ARCHDIR/previous
LATEST=$ARCHDIR/latest
#
# this is the backup directory on the Docker host
DDIR=/backups
# 1. get mongo container (this assumes only one running on the machine, with "mongo" in is name!)
ID=`docker ps | grep "mongo" | cut -c 1-12`
##echo "working with ID = $ID"
#
# 2. back up databases
# 2.1 backup to the local $DDIR on the Docker host
MSG=`docker exec -i $ID mongodump --quiet --out $DDIR`
##echo $MSG
# go into local backup dir
CWD=`pwd`
cd $DIR
# 2.2. get list of backup databases
DBS=`find $DIR -mindepth 1 -maxdepth 1 -type d -exec basename {} \;`
for DB in $DBS
do
    ARCH="${DB}.tgz"
    if [ -f $PREV/$ARCH ] ; then
        ##echo "explicitly jettison old backup of $PREV/$ARCH"
        rm $PREV/$ARCH
    else
        echo "***no previous version of $ARCH in $PREV on $DATE"
    fi
    if [ -f $LATEST/$ARCH ] ; then
        ##echo "move previous $LATEST/$ARCH into $PREV (overwritting old)"
   mv $LATEST/$ARCH $PREV/$ARCH
    else
        echo "***no latest backup $ARCH in $LATEST on $DATE"
    fi
    ##echo "archiving contents of $DB into $LATEST/$ARCH"
    tar cvfz $LATEST/$ARCH $DB
    #echo "removing $DB *.json and *.bson and the dir"
    rm -f $DB/*.bson $DB/*.json
    rmdir $DB
done
# return to where we started
cd $CWD
# exit with a happy 0, which shows we were successful!
exit 0

Save that file, and then run this to fix the ownership of the file and make the script executable:

sudo chown -R mongo:mongo /home/mongo/* sudo chmod a+x /home/mongo/bin/mongo_backup.sh

You can try running the script to make sure it creates a backup of your mongo database (such as it is! There might not be much there yet, if you haven't got any apps installed that use MongoDB - you can wait until after that before testing. If you run it twice, you should see both a "latest" and "previous" backup directory containing suitably dated directories - this provides extra data security):

sudo /home/mongo/bin/mongo_backup.sh

Once you're happy that's working, set up a cron job to make it run daily... Again, edit a file with your preferred text editor:

sudo vim /etc/cron.d/mongodb-backup

and fill it with the following (replacing [an email that will get to you!] appropriately):

#
# cron.d/mongodb-backup -- backups up the Mongo DBs running in a docker container
#
#
MAILTO=[and email that will get to you!]
# run daily at 12:17pm as the root user
17 12 * * * root /home/mongo/bin/mongo_backup.sh

Recovering a MongoDB database

In the event that you ever have to recover or transfer a MongoDB database, aka container, you can do so like this (sort of the reverse of the backup script):

work out the name of the container you want to restore and copy the backup of the database directory (e.g. rocketchat or wekan) into /home/mongo/backups - to be clear: for MongoDB, a "database" is represented as a directory of files holding all the relevant data.

Then log into Docker instance:

ID=`docker ps | grep mongo | cut --characters=1-12` docker exec -it $ID bash

and on the Docker container's command line run this, replacing the [names] appropriately below. Note, you might want to change the database name from the backup's original to a new name, so they don't need to be the same (but they can be):

mongorestore --db [recovered DB Name] --drop /backups/[backed up DB Name]/

Upgrading MongoDB

Note, if you have other apps in other containers that depend on MongoDB, it's usually most convenient to upgrade them all at the same time. Look at our instructions for, for example, running Wekan, to see how to upgrade MongoDB.

Blog comments

Really helpful post - thx I…

Keith W (not verified) Thu 15/07/2021 - 08:48

Really helpful post - thx

I had to remove the --smallfiles from the command to start the docker/mongo container