backup http://tech.oeru.org/ en Automatic versioned backups of Sqlite in a Docker Compose container http://tech.oeru.org/automatic-versioned-backups-sqlite-docker-compose-container <span class="field field--name-title field--type-string field--label-hidden">Automatic versioned backups of Sqlite in a Docker Compose container</span> <div class="field field-node--field-blog-tags field-name-field-blog-tags field-type-entity-reference field-label-above"> <h3 class="field__label">Blog tags</h3> <div class="field__items"> <div class="field__item field__item--sqlite"> <span class="field__item-wrapper"><a href="/taxonomy/term/72" hreflang="en">sqlite</a></span> </div> <div class="field__item field__item--bash"> <span class="field__item-wrapper"><a href="/taxonomy/term/58" hreflang="en">bash</a></span> </div> <div class="field__item field__item--backup"> <span class="field__item-wrapper"><a href="/taxonomy/term/57" hreflang="en">backup</a></span> </div> <div class="field__item field__item--ubuntu-linux"> <span class="field__item-wrapper"><a href="/taxonomy/term/12" hreflang="en">ubuntu linux</a></span> </div> </div> </div> <span class="field field--name-uid field--type-entity-reference field--label-hidden"><span lang="" about="/user/1" typeof="schema:Person" property="schema:name" datatype="">dave</span></span> <span class="field field--name-created field--type-created field--label-hidden">Tue 04/02/2020 - 12:17</span> <div class="clearfix text-formatted field field-node--body field-name-body field-type-text-with-summary field-label-hidden"> <div class="field__items"> <div class="field__item"><p>For relatively lightweight applications that are either in development, or single user, or have limited requirements for concurrency and massive data sets, <a href="https://en.wikipedia.org/wiki/SQLite">SQLite</a> is a superb, full-function, but compact, <em>almost ubiquitous</em> database (it's used on every mobile device, for example).</p> <p>What's more, it's also Free and Open Source Software (FOSS) - its code has been dedicated to the public domain - making it an ideal tool for a principled open organisation like the OER Foundation to incorporate in its fully Free and Open Source Software stack.</p> <p>We use SQLite for quite a few things here at the OER Foundation. For example, this implementation of the backup script was developed for the very impressive fully FOSS version of the <a href="/setting-your-own-bitwarden-password-keeper-and-sync-server">password manager BitWarden </a>we use to manage our secrets at the OER Foundation. Because the data it holds, although very compact, is very precious, we want to back it up right, and ensure we can go back in time and recreate the data from the past (like, if we realised at some point that in the past we'd deleted the wrong secrets and needed to recover them! It hasn't happened yet, but it's inevitable it <em>will happen</em> at some point). That's why we've created this backup solution that creates dated dumps of an SQLite database that it keeps to ensure that relevant past versions of the data can be recovered, but without storing more data than necessary (a full hard disk is never fun).</p> <p>Those among you who know about SQLite might rightly point out that an SQLite database (unlike more powerful databases) is simply a single file that can be copied to make a backup. In this case, we're using SQLite's ".backup" command to create the database backup, which we see as a mechanism for future-proofing - at some future time, it might be that the option to copy the file without corrupting the database may no longer be available, but we'd expect that the .backup command would continue to work.</p> <p>This backup process involves 3 files: the backup BASH script itself, which has all the smarts, the .conf file which defines the file paths and other details specific to your installation - like where the SQLite database actually is, and where to put the backup files - and a cron file which runs the script automatically when you tell it to run on your server.</p> <p>We've got the <a href="https://git.oeru.org/dave/sqlite-backup">whole thing</a> in our <a href="https://git.oeru.org">Gitlab instance</a>, which is the best place to learn more about how to use it - see the <a href="https://git.oeru.org/dave/sqlite-backup/blob/master/README.md">README.md</a> for installation instructions!</p> </div> </div> </div> <section class="field field-node--field-blog-comments field-name-field-blog-comments field-type-comment field-label-above comment-wrapper"> <a name="comments"></a> <div class="comment-form-wrapper"> <h2 class="comment-form__title">Add new comment</h2> <drupal-render-placeholder callback="comment.lazy_builders:renderForm" arguments="0=node&amp;1=30&amp;2=field_blog_comments&amp;3=comment" token="LUsqyib8Nl62guCFKZHLxak9ymOGimis4ahdLWzFAfE"></drupal-render-placeholder> </div> </section> Mon, 03 Feb 2020 23:17:20 +0000 dave 30 at http://tech.oeru.org Hourly versioned MongoDB backup http://tech.oeru.org/hourly-versioned-mongodb-backup <span class="field field--name-title field--type-string field--label-hidden">Hourly versioned MongoDB backup</span> <div class="field field-node--field-blog-tags field-name-field-blog-tags field-type-entity-reference field-label-above"> <h3 class="field__label">Blog tags</h3> <div class="field__items"> <div class="field__item field__item--mongodb"> <span class="field__item-wrapper"><a href="/taxonomy/term/14" hreflang="en">mongodb</a></span> </div> <div class="field__item field__item--backup"> <span class="field__item-wrapper"><a href="/taxonomy/term/57" hreflang="en">backup</a></span> </div> <div class="field__item field__item--docker"> <span class="field__item-wrapper"><a href="/taxonomy/term/16" hreflang="en">docker</a></span> </div> <div class="field__item field__item--docker-compose"> <span class="field__item-wrapper"><a href="/taxonomy/term/49" hreflang="en">docker-compose</a></span> </div> <div class="field__item field__item--bash"> <span class="field__item-wrapper"><a href="/taxonomy/term/58" hreflang="en">bash</a></span> </div> </div> </div> <span class="field field--name-uid field--type-entity-reference field--label-hidden"><span lang="" about="/user/1" typeof="schema:Person" property="schema:name" datatype="">dave</span></span> <span class="field field--name-created field--type-created field--label-hidden">Mon 29/04/2019 - 16:28</span> <div class="clearfix text-formatted field field-node--body field-name-body field-type-text-with-summary field-label-hidden"> <div class="field__items"> <div class="field__item"><p>Because the collaboration of an open community is its real history, I place a high value on backing up the Rocket.Chat servers I'm responsible for, and <em>especially</em> the data they generate, held in MongoDB files (on the host) managed by a MongoDB container.</p> <p>To do that reliably, I have set up <a href="https://git.oeru.org/oeru/mongobackup">a bash script</a> which does an hourly backup of all MongoDB "databases" and automatically maintains 24 hourly, 7 daily, 4 weekly, 12 monthly, and 7 yearly snapshots of the databases.</p> <p>Once you've configured your backup script properly, you should be able to run this command to do a backup...</p> <p><code>/etc/mongobackup/dbbackup-mongo --hourly</code></p> <p>easy-peasy.</p> </div> </div> </div> <section class="field field-node--field-blog-comments field-name-field-blog-comments field-type-comment field-label-above comment-wrapper"> <a name="comments"></a> <div class="comment-form-wrapper"> <h2 class="comment-form__title">Add new comment</h2> <drupal-render-placeholder callback="comment.lazy_builders:renderForm" arguments="0=node&amp;1=27&amp;2=field_blog_comments&amp;3=comment" token="a3hwmZDJbiAJ569rEIXJYUfTF3BaDQbUaOh1gRHfc5g"></drupal-render-placeholder> </div> </section> Mon, 29 Apr 2019 04:28:43 +0000 dave 27 at http://tech.oeru.org