3-2-1 Backups

3-2-1 System

In our household we use OneDrive as our cloud file service. In fact OneDrive has the only complete copy of our files. There is not enough space on our day to day laptop. Clearly having a complete copy of all your files in only place is risky. That place being a cloud provider is even more so. It's not exactly the 3-2-1 system:

  • Three copies of your data: Your three copies include your original data (also called production data), plus two more copies.
  • On two different media: You should store your data on two different storage media, such as a local drive and a cloud storage service.
  • One copy off-site: You should keep one copy of your data off-site in a remote location, ideally more than a few miles away from your other two copies to protect against natural and physical disasters that could affect local copies.

My interpretation

We only have one complete copy of the files, on OneDrive. I decided to investigate how I could setup a 3-2-1 backup system, ideally using what I already have, rather than investing in a Synology NAS for example. I do have an old Raspberry Pi 3 B+ running with LibreELEC installed on it and I recently bought a new router to which I have a 1 TB hard disk drive attached, as well as an old laptop with a 1 TB disk that is not used, on which I installed Debian a while ago as Windows was borked.

In my thinking I could use OneDrive as the original full copy, as well as off site copy. I could use the 1TB disk attached to the router as a second copy and the 1TB hard disk as the third copy. This would satisfy the first condition (3 copies), the second condition (two different media - HDD and cloud) and the third condition (one off-site copy).

To manage the backups I needed a way to make a full copy of the OneDrive files and keep it synced, on two different disks. With some Googling and Rediting I came across the awesome rclone tool. Using Google Gemini and configured the laptop to pull a complete copy of our family OneDrive accounts (2 of them) on to the laptop's hard disk. This took over 24 hours to download each of the accounts which were ~220GB in size as OneDrive throttles the download speed. But the system ran very well.

Configuration

The rclone config looks like this:

[onedrive]
type = onedrive
token = {"access_token":"HIDDEN","token_type":"Bearer","refresh_token":"M.C552_SN1.0.U.MsaArtifacts.-HIDDEN","expiry":"2026-07-18T09:12:31.619611285+02:00","expires_in":3599}
drive_id = HIDDEN
drive_type = personal

The above config is repeated for each of the OneDrive accounts.

Here is the script I used to run rclone:

rclone sync "$REMOTE_ONEDRIVE_SOURCE" "$LOCAL_NAS_DEST" \
  --backup-dir "$BACKUP_ARCHIVE" \
  --modify-window 1s \
  --exclude='/Personal Vault/**' \
  --low-level-retries 10 \
  --log-file="$LOG_FILE" \
  --log-level INFO \
  --stats 1m

The above config was then repeated on the LibreELEX Pi, which took a little further investigation as LibreELEC is a locked down OS where the only space you can add files is the /storage folder. But I got it to work, writing to the USB drive networked through the router. The rclone configuration is similar to the above.

In both cases I now have cronjobs running to one-way sync the files to the two disks.

Costs

Initially I was hoping to not have to buy extra kit, but I did need another USB drive, as the one I had was used for my media server and at 1TB risked being a little small. I bought a 4TB USB hard disk from Toshiba (~€160). Unfortunately my router only had one USB slot so I needed to buy a powered USB hub (€23) as well.

Conclusions

It took me quite a while to get this setup. Most of it was waiting for the files to sync across to the hard disks. But I also had to repartition the hard disk on the laptop to be native Linux format (from NTFS to ext4). On a 1 TB laptop that takes time. A lot of it was also trial and error trying to get the disks attached properly (and permanently) to the Pi and to ensure the rclone config was correct. But it's done now and both backups have been running just fine for the last couple of days. I will keep an eye on this.

Tagged: