I love backing data up. I'm at the low end of computer hardware, I know that my gear is old and not the best. This means I need to have numerous backups for when the harddrives die. My numerous computers all mirror, via syncthing , to the main RPI server. This does a good job of dealing with conflicts with multiple shared folders and can be run as service on servers and in a gui e.g., on the Pinebookpro .

However, this dynamic mirror means that I can't go back in time to find a deleted file if there is a problem/file corruption or just removed the wrong file! I've been using rdiff-backup for photos/Documents as it keeps nice records of daily, weekly, monthly changes. With this, i can find a modification made to file back in time upto a year old. I reason that after a year, I won't remember where the file is, let alone the change! It also can't be that important. This system works really well except for recording the changes. After one week, the file changes in the mirror were 60GB! I keep a mirror of our important documents on aws so if the house burnt down, we wouldn't lose our Photos/Documents. This upload isn't really feasible to do over 4G as it would clog up our internet! Reducing the upload rate would just mean that the upload would never finish before the content is changed Digging deeper, rdiff-backup hasn't been updated for nearly 11 years and works as is!

I needed a more modern system

Enter Borg . It analyses files into chunk and then keeps a database of these chunks. Only if a chunk changes does it save the original and then update the db. This means that files can be reorganised, moved or have a timestamp change but borg will only update the db not the stored data. The data is also stored with compression which resulted in a 25% reduction in storage but more imporantly a reduction in required uploads and upload time. It's written in python/Cython too! The only slight downside is the the borg repo is not directly readable but it can be mounted as a filesystem