You are not logged in.

#1 2020-01-15 14:45:12

clusterF
Member
Registered: 2019-05-07
Posts: 539

Backing Up With Borg

Thought there should be a dedicated thread on how to use borg for a full system backups and other methods of creating backups with borg...

https://borgbackup.readthedocs.io/en/stable/#
https://packages.debian.org/buster/borgbackup


I was using rsync for a long while but decided to get educated on borg, it was well worth it today as i have managed to back up 200GB of files into a 140GB borg repo that i can search with borgfs, but more on that later.

1. make sure you have a large amount of space somewhere, i use backups to an external drive via usb 3.0 cable, there are ways to backup over a network but i dont know how yet.

#### For a Full System Backup ####

This is how i do it below, i post this up for informational purposes only, it pays to RTFM https://borgbackup.readthedocs.io/en/stable/


Note that this backs up everything on /
Note that BACKUPNAME means you have to create that directory (naming scheme of your choice, eg; "bunsenlabs_backups") in the first instance of mkdir -p

#### Borg Backup ####

## Create an encrypted repository

# creates directory for backed up files

mkdir -p /mnt/BACKUPNAME/borg/repos/$HOSTNAME/

# changes to this directory

cd /mnt/BACKUPNAME/borg/repos/$HOSTNAME/  

 
# initializes a borg repository     

borg --verbose init --encryption=repokey

# verify the creation was executed successfully                   
cat README



## Backup of the whole system:

sudo borg --progress --verbose create --comment "Use as root. full system backup." -e /dev -e /run -e /tmp -e /sys -e /proc -e /mnt -e /media -s /mnt/BACKUPNAME/borg/repos/$HOSTNAME/::fullsystembackup /

# This creates a backup of the whole system, excluding locations
# that are re-generated at every boot up, as well as, the `/mnt` location
# as we don't want to back up our backups, especially when you
# have a backup drive containing several hundred GB of data.


## Bashrc function - usage = borg_full "This is my comment."

borg_full(){
borg_comment=$1;
repo=/BACKUPNAME/borg/repos/laptop::fullsystembackup;
sudo borg --progress --verbose \
create --comment "Use as root. $borg_comment" \
-e /dev -e /run -e /tmp -e /sys -e /proc -e /mnt -e /media -e /var/log -s \
/mnt/BACKUPNAME/$repo::\{hostname\}+\{user\}:\{now:%Y-%m-%dT%H:%M\} /;
}


## usefull borg commands

sudo borg list /mnt/BACKUPNAME/borg/repos/$HOSTNAME/
sudo borg info /mnt/BACKUPNAME/borg/repos/$HOSTNAME/
sudo borg delete /mnt/BACKUPNAME/borg/repos/$HOSTNAME/::fullsystembackup

Much of this technique relies on the backup destination being mounted to /mnt so make sure this is so.

Please let me know if ive missed anything or this tutorial should/could be improved.

Last edited by clusterF (2020-01-15 15:05:47)

Offline

#2 2020-01-15 16:53:39

iMBeCil
WAAAT?
From: Edrychwch o'ch cwmpas
Registered: 2015-09-29
Posts: 767

Re: Backing Up With Borg

^Interesting, thanks for sharing.

This 'borg' actually looks like a mix of rsync and git: it will backup ('copy') data as good as rsync, while simultaneously keep old files (as git). And all this with reasonable syntax.

Definitely worth a try, especially looks convenient for incremental(!) backup of /home/$HOME directory ...


Postpone all your duties; if you die, you won't have to do them ..

Offline

#3 2020-01-16 08:18:34

ohnonot
...again
Registered: 2015-09-29
Posts: 4,510
Website

Re: Backing Up With Borg

I started using borg when I was looking for a backup solution that does deduplication - at that time I wasn't aware that rsync can do that too, just calls it differently.
But I am satisfied and see no reason to switch now. I guess borg has a higher convenience factor. I do weekly backups (almost full system) that add almost nothing to my backup disk's usage, and have restored one or two files succesfully (I don't use a trash bin). Never done a full restore though.
https://dt.iki.fi/borgbackup


BL quote proposals to this thread please.
how to ask smart questions | my repos / my repos | my blog
---
Thank you for posting direct image links!

Offline

#4 2020-01-16 11:31:41

clusterF
Member
Registered: 2019-05-07
Posts: 539

Re: Backing Up With Borg

@ imbecil, thanks im quite happy with it, been playing around with it for a few days now.

@ ohonot, was not aware that rsync can do deduplication, ill have to investigate, usually all i do is mount my encrypted usb drive and rsync as per arch wiki https://wiki.archlinux.org/index.php/Rs … tem_backup

Figured out mounting the full system backup using borg mount so i can search through the backup and copy files if need be.Borgfs is a bit complex as you can have multiple borgfs option like exclusion and archive filters , sort of like a reverse backup but just mounting using fuse, but if you just want to mount the whole backup i did the following. I did need python-llfuse for this.

create a mountpoint mine was <borg-mount>, has to be read write.

# borg mount /mnt/BACKUPNAME/borg/repos/$HOSTNAME/::fullsystembackup::laptop+root:2020-01-16T20:49 <borg-mount>

Last edited by clusterF (2020-01-16 11:34:23)

Offline

#5 2020-01-16 13:42:31

iMBeCil
WAAAT?
From: Edrychwch o'ch cwmpas
Registered: 2015-09-29
Posts: 767

Re: Backing Up With Borg

Thanks clusterF and ohnononono.

clusterF wrote:

@ ohonot, was not aware that rsync can do deduplication

Deduplication ... if it means NOT syncing identical files, then rsync does this ... i.e. it will not waste time to copy src-->dest for files which are identical. This really reduces backup time.

What I don't know: can rsync not only sync new files, but also keep previous version of those new files (a reminscence of git)? I think not, but I would like to know if there is such an option ...


Postpone all your duties; if you die, you won't have to do them ..

Offline

#6 2020-01-16 13:59:07

clusterF
Member
Registered: 2019-05-07
Posts: 539

Re: Backing Up With Borg

Not sure imbecil, i cant find any documentation that corresponds to what borg is capable of.

Ive made 5 incremental backups so far, i could automate this as long as my usb is connected and mounted but i like to manually do it.

sudo borg list /mnt/usb/backups/borg/repos/laptop/

firstbackup                          Wed, 2020-01-15 10:17:25 [xxx]
fullsystembackup::laptop+root:2020-01-15T11:57 Wed, 2020-01-15 11:57:13 [xxx]
fullsystembackup::laptop+root:2020-01-15T12:03 Wed, 2020-01-15 12:03:09 [xxx]
fullsystembackup::laptop+root:2020-01-16T01:20 Thu, 2020-01-16 01:20:49 [xxx]
fullsystembackup::laptop+root:2020-01-16T20:49 Thu, 2020-01-16 20:49:33 [xxx]

sudo borg info /mnt/usb/backups/borg/repos/laptop/

------------------------------------------------------------------------------
                       Original size      Compressed size    Deduplicated size
All archives:               37.46 GB             12.78 GB              3.85 GB

                       Unique chunks         Total chunks
Chunk index:                  161668               628120

Last edited by clusterF (2020-01-16 14:00:28)

Offline

#7 2020-01-16 16:02:27

iMBeCil
WAAAT?
From: Edrychwch o'ch cwmpas
Registered: 2015-09-29
Posts: 767

Re: Backing Up With Borg

clusterF wrote:

Not sure imbecil, i cant find any documentation that corresponds to what borg is capable of.

Oh ... it was actually question for ohnononono

I'm also not aware of rsync keeping old file version, and I have read documentation, too.


Postpone all your duties; if you die, you won't have to do them ..

Offline

#8 2020-01-16 20:34:42

ohnonot
...again
Registered: 2015-09-29
Posts: 4,510
Website

Re: Backing Up With Borg

iMBeCil wrote:

What I don't know: can rsync not only sync new files, but also keep previous version of those new files (a reminscence of git)? I think not, but I would like to know if there is such an option ...

Borg definitely does that.
rsync itself - I don't know, sorry.
Some of the apps based on rsync probably can - but then, borg is also based on (lib)rsync.


BL quote proposals to this thread please.
how to ask smart questions | my repos / my repos | my blog
---
Thank you for posting direct image links!

Offline

#9 2020-01-16 20:36:51

iMBeCil
WAAAT?
From: Edrychwch o'ch cwmpas
Registered: 2015-09-29
Posts: 767

Re: Backing Up With Borg

^Confirms what I thought. Thanks.


Postpone all your duties; if you die, you won't have to do them ..

Offline

#10 2020-01-17 05:53:46

johnraff
nullglob
From: Nagoya, Japan
Registered: 2015-09-09
Posts: 6,778
Website

Re: Backing Up With Borg

iMBeCil wrote:

What I don't know: can rsync not only sync new files, but also keep previous version of those new files (a reminscence of git)?

Would the --backup option not help a bit?

        -b, --backup            make backups (see --suffix & --backup-dir)
        --backup-dir=DIR        make backups into hierarchy based in DIR
        --suffix=SUFFIX         backup suffix (default ~ w/o --backup-dir)

Make SUFFIX date-dependent and every backed-up file would be different. Perhaps nowhere near as efficient as git, though.


...elevator in the Brain Hotel, broken down but just as well...
( a boring Japan blog (currently paused), idle Twitterings and GitStuff )

Introduction to the Bunsenlabs Lithium Desktop

Offline

#11 2020-01-17 08:56:56

iMBeCil
WAAAT?
From: Edrychwch o'ch cwmpas
Registered: 2015-09-29
Posts: 767

Re: Backing Up With Borg

^Indeed, after reading manual, this can be used for incremental backup, with old files still present. But it would need a bit scripting to automate this ... borg seems to do it kind of directly/cleanly, user doesn't have to think about the options too much.

Thanks for the 'RTFM' johnraff smile (I should have done it myself ...)


Postpone all your duties; if you die, you won't have to do them ..

Offline

#12 2020-01-17 13:38:03

twoion
ほやほや
Registered: 2015-08-10
Posts: 2,771

Re: Backing Up With Borg

ohnonot wrote:
iMBeCil wrote:

What I don't know: can rsync not only sync new files, but also keep previous version of those new files (a reminscence of git)? I think not, but I would like to know if there is such an option ...

Borg definitely does that.
rsync itself - I don't know, sorry.
Some of the apps based on rsync probably can - but then, borg is also based on (lib)rsync.

https://github.com/rdiff-backup/rdiff-backup : rdiff-backup uses the rsync algorithms for transfer but also has an incremental backup future - you can roll back increments and restore an old version of the file.

That being said, I stick to rsync for my core data backup; I somehow trust it more to do the right thing than borg or rdiff-backup.

I use rmlint  for deduplication of data in my file collections. It is smart enough to distinguish hardlinks (USEFUL when you want to have the really same file both in your 'taxes' and 'bank statements' folders, for organization) from actual duplicates (different inodes, same data, it is SIMPLE and PREDICTABLE and has REVIEW functionality built in, so I like using it more than 100% integrated tools that do everything.

I think it's important to keep your tools in important areas such as backups of personal data as simple as possible and the FORMATS easily reversable. rdiff-clone and borg use special metadata; rsync + rmlint just use the file system (in a simple way).


Per aspera ad astra.

Offline

#13 2020-01-18 04:13:42

johnraff
nullglob
From: Nagoya, Japan
Registered: 2015-09-09
Posts: 6,778
Website

Re: Backing Up With Borg

twoion wrote:

I think it's important to keep your tools in important areas such as backups of personal data as simple as possible

+1


...elevator in the Brain Hotel, broken down but just as well...
( a boring Japan blog (currently paused), idle Twitterings and GitStuff )

Introduction to the Bunsenlabs Lithium Desktop

Offline

#14 2020-01-18 13:27:56

iMBeCil
WAAAT?
From: Edrychwch o'ch cwmpas
Registered: 2015-09-29
Posts: 767

Re: Backing Up With Borg

^+1 fully agree.

Hadn't stumbled on rmlint before, thanks @twoion ... seems interesting, now I'm reading its documentation instead of doing my paid work ops


Postpone all your duties; if you die, you won't have to do them ..

Offline

#15 2020-01-18 16:39:40

clusterF
Member
Registered: 2019-05-07
Posts: 539

Re: Backing Up With Borg

twoion wrote:
ohnonot wrote:
iMBeCil wrote:

What I don't know: can rsync not only sync new files, but also keep previous version of those new files (a reminscence of git)? I think not, but I would like to know if there is such an option ...

Borg definitely does that.
rsync itself - I don't know, sorry.
Some of the apps based on rsync probably can - but then, borg is also based on (lib)rsync.

https://github.com/rdiff-backup/rdiff-backup : rdiff-backup uses the rsync algorithms for transfer but also has an incremental backup future - you can roll back increments and restore an old version of the file.

That being said, I stick to rsync for my core data backup; I somehow trust it more to do the right thing than borg or rdiff-backup.

I use rmlint  for deduplication of data in my file collections. It is smart enough to distinguish hardlinks (USEFUL when you want to have the really same file both in your 'taxes' and 'bank statements' folders, for organization) from actual duplicates (different inodes, same data, it is SIMPLE and PREDICTABLE and has REVIEW functionality built in, so I like using it more than 100% integrated tools that do everything.

I think it's important to keep your tools in important areas such as backups of personal data as simple as possible and the FORMATS easily reversable. rdiff-clone and borg use special metadata; rsync + rmlint just use the file system (in a simple way).

Could you perhaps share some part of your backup routine using rmlint and rsync ?

Offline

#16 2020-01-19 13:14:49

Zeno
Member
Registered: 2020-01-16
Posts: 27

Re: Backing Up With Borg

I only want to take snap shot of only Installed BL & all its settings and Configurations. Thats it. Like Timeshift does. Can I use Timeshift ???

Thanks

Offline

#17 2020-01-19 13:20:01

clusterF
Member
Registered: 2019-05-07
Posts: 539

Re: Backing Up With Borg

Zeno wrote:

I only want to take snap shot of only Installed BL & all its settings and Configurations. Thats it. Like Timeshift does. Can I use Timeshift ???

Thanks

https://github.com/teejee2008/timeshift

Timeshift is similar to applications like rsnapshot, BackInTime and TimeVault but with different goals. It is designed to protect only system files and settings. User files such as documents, pictures and music are excluded. This ensures that your files remains unchanged when you restore your system to an earlier date. If you need a tool to backup your documents and files please take a look at the excellent BackInTime application which is more configurable and provides options for saving user files.

Offline

#18 2020-01-19 13:28:17

Zeno
Member
Registered: 2020-01-16
Posts: 27

Re: Backing Up With Borg

Thanks. I just want System & Settings etc. So i would prefer Timeshift and for other Data backup i use Grsync.

Is my choice ok?? Thanks

Offline

#19 2020-01-19 13:36:23

clusterF
Member
Registered: 2019-05-07
Posts: 539

Re: Backing Up With Borg

Zeno wrote:

Thanks. I just want System & Settings etc. So i would prefer Timeshift and for other Data backup i use Grsync.

Is my choice ok?? Thanks

You do what works for you Zeno smile

Offline

#20 2020-01-19 13:38:32

Zeno
Member
Registered: 2020-01-16
Posts: 27

Re: Backing Up With Borg

Ok Thanks

Offline

#21 2020-01-20 07:29:35

ohnonot
...again
Registered: 2015-09-29
Posts: 4,510
Website

Re: Backing Up With Borg

Zeno wrote:

I only want to take snap shot of only Installed BL & all its settings and Configurations. Thats it. Like Timeshift does. Can I use Timeshift ???

That's "Only" the whole system, no? Your complete install? Not just some config files under your $HOME?
You have to be clear what you want.
https://github.com/teejee2008/timeshift
Read the Readme.


BL quote proposals to this thread please.
how to ask smart questions | my repos / my repos | my blog
---
Thank you for posting direct image links!

Offline

Board footer

Powered by FluxBB