StoreBackup 3.5.1 https://savannah.nongnu.org/projects/storebackup January 27, 2020 1 Super Quick Start StoreBackup is a very space efficient disk-to-disk backup suite for GNU/Linux and other unixoid systems. Additional details and help are provided in later sections of this document. In these brief quick start steps we make certain simplifying assumptions. If you are OK with that, then proceed as follows: 1. Download the source from https://download.savannah.nongnu.org/releases/storebackup/ 2. Unpack it (using tar -jxvf) into /opt (it will make the folder /opt/storeBackup.)1 3. Create symbolic links. In a terminal, run these 2 commands (the 2nd line ends with: space,dot): # cd /usr/local/bin # ln -s /opt/storeBackup/bin/* . 4. Run your first backup with this command (substituting your actual username in the command):2 storeBackup.pl --sourceDir /home/your_username --backupDir /tmp/my_master_backup This may take a while. Open a second shell and see what happens in the backup directory. You have now backed up your home directory to /tmp/my master backup. For more details, please continue reading; especially see installation, section 2 and storeBackup.pl, section 6.2. If the above steps gave you any challenges, don’t worry. This document will cover everything from storeBackup installation to NFS server settings in much more detail. See storeBackup’s Top Features on the next page 1You need root permissions to install storeBackup at /opt/storeBackup and to follow the next steps. You can also unpack and run storeBackup from a place where you do not have root permissions. If you start storeBackup without root permissions, it will run with the permissions you have at that moment. 2If you install storeBackup from the Debian or Ubuntu repository via the packet manager, all programs will come without the “.pl” at the end, so instead of storeBackup.pl you have to call storeBackup. 1 1.1 storeBackup’s Top Features • Restore easily – even without storeBackup! The most important aspect of a backup tool is easy restoring from a transparent (native) storage format. • Copies / compresses files to another disk and generates backups with time stamps while saving space by recognizing files with identical contents (even if renamed or copied) and hardlinking them (so each backup is totally complete, independent and autonomous) • Detects identical files in different, independent backups (eg. of different computers) • Splits big image files (from e.g., TrueCrypt, mbox, Xen, KVM, VMware, etc.) or complete devices into small pieces and saves only differences to existing backups, thereby saving space and time • Sophisticated possibilities for including and excluding files and directories. • Support of sparse files. • Supports isolated incremental backups (e.g., when traveling with a laptop) and later integration in master backup. • Supports time shifted replication of backups to additional other disks / locations, even for complex schemes. • Supports checking of backups via md5 sums to eg. recognize bit rot of the hard disk. • Enables checking of (old) files in the source directory with md5 sums in the backup to eg. recognize bit rot of the hard disk. • Fast backups even over slow or high latency network connections. • Contains an entire suite of backup-related tools. 1.2 Why should you back up your files? Simple answer – for two reasons. 1. To restore the last state after a hardware or software crash. 2. To recover old versions of a file or folder if it was deleted / destroyed unnoticed (e.g., by a software bug) or by mistake. The most important backup is the one you did not make! New releases are announced at http://freshmeat.net/projects/storebackup. Please subscribe to get recent information. If you have any hints, comments or questions, send an email to hjclaes at web.de StoreBackup is licensed under the terms of the GPL-v3 or any later version. Heinz-Josef Claes with support of contributors, January 27, 2020 2 Contents 1 Super Quick Start 1 1.1 storeBackup’s Top Features . 2 1.2 Why should you back up your files? . 2 2 Installation 4 3 Getting Started 5 4 What’s new? 6 4.1 Version 3.5.1 . 6 4.2 Version 3.5 . 7 4.3 Version 3.4.3 . 7 4.4 Version 3.4.2 . 8 4.5 Version 3.4.1 . 8 4.6 Version 3.4 . 8 4.7 Version 3.3.1 . 8 4.8 Version 3.3 . 9 5 The Idea Behind It 9 5.1 Abstract . 9 5.2 Another Backup Tool? – The roots of storeBackup . 10 5.3 What would be an ideal Backup Tool? . 10 5.4 How storeBackup works . 11 5.4.1 Illustration . 11 5.4.2 Reducing Disk Space . 12 5.4.3 Performance . 13 5.4.4 Example of a Run . 15 6 Components / Programs to use 17 6.1 Supported Platforms and Tips . 17 6.2 storeBackup.pl . 18 6.2.1 storeBackup.pl Options . 20 6.3 storeBackupUpdateBackup.pl . 31 6.4 storeBackupRecover.pl . 33 6.5 storeBackupVersions.pl . 35 6.6 storeBackupSearch.pl . 35 6.7 storeBackupSetupIsolatedMode.pl . 36 6.8 storeBackupMergeIsolatedBackup.pl . 37 6.9 storeBackupls.pl . 38 6.10 storeBackupDel.pl . 40 6.11 storeBackupMount.pl . 41 6.12 storeBackupCheckBackup.pl . 44 6.13 storeBackupCheckSource.pl . 45 6.14 storeBackup du.pl . 46 6.15 storeBackupConvertBackup.pl . 46 6.16 linkToDirs.pl . 47 6.17 llt . 48 6.18 multiTail.pl . 49 7 General concepts 50 7.1 Configuration file and command line . 50 7.2 Selecting Directories / Files to Backup . 52 7.3 Deletion of old Backups . 53 7.4 Defining rules . 54 7.4.1 How to define if a file should be compressed . 58 7.5 Saving Image Files / raw Devices / Blocked Files . 59 3 7.6 using option lateLinks . 63 7.7 Isolated Mode / Offline Backups . 65 7.7.1 You can use isolated backup in the following way: . 66 7.7.2 Setting up isolated mode . 67 7.7.3 Isolated Mode as Protection against Ransomware . 69 7.8 Replication of backups . 70 7.8.1 Quick start using storeBackup’s Replication Wizard . 70 7.8.2 Why copying backups is not a substitute for replication functionality . 72 7.8.3 Basic concepts to know before using storeBackup’s replication . 74 7.8.4 Understanding storeBackup’s Replication Wizard via an example . 76 7.8.5 A simple replication example without the Replication Wizard . 78 7.8.6 How storeBackup’s replication works . 80 7.8.7 Using Wildcards for Replication . 83 7.9 Special files generated and used by storeBackup . 84 7.10 Configuring NFS . 86 7.11 What is an inode . 86 7.12 Statistical Output of storeBackup.pl . 87 7.13 Monitoring . 89 7.14 Limitations . 89 8 Internals 89 8.1 Remark . 89 8.2 Completed Backups . 90 8.3 Late Links Backups . 91 8.4 Replication . 97 9 How to use storeBackup (Examples) 107 9.1 Some Information in the Beginning . 107 9.2 Example 1: Very simple backup . 109 9.3 Example 2: Backup of multiple directories . 109 9.4 Example 3: Make a big backup once a week, a small every day . 109 9.5 Example 4: Backup from different machines, share data . 110 9.6 Example 5: Different keepTimes for some directories . 111 9.7 Example 6: Using lateLinks . 111 10 FAQ, Frequently asked Questions 112 11 Contributors 117 12 Change Log 117 13 License 137 2 Installation You should read The Idea Behind It (Abstract), see section 5.1, as well as Supported Platforms and Tips, see section 6.1, to see if storeBackup fits to your needs. Installation is straightforward: Download the archive from http://download.savannah.gnu.org/releases/storebackup/ and go to a directory where you want to.