Faster backups

Backups come in three flavors:

• Full – backup everything
• Differential – backup everything since the last Full backup
• Incremental – backup everything since the last backup (Full or Incremental)

I took the above from What Is Bacula?

Most systems use mtime to determine if a file has changed. Thus, any file that has been modified since the last backup will be picked up in the next incremental or differential backup.

Full backups are easy: get a list of files, and back them up.

Incremental and Differential backups are more complicated. You need to look at every file in the collection of files to be backed up. You check the mtime, and if it’s newer than the backup in question, you backup that file. This checking can be quite time consuming.

I have long thought that a filesystem should be able to give you a list of all files altered since $TIMESTAMP. Thus, in very close to nil time, you have a list of files for your backup.

‘zfs send -i’ has a way to extract blocks that have been created between two snapshots. From there, you’d need to go from block to filenames. From there, you have your list.

It can’t be that hard. :)