Accessing "diff only" ZFS snapshot

Question

Is there a way to mount a virtual partition containing only the files specific to a snapshot? I know about the hidden zfs directory but it contains all files at the snapshot time. My goal is to make diff backup faster...

Thanks in advance

greg

Vaccaro

Answer 1

Although Andrew's suggestion of zfs send is the right way to work with differential snapshots, if you just want to see the differences and work with them in your own scripts or on other platforms without ZFS support, there also is zfs diff:

zfs diff [-FHt] snapshot snapshot|filesystem

Display the difference between a snapshot of a given filesystem
and another snapshot of that filesystem from a later time or
the current contents of the filesystem.  The first column is a
character indicating the type of change, the other columns
indicate pathname, new pathname (in case of rename), change in
link count, and optionally file type and/or change time.

The types of change are:
  -       The path has been removed
  +       The path has been created
  M       The path has been modified
  R       The path has been renamed

-F
    Display an indication of the type of file, in a manner
    similar to the -F option of ls(1).
      B       Block device
      C       Character device
      /       Directory
      >       Door
      |       Named pipe
      @       Symbolic link
      P       Event port
      =       Socket
      F       Regular file
-H
    Give more parsable tab-separated output, without header
    lines and without arrows.
-t
    Display the path's inode change time as the first column of
    output.

Note that the order of the two datasets must be chronological. You could parse the resulting list and only work with those filenames you are interested in.

Example output from the man page:

# zfs diff -F tank/test@before tank/test
M       /       /tank/test/
M       F       /tank/test/linked      (+1)
R       F       /tank/test/oldname -> /tank/test/newname
-       F       /tank/test/deleted
+       F       /tank/test/created
M       F       /tank/test/modified

Also, if you use Oracle Solaris 11.3, you also have the -r switch to recursively diff all children datasets.

Toothache

Hum interesting... then I could feed that to rsync, couln't I? – Vaccaro

Yes, I would suggest you use the -H option and then awk or grep. Example: zfs diff fs@oldsnap fs@newsnap | awk '/^(R|M|\+).*/{print $2}' gives you all changed files (except deleted ones). – Toothache

Answer 2

There is no way to access differential data directly via "normal" file access, and there's no way to apply the data obtained from one even if you could get it. How could you read just the differences from a file if only a block or two changed? And if you could read just the differences, how would you know how to apply just the changed data to the file that changed? If you're trying to speed up differential backups, that's a "patch" style of update that is likely to be extremely slow.

Simple, "normal" file access does not provide the information needed to do a differences-only backup.

To do a differential backup of ZFS, use an incremental zfs send ... command:

zfs send -i pool@snap1 pool@snap2 ...

That's what it's meant for, and there's really no way to do it faster, since the ZFS filesystem is designed from the ground up to know the differences.

Bencher

Thanks for your answer. My idea was that zfs actually knows which files have changed, whereas scanning for changes can take several hours (even days). Does the zfs send command creates a target with only the modified files? – Vaccaro

Maybe I forgot to mention that my target backup host is not a zfs system, I use rclone to a regular filesystem. – Vaccaro

ZFS does in fact "know" what the differences are, so the incremental zfs send ... command will send only the difference between the two snapshots. The problem you're going to have is the only way to access the data in it is to use it as the input of a zfs recv ... command. You can store the output from zfs send ... as a regular file and use it to restore if you need to, but you won't be able to access the data without doing that. – Bencher