Solved ZFS Permissions Problem with zxfer (zfs receive)

C

ccammack

I've had really good luck so far running zxfer as root, backing up to a removable drive on the same machine, but now I'm trying it as a regular user, backing up to a different machine over ssh, and it's no longer going so well. I'm trying to follow the man page and the pull example in Advanced ZFS and am getting a permission error on the receiving side:
Bash: 
$ zxfer -dFkPv -g 376 -I com.sun:auto-snapshot -O replicator@192.168.1.108 -R zroot backup/bsdclient
Checking grandfather status of all snapshots marked for deletion...
Grandfather check passed.
Creating destination filesystem "backup/bsdclient/zroot" with specified properties.
cannot create 'backup/bsdclient/zroot': permission denied
Error when creating destination filesystem.

However, I can immediately create that zroot filesystem by hand, so I don't understand why it didn't work.
Bash: 
$ zfs create backup/bsdclient/zroot

I am the replicator user and I think I have the correct permissions.
Bash: 
$ whoami
replicator

$ zfs allow backup
---- Permissions on backup -------------------------------------------
Local+Descendent permissions:
        user replicator compression,create,destroy,mount,mountpoint,receive,rollback

$ sysctl vfs.usermount
vfs.usermount: 1

$ ls -alg /backup
total 15
drwxr-xr-x   6 root        wheel        6 Jun 23 21:45 .
drwxr-xr-x  20 root        wheel       26 Jun 23 21:44 ..
drwxr-xr-x   3 replicator  replicator   3 Jun 24 00:02 bsdclient

$ ls -alg /backup/bsdclient/
total 2
drwxr-xr-x  3 replicator  replicator  3 Jun 24 00:02 .
drwxr-xr-x  6 root        wheel       6 Jun 23 21:45 ..
drwxr-xr-x  2 replicator  replicator  2 Jun 24 00:02 zroot

If I run zxfer again it gets farther, but again fails with a similar error.
Bash: 
$ zxfer -dFkPv -g 376 -I com.sun:auto-snapshot -O replicator@192.168.1.108 -R zroot backup/bsdclient
Checking grandfather status of all snapshots marked for deletion...
Grandfather check passed.
Sending zroot@zfs-auto-snap_frequent-2020-06-23-23h45 to backup/bsdclient/zroot. Sending zroot@zfs-auto-snap_hourly-2020-06-24-00h00 to backup/bsdclient/zroot.
  (incremental to zroot@zfs-auto-snap_frequent-2020-06-23-23h45.)
Sending zroot@zfs-auto-snap_daily-2020-06-24-00h07 to backup/bsdclient/zroot.
  (incremental to zroot@zfs-auto-snap_hourly-2020-06-24-00h00.)
Creating destination filesystem "backup/bsdclient/zroot/ROOT" with specified properties.
cannot mount '/backup/bsdclient/zroot/ROOT': failed to create mountpoint
filesystem successfully created, but not mounted
Error when creating destination filesystem.

Can anyone see which permissions I'm misssing? Is there a log that might give more info?
 
Funny. If I run that same command as root it works properly, but I have to enter the replicator's ssh password a few dozen times:
Code: 
# zxfer -dFkPv -g 376 -I com.sun:auto-snapshot -O replicator@192.168.1.108 -R zroot backup/bsdclient
Password for replicator@freebsd:
[...]
Password for replicator@freebsd:
Creating destination filesystem "backup/bsdclient/zroot/var/tmp" with specified properties.
Writing backup info to location /backup/bsdclient/.zxfer_backup_info.zroot

Changing the owner of those new backup files after running once as root allows the replicator to run the command as desired, so the only holdup seems to be that the replicator can't create any new datasets on receive.
Code: 
# chown -R replicator:replicator /backup/bsdclient
   
# su - replicator
   
$ zxfer -dFkPv -g 376 -I com.sun:auto-snapshot -O replicator@192.168.1.108 -R zroot backup/bsdclient
[...]
Writing backup info to location /backup/bsdclient/.zxfer_backup_info.zroot
 
Okay, making progress. Running it with -x reveals the problem command:
Code: 
$ sh -x /usr/local/sbin/zxfer -dFkPv -g 376 -I com.sun:auto-snapshot -O replicator@192.168.1.108 -R zroot backup/bsdclient
[...]
+ /sbin/zfs create -o 'quota=none' -o 'reservation=none' -o 'recordsize=131072' -o 'sharenfs=off' -o 'checksum=on' -o 'compression=lz4' -o 'atime=off' -o 'exec=on' -o 'setuid=on' -o 'readonly=off' -o 'jailed=off' -o 'snapdir=hidden' -o 'canmount=on' -o 'copies=1' -o 'utf8only=off' -o 'normalization=none' -o 'casesensitivity=sensitive' -o 'sharesmb=off' -o 'refquota=none' -o 'refreservation=none' -o 'logbias=latency' -o 'dedup=off' -o 'sync=standard' -o 'volmode=default' -o 'filesystem_limit=none' -o 'snapshot_limit=none' -o 'redundant_metadata=all' backup/bsdclient/zroot
cannot create 'backup/bsdclient/zroot': permission denied

Indeed, that command doesn't work when run by hand either, so one or more of those 27 options are causing the problem:
Code: 
$ zfs create -o 'quota=none' -o 'reservation=none' -o 'recordsize=131072' -o 'sharenfs=off' -o 'checksum=on' -o 'compression=lz4' -o 'atime=off' -o 'exec=on' -o 'setuid=on' -o 'readonly=off' -o 'jailed=off' -o 'snapdir=hidden' -o 'canmount=on' -o 'copies=1' -o 'utf8only=off' -o 'normalization=none' -o 'casesensitivity=sensitive' -o 'sharesmb=off' -o 'refquota=none' -o 'refreservation=none' -o 'logbias=latency' -o 'dedup=off' -o 'sync=standard' -o 'volmode=default' -o 'filesystem_limit=none' -o 'snapshot_limit=none' -o 'redundant_metadata=all' backup/bsdclient/zroot
cannot create 'backup/bsdclient/zroot': permission denied

Do any of those options seem alarming?
 
I gave up trying to figure this out. Here's a workaround in case someone else needs it.

Wrap the backup call in a script. Add the -i option so the root knows where to find the keys.
Code: 
# cat /usr/home/replicator/backup.sh
#!/bin/sh
sh -x /usr/local/sbin/zxfer -dFkPv -g 376 -I com.sun:auto-snapshot -O 'replicator@192.168.1.108 -i /usr/home/replicator/.ssh/id_rsa' -R zroot backup/bsdclient

Install and configure doas:
Code: 
# pkg install -y doas
[...]
# touch /usr/local/etc/doas.conf
# echo 'permit nopass replicator as root cmd /usr/home/replicator/backup.sh' >> /usr/local/etc/doas.conf

Run the script as the replicator user using doas:
Code: 
# su - replicator
$ doas /usr/home/replicator/backup.sh; echo $?
[...]
0
$

If you see this and figure out how to solve the ZFS permission problems, please let me know.
 
I ran into the exact problem you're having, and I did 2 things before testing, and at least one of them worked :)

The first was to give my replicator user the following zfs permissions to the replicated dataset: atime,canmount,casesensitivity,checksum,clone,compression,copies,create,dedup,destroy,exec,filesystem_limit,hold,jailed,logbias,mount,mountpoint,normalization,promote,quota,readonly,receive,recordsize,redundant_metadata,refquota,refreservation,release,rename,reservation,rollback,send,setuid,sharenfs,sharesmb,snapdir,snapshot,snapshot_limit,sync,userprop,utf8only,volmode

I came to that list by following your suggestion of sh -x and seeing what the -o arguments that zxfer used.

I also used the -U flag to zxfer.

After doing both of those things, zxfer was able to create the dataset and syncing is now functional.
 
You must log in or register to reply here.
Back
Top