Solved System stop working after snapshot rollback.

Hermann

Hermann

Hi all, I'm just a FreeBSD learner.

Learning and practicing about ZFS filesystem , snapshots, rollback...;

System: FreeBSD LUNA 12.2-RELEASE-p4 FreeBSD 12.2-RELEASE-p4 GENERIC amd64

If I do a rollback from snapshot, first time all runs OK;
But Second time I rollback, system hangs, no response from command shell (only let me write text);
If I change terminal can't log; Nevertheless, rebooting shows me rollback went fine.

Please help me, or give me some advice about how to further research, finding where I could search logs or info to solve it. Maybe it's a hardware problem??
Thank you.
 
Ctrl +t shows:

Cmd: zfs 1626 vmopar 1000.19r 0.00u 0.00s 0% 1480k,

Please any suggestions?
 

Attachments

  • 9D5A6D54-C721-4745-8524-E4B1E6093C80.jpeg
    9D5A6D54-C721-4745-8524-E4B1E6093C80.jpeg
    765 KB · Views: 226
Hermann said:
If I do a rollback from snapshot, first time all runs OK;
But Second time I rollback, system hangs, no response from command shell (only let me write text);
If I change terminal can't log; Nevertheless, rebooting shows me rollback went fine.
Click to expand...
I don't use ZFS so I can't really help, other than say I don't understand what the above means?

What happens after the first rollback? Everything works and you have no issues?

Why do you rollback a second time? Just to see this bug/issue?

"Rebooting shows me rollback went fine" - so if it was fine, what is the problem? Can you not login after this or not?

Terminal can't log? Log what to where? Are you meaning log messages? Or that you don't get a login prompt?

What is the command shell? Do you mean you HAVE been able to login, but any commands you type are ignored? What prompts do you see (if any?)

I think if you can make it a bit clearer what you are trying to achieve and what steps you are taking, and what exactly you are seeing - then someone will be able to help.
 
Hermann said:
If I do a rollback from snapshot, first time all runs OK;
But Second time I rollback, system hangs, no response from command shell (only let me write text);
Click to expand...
What do you mean by that (second time)?

Provide us some extra information zpool status, zpool history and zfs list -t all

If it is not working, create yourself a bootable (any cheap SSD will do) device with fresh FreeBSD install. Connect both drives to the computer, import old pool and see what is there. When installing your rescue system, do not use the default 'zroot' name for the pool. That will give you some extra freedom manipulating external pools.
 
richardtoohey2 said:
I don't use ZFS so I can't really help, other than say I don't understand what the above means?

What happens after the first rollback? Everything works and you have no issues?

Why do you rollback a second time? Just to see this bug/issue?

"Rebooting shows me rollback went fine" - so if it was fine, what is the problem? Can you not login after this or not?

Terminal can't log? Log what to where? Are you meaning log messages? Or that you don't get a login prompt?
Click to expand...

richardtoohey2 said:
What is the command shell? Do you mean you HAVE been able to login, but any commands you type are ignored? What prompts do you see (if any?)

I think if you can make it a bit clearer what you are trying to achieve and what steps you are taking, and what exactly you are seeing - then someone will be able to help.
Click to expand...

Thank you Richard and Argentum; First of all, excuse my lack of knowledge, also I'm not a native english speaker;

a) After first rollback all things go correct, shell (command prompt) respond as usual and files come back to restore snapshot point;
b ) But, if I make any new change and rollback again, shell doesn't come back, also if I change terminal ttyv0 --> ttvv2 or ttyv3, shell appears but I can not type so I need to reboot; Once rebooted files are Ok, snapshot worked as expected . I do a second rollback just for practice, no more; I'm playing and practicing just before doing something more important.

root@LUNA:/ # rm COPYRIGHT

root@LUNA:/ # zfs rollback zroot/ROOT/default@16_03_2021

root@LUNA:/ # ls

.cshrc bin entropy lib mnt rescue sys var

.profile boot etc libexec net root tmp zroot

COPYRIGHT dev home media proc sbin usr

root@LUNA:/ # zfs rollback zroot/ROOT/default@16_03_2021

And then shell doesn't work. I need to reboot.
 
Argentum said:
What do you mean by that (second time)?

Provide us some extra information zpool status, zpool history and zfs list -t all

If it is not working, create yourself a bootable (any cheap SSD will do) device with fresh FreeBSD install. Connect both drives to the computer, import old pool and see what is there. When installing your rescue system, do not use the default 'zroot' name for the pool. That will give you some extra freedom manipulating external pools.
Click to expand...

Thanks Argentum; I'll try to clear things up.
First rollback: All goes fine;
Second rollback, after some files changes, then the problem happens.


root@LUNA:/home/hermann # zpool status
pool: zroot
state: ONLINE
config:
NAME STATE READ WRITE CKSUM
zroot ONLINE 0 0 0
nvd0p3 ONLINE 0 0 0

errors: No known data errors


root@LUNA:/home/hermann # zpool history

History for 'zroot':

2021-03-18.00:31:43 zpool create -o altroot=/mnt -O compress=lz4 -O atime=off -m none -f zroot nvd0p3
2021-03-18.00:31:43 zfs create -o mountpoint=none zroot/ROOT
2021-03-18.00:31:43 zfs create -o mountpoint=/ zroot/ROOT/default
2021-03-18.00:31:43 zfs create -o mountpoint=/tmp -o exec=on -o setuid=off zroot/tmp
2021-03-18.00:31:43 zfs create -o mountpoint=/usr -o canmount=off zroot/usr
2021-03-18.00:31:43 zfs create zroot/usr/home
2021-03-18.00:31:43 zfs create -o setuid=off zroot/usr/ports
2021-03-18.00:31:43 zfs create zroot/usr/src
2021-03-18.00:31:43 zfs create -o mountpoint=/var -o canmount=off zroot/var
2021-03-18.00:31:43 zfs create -o exec=off -o setuid=off zroot/var/audit
2021-03-18.00:31:43 zfs create -o exec=off -o setuid=off zroot/var/crash
2021-03-18.00:31:43 zfs create -o exec=off -o setuid=off zroot/var/log
2021-03-18.00:31:43 zfs create -o atime=on zroot/var/mail
2021-03-18.00:31:43 zfs create -o setuid=off zroot/var/tmp
2021-03-18.00:31:43 zfs set mountpoint=/zroot zroot
2021-03-18.00:31:43 zpool set bootfs=zroot/ROOT/default zroot
2021-03-18.00:31:43 zpool set cachefile=/mnt/boot/zfs/zpool.cache zroot
2021-03-18.00:31:46 zfs set canmount=noauto zroot/ROOT/default
2021-03-17.23:52:14 zfs snapshot zroot/ROOT/default@hoy
2021-03-17.23:53:58 zfs rollback zroot/ROOT/default@hoy
2021-03-18.00:04:31 zfs rollback zroot/ROOT/default@hoy
2021-03-18.18:30:22 zfs snapshot zroot/ROOT/default@16_03_2021
2021-03-18.18:31:33 zfs rollback zroot/ROOT/default@16_03_2021
2021-03-18.19:43:17 zfs rollback zroot/ROOT/default@16_03_2021


root@LUNA:/home/hermann # zfs list -t all


NAME USED AVAIL REFER MOUNTPOINT
zroot 937M 429G 96K /zroot
zroot/ROOT 933M 429G 96K none
zroot/ROOT/default 933M 429G 932M /
zroot/ROOT/default@hoy 320K - 932M -
zroot/ROOT/default@16_03_2021 280K - 932M -
zroot/tmp 144K 429G 144K /tmp
zroot/usr 416K 429G 96K /usr
zroot/usr/home 128K 429G 128K /usr/home
zroot/usr/ports 96K 429G 96K /usr/ports
zroot/usr/src 96K 429G 96K /usr/src
zroot/var 692K 429G 96K /var
zroot/var/audit 96K 429G 96K /var/audit
zroot/var/crash 96K 429G 96K /var/crash
zroot/var/log 212K 429G 212K /var/log
zroot/var/mail 96K 429G 96K /var/mail
zroot/var/tmp 96K 429G 96K /var/tmp

Thank you again for your patience.
 
Hermann said:
root@LUNA:/ # zfs rollback zroot/ROOT/default@16_03_2021
Click to expand...
My hypothesis here is that you try to roll back the file system which can not be unmounted. If the file system cannot be unmounted, the rollback fails. Rollback automatically tries to unmount the dataset and mount it back after that. You are trying to roll back the main dataset in your system you are working on. This is just hypothesis. You can try to import this pool to another system and try there. Also, when installing your rescue system, use some other pool name than default zroot. Just because you can not import a pool to the system which already has a pool with a same name.
 
Remember to use the zfs snapshot -r option.
And when rollbacking, you might want or not want to use the zfs rollback -r option.
Remember also, you have to rollback every dependent dataset.

See man zfs :
zfs snapshot|snap [-r] [-o property=value]...
filesystem@snapname|volume@snapname
filesystem@snapname|volume@snapname...

Creates snapshots with the given names. All previous modifications by
successful system calls to the file system are part of the snapshots.
Snapshots are taken atomically, so that all snapshots correspond to
the same moment in time. See the "Snapshots" section for details.

-r Recursively create snapshots of all descendent datasets

-o property=value
Sets the specified property; see "zfs create" for details.

zfs rollback [-rRf] snapshot

Roll back the given dataset to a previous snapshot. When a dataset is
rolled back, all data that has changed since the snapshot is
discarded, and the dataset reverts to the state at the time of the
snapshot. By default, the command refuses to roll back to a snapshot
other than the most recent one. In order to do so, all intermediate
snapshots and bookmarks must be destroyed by specifying the -r
option.

The -rR options do not recursively destroy the child snapshots of a
recursive snapshot. Only direct snapshots of the specified
filesystem are destroyed by either of these options. To completely
roll back a recursive snapshot, you must rollback the individual
child snapshots.

-r Destroy any snapshots and bookmarks more recent than the one
specified.

-R Destroy any more recent snapshots and bookmarks, as well as
any clones of those snapshots.

-f Used with the -R option to force an unmount of any clone file
systems that are to be destroyed.
 
Argentum said:
My hypothesis here is that you try to roll back the file system which can not be unmounted. If the file system cannot be unmounted, the rollback fails. Rollback automatically tries to unmount the dataset and mount it back after that. You are trying to roll back the main dataset in your system you are working on. This is just hypothesis. You can try to import this pool to another system and try there. Also, when installing your rescue system, use some other pool name than default zroot. Just because you can not import a pool to the system which already has a pool with a same name.
Click to expand...
Perfect Snurg; Thank you. I did what you told me. New FreeBSD system in usb disc. Imported old zroot as zback, and mounted. Then rollbacks worked perfect, no hangs, no problems;
Your hyphotesis is RIGHT: I was trying to roll back the main dataset in the system I was working on! So that rollback didnt't work!
Sorry but I've not read this before, I'm a newbie :). This problem has made me learn a lot, thank you all again.
 
Well, I learned the it similar way like you when I tried snapshotting and wondered :)
I try reading man pages, but it sometimes takes quite a while until I understand.

And richardtoohey2 is right... a dedicated computer to experiment with is a must!
 
Hermann said:
Perfect Snurg; Thank you. I did what you told me. New FreeBSD system in usb disc. Imported old zroot as zback, and mounted. Then rollbacks worked perfect, no hangs, no problems;
Your hyphotesis is RIGHT: I was trying to roll back the main dataset in the system I was working on! So that rollback didnt't work!
Sorry but I've not read this before, I'm a newbie :). This problem has made me learn a lot, thank you all again.
Click to expand...
Only I am not Snurg ?
I think you can you can close this thread now by setting it solved.
 
Hermann said:
Perfect Snurg; Thank you. I did what you told me. New FreeBSD system in usb disc. Imported old zroot as zback, and mounted. Then rollbacks worked perfect, no hangs, no problems;
Your hyphotesis is RIGHT: I was trying to roll back the main dataset in the system I was working on! So that rollback didnt't work!
Sorry but I've not read this before, I'm a newbie :). This problem has made me learn a lot, thank you all again.
Click to expand...

For your awareness, the layout you have was designed to support boot environments.

Boot environments are a great way to be able to "roll back" to a previous system state (at least, for those parts of the filesystem that are contained in them, which for your setup is everything except /tmp, /usr/home, ... the set of usr/* and var/* subdirectories that were created as separate zfs filesystems.) This is achieved through snapshots, clones, the zpool bootfs property, and reboots (as you've noticed, it is hard to swap out your active / mount).

Check out sysutils/beadm or bectl(8). (I personally prefer beadm; bectl is more willing to allow foot-shooting.) Everything could be done manually, but these tools help automate the snapshot / clone / clone promotion / zpool settings in one step.

With them, testing something out, and then going back, can be as easy as:

beadm create working
<do modifications; decide you don't like the results>
beadm activate working
shutdown -r now

They're one of the best features of running your root filesystem on zfs, the other being automated snapshots via tools like sysutils/zfstools or sysutils/zfsnap2.

Pro tip: you can create a new boot environment from an existing snapshot, so if you're running auto-snapshots and then accidentally foul something up, you can:

beadm create -e beName@working_time recovered
beadm activate recovered
shutdown -r now

and act like it never happened.
 
Eric A. Borisch said:
For your awareness, the layout you have was designed to support boot environments.

Boot environments are a great way to be able to "roll back" to a previous system state (at least, for those parts of the filesystem that are contained in them, which for your setup is everything except /tmp, /usr/home, ... the set of usr/* and var/* subdirectories that were created as separate zfs filesystems.) This is achieved through snapshots, clones, the zpool bootfs property, and reboots (as you've noticed, it is hard to swap out your active / mount).

Check out sysutils/beadm or bectl(8). (I personally prefer beadm; bectl is more willing to allow foot-shooting.) Everything could be done manually, but these tools help automate the snapshot / clone / clone promotion / zpool settings in one step.

With them, testing something out, and then going back, can be as easy as:

beadm create working
<do modifications; decide you don't like the results>
beadm activate working
shutdown -r now

They're one of the best features of running your root filesystem on zfs, the other being automated snapshots via tools like sysutils/zfstools or sysutils/zfsnap2.

Pro tip: you can create a new boot environment from an existing snapshot, so if you're running auto-snapshots and then accidentally foul something up, you can:

beadm create -e beName@working_time recovered
beadm activate recovered
shutdown -r now

and act like it never happened.
Click to expand...
Thanks for your great advice. Last three days (in free time) were intense grasping the basic freebsd directories and datasets structure and boot environments;
My first aim is to reverse the main system in case I do something wrong, and so I think beadm (correct me if no) is not only for Update System but for user mistakes;
I'm now reading beadm man, is not a very difficult program for me, and very helpful.
 
Hermann said:
Thanks for your great advice. Last three days (in free time) were intense grasping the basic freebsd directories and datasets structure and boot environments;
My first aim is to reverse the main system in case I do something wrong, and so I think beadm (correct me if no) is not only for Update System but for user mistakes;
I'm now reading beadm man, is not a very difficult program for me, and very helpful.
Click to expand...
If you need a complete protection, run recursive snapshot: zfs snapshot -r your_pool@workin_good. You can use the zfs send to move this snapshot to a new pool when needed and make this new pool bootable after that. This gives a full protection against everything you do in the system after that snapshot.
 
You must log in or register to reply here.
Back
Top