UFS Experiences with building a softraid 1 using gmirror(8)

Hello folks,

I'm using a remote headless server on 11.4-REALEASE-p8 that is used to host services for my personal needs. The motherboard (an Intel DH67BL) handles softraid capabilities and thinking of using a second disk to make a raid 1.

Reading the handbook at chapter 19.3.3, it's possible to make a raid on an already installed system using gmirror(8) without having to reinstall the system, that's good news! Of course, before doing so, i would like to ask if anyone has gone through such a process and how well it was going (all datas are safely backed up twice a day to a remote host at home so the only thing I can lost is time if I need to reinstall my FreeBSD).
 

Attachments

gmirror(8) is software only, while graid(8) does leverage the hardware-assisted RAID of the chipset on the MB. Beware though, that this means you can not easily migrate the disks to another MB with a different chipset. If you want to to go ZFS (usually you should), you only need to mirror swap with gmirror(8) or graid(8) and use ZFS mirror for your data. If your disk is >9TB, please configure a 3-way mirror (the statistics of likelihood of an error demands that).
 
gmirror(8) is software only, while graid(8) does leverage the hardware-assisted RAID of the chipset on the MB. Beware though, that this means you can not easily migrate the disks to another MB with a different chipset. If you want to to go ZFS (usually you should), you only need to mirror swap with gmirror(8) or graid(8) and use ZFS mirror for your data. If your disk is >9TB, please configure a 3-way mirror (the statistics of likelihood of an error demands that).
Thanks for your input and insights of differences between gmirror(8) and graid(8) :beer:
Unfortunately that hosting provider allow only 2x2TB disks and when I did install that system back in 2002, I've gone UFS and still using it (despite all ZFS's goodnesses).
 
You can use graid(8) only if you first set the SATA option in the bios to RAID and after you create the Volume select it as first boot device in the BIOS.
For Intel Raid1 you can access the disk on any motherboard so it's not a problem in case of motherboard fail. Otherwise use the software raid from gmirror(8)
 
Do you have a possibility to use IP-KVM for remote console access?

For my opinion, gmirror gives the better results than graid. It is well-confirured mainboard-independent solution.
I use gmirror at least for 12 years. It is a good cost-efficient solution.
There are many ways for moving an existing system to gmiror.
Handbook article not the best one.

My vision of moving system to gmirror:
Assume the FreeBSD, MBR, UFS. One of two 2TB disks is completely not used and clean.
It is not a step-by-step manual. You have to understand what are you doing.

0) Create and check backup of your data.

1) Create singledrive gmirror from the spare second disk.
gmirror label -F -b prefer name prov1

2) Load gmirror moule gmirror load

3) Create MBR and install BootLoader
fdisk -vBI /dev/mirror/name

4) Create FreeBSD partitions and second boot-loader:
bsdlabel -w -B /dev/mirror/names1
bsdlabel /dev/mirror/names1 > mlabel
ee mlabel
Set the partitions and sizes
Code:
#        size   offset    fstype   [fsize bsize bps/cpg]
  a: 2G       16    4.2BSD        0     0
  b: 8G * swap
  c: DO NOT TOUCH IT
  d: 30G * 4.2BSD
  e: 50G * 4.2BSD
  f: 100G * 4.2BSD
  g: * * 4.2BSD
Restore edited FreeBSD partitions:
bsdlabel -R /dev/mirror/names1 mlabel

5) Create the filesystems for every partition exept b and c:
newfs /dev/mirror/names1a
newfs -U /dev/mirror/names1d
newfs -U /dev/mirror/names1e
....


6) mount every new partition to /mnt/[a-g]

7) stop your daemons, copy every existing filesystems to new one on a mirror
cd / && pax -p eme -X -rw . /mnt/a/
cd /var && pax -p eme -X -rw . /mnt/d/
cd /usr && pax -p eme -X -rw . /mnt/e/
...


7) edit new fstab on /mnt/a/etc/fstab, replacing old disk name by /dev/mirror/name
Enable gmirror module on a second disk. Enable gmirror module in mnt/a/boot/loader.conf or /mnt/a/etc/rc.conf

8) unmount all partitions /mnt/*

9) Shutdown the server, ask support to temporary remove 1st disk (if your server has disks with rack)
Try to boot from the second disk, as a single-drive-mirror.
Check dmesg and mount. Be sure that gmirror loaded and filesystems mounted from gmirror-disk.
If system boot correctly - shutdown the server, attach the first source disk,
Enter mainboard bios setup, configure boot from 2nd disk,
Boot OS from half-mirror,
doublecheck everything and add the first disk to the mirror
gmirror insert -p 20 name prov0

10) check how mirror is synchronizing
gmirror status

11) Add the line to the file /etc/periodic.conf
Code:
daily_status_gmirror_enable="YES"                        # Check gmirror(8)

You can try to do it using virtualbox.

If you have limited time for using IP-KVM - then try to skip copying of the largest partition. Just boot from second disk and mount large partition from source disk. You may copy it later without console-access.
For speedup copying useful tool is rsync.
You can prepare a copy of your live system to the 'mirror' using rsync -Havx --numeric-ids /usr/ /mnt/newusr/, and when ready just stop your daemons, and sync the data finally using rsync -Havx [-n] --delete --numeric-ids /usr/ /mnt/newusr/.
Be careful with --delete option.

P.S.
For a long years ago was a method which creates a gmirror directly from a system disk, but nowadays you should not use it.
It started from commands like "sysctl kern.geom.debugflags=17, gmirror label name name-of-existing-system-disk"
 
Do you have a possibility to use IP-KVM for remote console access?

Yes.

For my opinion, gmirror gives the better results than graid. It is well-confirured mainboard-independent solution.
I use gmirror at least for 12 years. It is a good cost-efficient solution.
There are many ways for moving an existing system to gmiror.
Handbook article not the best one.

My vision of moving system to gmirror:
Assume the FreeBSD, MBR, UFS. One of two 2TB disks is completely not used and clean.
It is not a step-by-step manual. You have to understand what are you doing.

0) Create and check backup of your data.

1) Create singledrive gmirror from the spare second disk.
gmirror label -F -b prefer name prov1

2) Load gmirror moule gmirror load

3) Create MBR and install BootLoader
fdisk -vBI /dev/mirror/name

4) Create FreeBSD partitions and second boot-loader:
bsdlabel -w -B /dev/mirror/names1
bsdlabel /dev/mirror/names1 > mlabel
ee mlabel
Set the partitions and sizes
Code:
#        size   offset    fstype   [fsize bsize bps/cpg]
  a: 2G       16    4.2BSD        0     0
  b: 8G * swap
  c: DO NOT TOUCH IT
  d: 30G * 4.2BSD
  e: 50G * 4.2BSD
  f: 100G * 4.2BSD
  g: * * 4.2BSD
Restore edited FreeBSD partitions:
bsdlabel -R /dev/mirror/names1 mlabel

5) Create the filesystems for every partition exept b and c:
newfs /dev/mirror/names1a
newfs -U /dev/mirror/names1d
newfs -U /dev/mirror/names1e
....


6) mount every new partition to /mnt/[a-g]

7) stop your daemons, copy every existing filesystems to new one on a mirror
cd / && pax -p eme -X -rw . /mnt/a/
cd /var && pax -p eme -X -rw . /mnt/d/
cd /usr && pax -p eme -X -rw . /mnt/e/
...


7) edit new fstab on /mnt/a/etc/fstab, replacing old disk name by /dev/mirror/name
Enable gmirror module on a second disk. Enable gmirror module in mnt/a/boot/loader.conf or /mnt/a/etc/rc.conf

8) unmount all partitions /mnt/*

9) Shutdown the server, ask support to temporary remove 1st disk (if your server has disks with rack)
Try to boot from the second disk, as a single-drive-mirror.
Check dmesg and mount. Be sure that gmirror loaded and filesystems mounted from gmirror-disk.
If system boot correctly - shutdown the server, attach the first source disk,
Enter mainboard bios setup, configure boot from 2nd disk,
Boot OS from half-mirror,
doublecheck everything and add the first disk to the mirror
gmirror insert -p 20 name prov0

10) check how mirror is synchronizing
gmirror status

11) Add the line to the file /etc/periodic.conf
Code:
daily_status_gmirror_enable="YES"                        # Check gmirror(8)

You can try to do it using virtualbox.

If you have limited time for using IP-KVM - then try to skip copying of the largest partition. Just boot from second disk and mount large partition from source disk. You may copy it later without console-access.
For speedup copying useful tool is rsync.
You can prepare a copy of your live system to the 'mirror' using rsync -Havx --numeric-ids /usr/ /mnt/newusr/, and when ready just stop your daemons, and sync the data finally using rsync -Havx [-n] --delete --numeric-ids /usr/ /mnt/newusr/.
Be careful with --delete option.

P.S.
For a long years ago was a method which creates a gmirror directly from a system disk, but nowadays you should not use it.
It started from commands like "sysctl kern.geom.debugflags=17, gmirror label name name-of-existing-system-disk"
Wow thanks a lot im ! I'll for sure check every step and not blindly type command but what you described is a nice start to implement raid1 on that server (fortunately booting is done through MBR which as far as I understand ease the migration with graid(8)/gmirror(8) :cool:
 
Back
Top