ZFS Can one create a ZFS mirror of JBOD vdevs?

scotia

scotia

Hi all,

With regard to ZFS: I have a bunch of drives that add up to >22TB, and I have a single 22TB drive.

Can I build a JBOD of the first set of drives, and mirror that JBOD with the 22TB drive?

A bit like have a mirror top-level vdev made up of two second-level vdevs: one being a RAID0/stripe, the other a single device.

I hope I explained the question well enough. Please don't ask why or try to explain the pitfalls of such an idea. It'll save your time and others'. This is a simple "is it possible, and if so how?" question.

Thanks,
Scott

ps. In the past I have created a striped set of mirrors - this is the opposite: a mirrored set of stripes.
 
covacat said:
create a graid or gconcat and use the result as a member of the zfs mirror together with the large drive
Click to expand...
Done and done. Thanks. Am I right in saying that since ZFS does not control the “stripe” device then should a single drive in that stripe set fail then I couldn’t replace the drive and resilver - I’d have to 1) break the mirror 2) destroy and rebuild the stripe set, then 3) mirror the surviving drive again?

Cheers.
 
scotia said:
Done and done. Thanks. Am I right in saying that since ZFS does not control the “stripe” device then should a single drive in that stripe set fail then I couldn’t replace the drive and resilver - I’d have to 1) break the mirror 2) destroy and rebuild the stripe set, then 3) mirror the surviving drive again?

Cheers.
Click to expand...
im not a zfs expert but i don't see another way
 
scotia said:
Done and done. Thanks. Am I right in saying that since ZFS does not control the “stripe” device then should a single drive in that stripe set fail then I couldn’t replace the drive and resilver - I’d have to 1) break the mirror 2) destroy and rebuild the stripe set, then 3) mirror the surviving drive again?

Cheers.
Click to expand...
Correct. ZFS knows none of the fine details hidden inside its vdev providers. Assuming you have lost a disk from a gconcat(8), then the entire concat is lost, and therefore one side of your ZFS mirror has failed.
 
gconcat has no inherent redundancy, correct? If you did not have the other device in the ZFS mirror, lost a device in that gconcat, the whole gconcat is gone (at least that's my understanding).
If you replace that device in the gconcat, ZFS may recognize that and resilver the ZFS mirror over the whole gconcat.
If you lose the single device part of the ZFS mirror and then lose a device in the gconcat, you're in a pretty hard place with no way out.
Make sure you have good backups.
 
One down side to using gconcat that I can see is if I wanted to preemptively replace a disk (before it fails) it doesn't seem possible.
But still it's a solution to my question:
Code: 
# gconcat label GC-STRIPE-01 da0 da1
# gconcat label GC-STRIPE-02 da2 da3
# gconcat status
               Name  Status  Components
concat/GC-STRIPE-01      UP  da0
                             da1
concat/GC-STRIPE-02      UP  da2
                             da3
# zpool create ZP-MIRROR mirror /dev/concat/GC-STRIPE-01 /dev/concat/GC-STRIPE-02
# zpool status ZP-MIRROR
  pool: ZP-MIRROR
 state: ONLINE
config:

        NAME                     STATE     READ WRITE CKSUM
        ZP-MIRROR                ONLINE       0     0     0
          mirror-0               ONLINE       0     0     0
            concat/GC-STRIPE-01  ONLINE       0     0     0
            concat/GC-STRIPE-02  ONLINE       0     0     0

errors: No known data errors
I tried creating striped pools with ZFS, then created ZVOLs on eachpool, but when it came to creating a ZFS mirror with the two ZVOLs I got:
Code: 
# zpool create -f ZP-MIRROR-01 mirror /dev/zvol/ZP-STRIPE-01/ZP-STRIPE-01_zvol /dev/zvol/ZP-STRIPE-02/ZP-STRIPE-02_zvol
cannot create 'ZP-MIRROR-01': no such pool or dataset
 
covacat said:
you can create 1 disk gmirror out of your da's before and use them in gconcat
Click to expand...
I'm after a mirror of stripes (well, a mirror one drive and one stripe set of equal size). Mirroring before the stripe and then mirroring that stripe seems a bit redundant (pun intended).

Does anyone know why ZFS won't allow me to add a ZVOL to a pool? I can add devs, and simple files, but I get the above error when trying with a ZVOL. I thought I'd worked around the ZFS problem of not supporting mirrors of stripes; to no avail.
 
scotia said:
I'm after a mirror of stripes (well, a mirror one drive and one stripe set of equal size). Mirroring before the stripe and then mirroring that stripe seems a bit redundant (pun intended).
Click to expand...
the idea was of a fake mirror (with only one drive) which will allow to change any drive of the gconcat at will without destroying / losing the whole concated disk
so from disks da1 da2 da3 you create mirror/m1 mirror/m2 mirror/m3 and gconcat them
when you want do replace a drive just insert the new drive to the corresponding mirror and remove the old one
 
covacat said:
the idea was of a fake mirror (with only one drive) which will allow to change any drive of the gconcat at will without destroying / losing the whole concated disk so from disks da1 da2 da3 you create mirror/m1 mirror/m2 mirror/m3 and gconcat them when you want do replace a drive just insert the new drive to the corresponding mirror and remove the old one
Click to expand...
Oh I get it. Create single-disk mirrors, stripe those, mirror those stripe sets. Nice one.
This might be a bad idea but I think if I create a couple of ZFS "stripes" then create a file on each the entire pool size I could then create a mirror of those two files. I might try it a see how it performs. I still don't get why I can't mirror ZFS ZVOLs. I might start another thread with just that question.
Thanks.
 
You must log in or register to reply here.
Back
Top