Solved gpart 4k alignment

rdslw · Feb 19, 2018

I do not understand why gpart thinks that 4k alignment means starting first partition at sector 41, instead of usual 40. Sector size is 512, so 4k alignment should result in starting 40.

Code:

[root@cyan ~]# gpart create -s GPT ada0
ada0 created
[root@cyan ~]# gpart add -t freebsd-boot -s 1024 ada0
ada0p1 added
[root@cyan ~]# gpart show ada0
=>        40  3907029088  ada0  GPT  (1.8T)
          40           1        - free -  (512B)
          41        1024     1  freebsd-boot  (512K)
        1065  3907028063        - free -  (1.8T)

On the other hand if I manually specify starting point of the partition, gpart complains about alignment:

Code:

[root@cyan ~]# gpart delete -i 1 ada0
ada0p1 deleted
[root@cyan ~]# gpart add -t freebsd-boot -b 40 -s 1024 ada0
ada0p1 added, but partition is not aligned on 4096 bytes
[root@cyan ~]# gpart show ada0
=>        40  3907029088  ada0  GPT  (1.8T)
          40        1024     1  freebsd-boot  (512K)
        1064  3907028064        - free -  (1.8T)

It may be related to this specific disk stripe size (4096) and stripe offset, but I'm not sure.

Can someone tell me if I should force it with -b40 and ignore 'partition is not aligned on 4096 bytes', or rather trust -a4k option of gpart?
Bonus point: explain it with some calculations

This is diskinfo:

Code:

[root@cyan ~]# diskinfo -v ada0
ada0
        512             # sectorsize
        2000398934016   # mediasize in bytes (1.8T)
        3907029168      # mediasize in sectors
        4096            # stripesize
        3584            # stripeoffset
        3876021         # Cylinders according to firmware.
        16              # Heads according to firmware.
        63              # Sectors according to firmware.
        9WM020P5        # Disk ident.
        Not_Zoned       # Zone Mode

Beastie · Feb 19, 2018

40*512=20480/4096=5
41*512=20992/4096=5.125

rdslw · Feb 19, 2018

Beastie said:
40*512=20480/4096=5
41*512=20992/4096=5.125

Thats the part I understand

Problem is, gpart is insisting on 41, while complaining that 40 is NOT (sic!) aligned properly.

tingo · Feb 19, 2018

Obviously, you (and gpart) are confused. Then answer is (of course) 42. :-D

phoenix · Feb 19, 2018

What's the starting number for the first sector on a disk? 0 or 1?

swegen · Feb 20, 2018

rdslw said:
Sector size is 512, so 4k alignment should result in starting 40.

Correct. However, your diskinfo stripeoffset is off by one sector from 4k (4096-512=3584). I think this is why it offers sector 41 as 4k aligned start sector of the first partition. And the -a4k option of gpart is correct. What controller is this disk connected to?

I checked my disks which are connected directly to mainboard and the stripeoffset for all of them is 0.

rdslw · Feb 20, 2018

swegen said:
Correct. However, your diskinfo stripeoffset is off by one sector from 4k (4096-512=3584). I think this is why it offers sector 41 as 4k aligned start sector of the first partition. And the -a4k option of gpart is correct. What controller is this disk connected to?

I checked my disks which are connected directly to mainboard and the stripeoffset for all of them is 0.

Actually I have two disk connected to motherboard (details below). This is dual cpu asus serverboard. While disks are different devices, they are connected to the same controller: one (ada0) has this peculiar problem, while other (ada1) does not. Ada1 stipesize and stripeoffset are both 0.

What also puzzles me is that stripeoffset is not 512, but 3584, while 4k align sets just one sector offset.

Any other ideas?

Code:

ada0 at ahcich0 bus 0 scbus0 target 0 lun 0                                                                                                                                                    
ada0: <ST32000540AS CC83> ATA8-ACS SATA 2.x device                                                                                                                                             
ada0: Serial Number 9WM020P5                                                                                                                                                                   
ada0: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)                                                                                                                                   
ada0: Command Queueing enabled                                                                                                                                                                 
ada0: 1907729MB (3907029168 512 byte sectors)                                                                                                                                                  
ada1 at ahcich2 bus 0 scbus2 target 0 lun 0                                                                                                                                                    
ada1: <Hitachi HUA723020ALA641 MK7OA840> ATA8-ACS SATA 3.x device                                                                                                                              
ada1: Serial Number YFH1YX5A                                                                                                                                                                   
ada1: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)                                                                                                                                   
ada1: Command Queueing enabled                                                                                                                                                                 
ada1: 1907729MB (3907029168 512 byte sectors)                                                                                                                                                
                                                                                                                                                           
ahci0: <Intel Patsburg AHCI SATA controller> port 0x9070-0x9077,0x9060-0x9063,0x9050-0x9057,0x9040-0x9043,0x9020-0x903f mem 0xcf605000-0xcf6057ff irq 19 at device 31.2 numa-domain 0 on pci0  
ahci0: AHCI v1.30 with 6 6Gbps ports, Port Multiplier not supported                                                                                                                                                                                                                                         
ada0 at ahcich0 bus 0 scbus0 target 0 lun 0                                                                                                                                                    
ada1 at ahcich2 bus 0 scbus2 target 0 lun 0

rdslw · Feb 20, 2018

phoenix said:
What's the starting number for the first sector on a disk? 0 or 1?

Thats a very good question who solved the puzzle

Thank you for it. I must admit I was very close to dismiss you message (says a lot about cognitive confirmation bias), but managed to get some thinking around your question.

Solution:
Actually if I want parition to be 4k aligned, the first partition must start (contrary to my thinking and some post in the thread

) on sector 41, not 40.

Hence disk ada0 (and gpart) tried to suggest me correctly into layout starting at sector 41 (and then 1065, 2049 etc, in my partition layout). I will verify freebsd handbook, because I think I picked up that 40 from some gpt partition table guide in handbook.

The problem did not manifest on my disk ada1 (and on majority of market disks) which is/are 512b/sector, while my strangely new seagate ada0 has underlying 4096 sector size.

Lookslike I'll be using tonight zfs capabilities to remove them one by one and re-partition correctly to be 4k aligned.

Elliott · Sep 30, 2019

I found this thread while trying to make sure my drives are aligned properly, and my results are different than the OP. This is aligning at block 40, which makes sense to me, assuming the first sector is number 0. Did the math change in some new version? I am running FreeNAS 11.2 U5.
I am posting here instead of their forums because this seems like an upstream issue and this is the best discussion I have found on the topic. I hope this is allowed here.

Code:

root@freenas[~]# gpart create -s GPT da22
da22 created
root@freenas[~]# gpart add -t freebsd-boot -s 1024 da22
da22p1 added
root@freenas[~]# gpart show da22
=>        40  1953525088  da22  GPT  (932G)
          40        1024     1  freebsd-boot  (512K)
        1064  1953524064        - free -  (932G)

CyberCr33p · May 12, 2022

Code:

diskinfo -v nvd0
nvd0
        512             # sectorsize
        3840755982336   # mediasize in bytes (3.5T)
        7501476528      # mediasize in sectors
        131072          # stripesize
        0               # stripeoffset
        SAMSUNG MZQL23T8HCLS-00A07      # Disk descr.
        S64HNE0T306250  # Disk ident.
        nvme0           # Attachment
        Yes             # TRIM/UNMAP support
        0               # Rotation rate in RPM

When I create partitions using commands:

Code:

gpart add -i 1 -b 40 -s 512k -t freebsd-boot nvd0
nvd0p1 added, but partition is not aligned on 131072 bytes
gpart add -i 2 -a 1m -s 16G -t freebsd-swap nvd0
gpart add -i 3 -a 1m -t freebsd-zfs nvd0

gpart show
=>        40  7501476448  nvd0  GPT  (3.5T)
          40        1024     1  freebsd-boot  (512K)
        1064         984        - free -  (492K)
        2048    33554432     2  freebsd-swap  (16G)
    33556480  7467919360     3  freebsd-zfs  (3.5T)
  7501475840         648        - free -  (324K)

When I do a bsdinstall and choose Auto (ZFS) partitioning :

Code:

gpart show
=>        40  7501476448  nvd0  GPT  (3.5T)
          40        1024     1  freebsd-boot  (512K)
        1064         984        - free -  (492K)
        2048    33554432     2  freebsd-swap  (16G)
    33556480  7467919360     3  freebsd-zfs  (3.5T)
  7501475840         648        - free -  (324K)

When I do a bsdinstall and choose Auto (UFS) partitioning :

Code:

gpart show
=>        40  7501476448  nvd0  GPT  (3.5T)
          40         216        - free -  (108K)
         256        1024     1  freebsd-boot  (512K)
        1280  7493122816     2  freebsd-ufs  (3.5T)
  7493124096     8352392     3  freebsd-swap  (4.0G)

Any idea why Auto ZFS and UFS partitioning choose different settings and which partitioning scheme is the correct?

Also why in the first example it says nvd0p1 added, but partition is not aligned on 131072 bytes?

swegen · May 12, 2022

The UFS partitioning script tries to take the stripe size into account and results in all three partitions being correctly aligned.

ZFS partitioning script leaves freebsd-boot misaligned but the other two partitions are still in this case correctly aligned. However, this won't matter much as the freebsd-boot partition is read only once during boot and written only on installation.

Still, the Auto (ZFS) script leaves a lot of space unused for the resulting zpool. To maximize the space utilization one would have to set the partition size according to ZFS internal metaslabs and add 4.5M of additional space for vdev labels and bootblock reservation.

The following creates enough space for exactly 223 × 16G metaslabs with no wasted space for your disk. You can verify this with zpool get -p size.

Code:

gpart add -t freebsd-boot -s 984 nvd0 # align to 512K
gpart add -t freebsd-zfs -s 3741323776K nvd0 # 3568G + 4.5M
gpart add -t freebsd-swap nvd0 # rest (about 9G)

CyberCr33p · May 12, 2022

swegen said:
Still, the Auto (ZFS) script leaves a lot of space unused for the resulting zpool. To maximize the space utilization one would have to set the partition size according to ZFS internal metaslabs and add 4.5M of additional space for vdev labels and bootblock reservation.

The following creates enough space for exactly 223 × 16G metaslabs with no wasted space for your disk. You can verify this with zpool get -p size.

Code:

gpart add -t freebsd-boot -s 984 nvd0 # align to 512K gpart add -t freebsd-zfs -s 3741323776K nvd0 # 3568G + 4.5M gpart add -t freebsd-swap nvd0 # rest (about 9G)

Code:

zpool get -p size
NAME   PROPERTY  VALUE          SOURCE
zroot  size      3813930958848  -

The disk is a 3.8TB, so it doesn't look like it wastes space, am I right? Or at least it doesn't waste many GB.

EDIT: I check again and it wastes 26GB. So it's ok as it's not much.