Jump to content
XPEnology Community

XPENology and 4k sectors?

ilciumbia

By ilciumbia
in Archives

 Share

Recommended Posts

ilciumbia Junior Member

ilciumbia

Posted
Posted

Hi all,

 

I have recently set up my system, starting with a pair of 4TB HSGT Deskstar NAS and then adding 1 TB WD green disks.

 

Now I wonder if DSM is capable of correctly recognizing and aligning 4k-sector disks. I don't know if the HGST are in fact 4k-sector disks, but if I run:

 

cat /sys/class/block/sda/queue/physical_block_size

 

I get:

4096

 

However, if I run parted on /dev/sda and I press p I get:

 

Model: HGST HDN724040ALE640 (scsi)

Disk /dev/sda: 4001GB

Sector size (logical/physical): 512B/512B

Partition Table: gpt

Disk Flags:

 

Number Start End Size File system Name Flags

1 1049kB 2551MB 2550MB ext4 raid

2 2551MB 4699MB 2147MB linux-swap(v1) raid

5 4840MB 4001GB 3996GB raid

 

So apparently parted and /sys/class/block/sda/queue/physical_block_size tell two different stories. In fact, you can see that the system partitions reported by gparted are not at all aligned, while the data partition, the most important one, appears to be, but I cannot be certain since the start is given in MB and not in kB, so the rounding could trick me.

 

Then I have another disk, a WD WD10EARS-14Y5B1, which, following cat /sys/class/block/sda/queue/physical_block_size, reports 4096 as well, whereas parted says:

 

Model: WDC WD10EARS-14Y5B1 (scsi)

Disk /dev/sdd: 1000GB

Sector size (logical/physical): 512B/512B

Partition Table: msdos

Disk Flags:

 

Number Start End Size Type File system Flags

1 1049kB 2551MB 2550MB primary raid

2 2551MB 4699MB 2147MB primary raid

3 4832MB 1000GB 995GB primary raid

 

Again, parted does not agree in the physycal sector section and system partitions are exactly as for the HGST (so not 4k aligned). However, the data partition appears to be aligned, with the same provision as before.

 

Curiously enough, then, I have another WD disk, this time the model is reported by parted as:

 

Model: WDC WD10EARS-00MVWB0 (scsi)

Disk /dev/sdc: 1000GB

Sector size (logical/physical): 512B/512B

Partition Table: msdos

Disk Flags:

 

Number Start End Size Type File system Flags

1 1049kB 2551MB 2550MB primary raid

2 2551MB 4699MB 2147MB primary raid

3 4832MB 1000GB 995GB primary raid

 

however, this one answers 512 to the cat /sys/class/block/sda/queue/physical_block_size. Possibly it does not report the physical sector size correctly, because, to my knowledge, the EARS are 4k "advanced format" drives... Am I correct?

 

Being in RAID1 with the previous one, the starting point of partitions are identical.

 

Last disk I have is:

 

Model: WDC WD20EARX-00PASB0 (scsi)

Disk /dev/sdf: 2000GB

Sector size (logical/physical): 512B/512B

Partition Table: msdos

Disk Flags:

 

Number Start End Size Type File system Flags

1 1049kB 2551MB 2550MB primary raid

2 2551MB 4699MB 2147MB primary raid

3 4832MB 2000GB 1996GB extended lba

5 4840MB 2000GB 1995GB logical raid

 

reported again as 4k by cat /sys/class/block/sda/queue/physical_block_size.

 

Also in this case the data partition would seem to be 4k aligned, even if I do not know why it ended up as logical partition instead of primary...???

 

To recap, my question is: why are system partitions not 4k aligned even if the disks report 4096 block size? Is it because it is not important that system partitions are, since they do not require speed, opposite to the data partition? Or did something go wrong during installation?

 

Is there a way to realign existing partitions in a non-destructive way?

 

Also: why the sdf has a logical partition in it?

 

Thanks!!!

Link to comment
Share on other sites
AllGamer Super Member

AllGamer

Posted
Posted

The best way to find out if a HDD is really an "Advance Format Drive" is by checking online the model number and spec from the manufacture websites.

 

I'm not expert but coming from personal experience, Linux systems (DSM) do not have problem with the 4K "Advance Format Drive"

 

Besides once the HDD becomes parts of the RAID, all the formatting goes out the window, as the RAID and format structure will be maintained by the mdadm, so the underlying physical cluster size no longer matters.

 

It only matters most when you try to use it for FAT32 or NTFS partitions for Windows, otherwise under linux and EXT4 partition just blankets over whatever it's underneath.

Link to comment
Share on other sites
TomS. Newbie

TomS.

Posted
Posted
The best way to find out if a HDD is really an "Advance Format Drive" is by checking online the model number and spec from the manufacture websites.

Absolutely agree. The datasheet for the HGST https://www.hgst.com/products/hard-drives/nas-desktop-drive-kit drives in question are silent on the topic though. There is an HGST Advanced Format Technology Brief https://www.hgst.com/sites/default/files/resources/AFtechbrief.pdf that discusses HGST drives and relevant operating system support. It emphasizes that Gparted 2.1 or later be used.

I'm not expert but coming from personal experience, Linux systems (DSM) do not have problem with the 4K "Advance Format Drive"

 

Besides once the HDD becomes parts of the RAID, all the formatting goes out the window, as the RAID and format structure will be maintained by the mdadm, so the underlying physical cluster size no longer matters.

 

It only matters most when you try to use it for FAT32 or NTFS partitions for Windows, otherwise under linux and EXT4 partition just blankets over whatever it's underneath.

I must gently disagree. For all operating systems and disk arrangements, sector size matters.

For single disks, the minimum file system allocation unit must be a multiple of the sector size. Otherwise, misaligned allocation units will cause two sectors to be read / modified / written whenever a file system allocation unit crosses a 4k sector boundary.

 

For multiple disk arrangements, the array stripe size must be a multiple of the drive physical sector size. I believe there should also be a relationship between stripe size and file system allocation unit size as well. I can't recall it offhand.

 

Fundamentally, make sure the start of each label / partition / volume, whatever the OS calls it is 4k aligned.

Link to comment
Share on other sites
ilciumbia Junior Member

ilciumbia

Posted
  • Author
Posted

Hi all, thanks for your replies.

 

The best way to find out if a HDD is really an "Advance Format Drive" is by checking online the model number and spec from the manufacture websites.

Absolutely agree. The datasheet for the HGST https://www.hgst.com/products/hard-drives/nas-desktop-drive-kit drives in question are silent on the topic though.

 

Exactly. WD EARS are indeed Advanced Format, while HGST says nothing about it, but, according to Linux, it does have 4k sectors.

 

Besides once the HDD becomes parts of the RAID, all the formatting goes out the window, as the RAID and format structure will be maintained by the mdadm, so the underlying physical cluster size no longer matters.

 

It only matters most when you try to use it for FAT32 or NTFS partitions for Windows, otherwise under linux and EXT4 partition just blankets over whatever it's underneath.

I must gently disagree. For all operating systems and disk arrangements, sector size matters.

For single disks, the minimum file system allocation unit must be a multiple of the sector size. Otherwise, misaligned allocation units will cause two sectors to be read / modified / written whenever a file system allocation unit crosses a 4k sector boundary.

 

I gently disagree too. Now that I think about it, when I initialized under DSM the EARS disk which is NOT reported as having 4k sectors, the file transfers were extremely slow, around 30MB/s, and I could not understand why. Now I do!

 

Fundamentally, make sure the start of each label / partition / volume, whatever the OS calls it is 4k aligned.

 

Yeah, the point is that, as I showed in my dumps in the OP, DSM system and cache partitions are not, while data partitions are. It is probably not so bad, since DSM system does not need that much speed, however the reason behind this strange behaviour puzzles me...

Link to comment
Share on other sites
TomS. Newbie

TomS.

Posted
Posted

Ilciumba,

 

On my XPE box, uname -rs returns "Linux 3.10.35" and parted returns "GNU Parted v3.1". Both values are more than sufficient for Advanced Format Drive support. I'm not up to par on *nix and *nix like OS, but have a fair amount of experience on other platforms.

 

There are a number of posts on askubuntu about Gparted bugs and AFD disks, no real good answers too much blame.

 

I just started searching the gnu bug-parted mailing list @ http://lists.gnu.org/arc.hive/html/bug-parted/. Maybe you'll have better luck finding an answer than I have so far. I belive one key answer from a developer may be that parted gets the logical & physical sector values from the kernel. It doesn't query the drive itself. Yes, I did note your output from /sys/class/block/sda/queue/physical_block_size at 4096 bytes. I don't know how parted queries the kernel for its information.

 

May I ask you to please repeat your parted output, (print list) for sda-f, but setting the units to bytes (units B) to eliminate rounding? That way we can divide starting sector by 4096 and at least confirm good alignment if it returns integers.

 

Would you also consider the output of parted's align, e.g. align opt 1, for each partition on each sample disk?

 

These are my results from a 250GB x2 quick disk setup creating 1 volume, SHR with data protection, no disk groups in Storage Manager.

(parted) unit B
unit B
(parted) print list
print list
Model: SAMSUNG SP2504C (scsi)
Disk /dev/hda: 250059350016B
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Disk Flags:

Number  Start        End            Size           Type      File system  Flags
1      1048576B     2551054335B    2550005760B    primary                raid
2      2551054336B  4698537983B    2147483648B    primary                raid
3      4831838208B  250052100095B  245220261888B  extended               lba
5      4840079360B  249945178111B  245105098752B  logical                raid


Model: SAMSUNG SP2504C (scsi)
Disk /dev/sda: 250059350016B
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Disk Flags:

Number  Start        End            Size           Type      File system  Flags
1      1048576B     2551054335B    2550005760B    primary                raid
2      2551054336B  4698537983B    2147483648B    primary                raid
3      4831838208B  250052100095B  245220261888B  extended               lba
5      4840079360B  249945178111B  245105098752B  logical                raid


Model: SAMSUNG SP2504C (scsi)
Disk /dev/sdb: 250055122432B
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Disk Flags:

Number  Start        End            Size           Type      File system  Flags
1      1048576B     2551054335B    2550005760B    primary                raid
2      2551054336B  4698537983B    2147483648B    primary                raid
3      4831838208B  250052100095B  245220261888B  extended               lba
5      4840079360B  249945178111B  245105098752B  logical                raid


Model: Linux Software RAID Array (md)
Disk /dev/md0: 2549940224B
Sector size (logical/physical): 512B/512B
Partition Table: loop
Disk Flags:

Number  Start  End          Size         File system  Flags
1      0B     2549940223B  2549940224B  ext4


Model: Linux Software RAID Array (md)
Disk /dev/md1: 2147418112B
Sector size (logical/physical): 512B/512B
Partition Table: loop
Disk Flags:

Number  Start  End          Size         File system     Flags
1      0B     2147418111B  2147418112B  linux-swap(v1)


Error: /dev/md2: unrecognised disk label
Model: Linux Software RAID Array (md)
Disk /dev/md2: 245104050176B
Sector size (logical/physical): 512B/512B
Partition Table: unknown
Disk Flags:

 

Just a quick glossary note:

Advanced Format Drives are 4096 physical sector, but report 512 logical sector, aka 512e or 512 emulation drives.

4k Native report 4096 physical and 4096 logical.

Link to comment
Share on other sites
ilciumbia Junior Member

ilciumbia

Posted
  • Author
Posted
Ilciumba,

 

May I ask you to please repeat your parted output, (print list) for sda-f, but setting the units to bytes (units B) to eliminate rounding? That way we can divide starting sector by 4096 and at least confirm good alignment if it returns integers.

 

Would you also consider the output of parted's align, e.g. align opt 1, for each partition on each sample disk?

 

 

Wow, how right you were! I did as you suggested:

 


Model: HGST HDN724040ALE640 (scsi)
Disk /dev/hda: 4000787030016B
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:

Number  Start        End             Size            File system     Name  Flags
1      1048576B     2551054335B     2550005760B     ext4                  raid
2      2551054336B  4698537983B     2147483648B     linux-swap(v1)        raid
5      4840079360B  4000681082879B  3995841003520B                        raid

 

So in fact partitions ARE 4k aligned!

1048576/4096 = 256

2551054336/4096 = 622816

4840079360/4096 = 1181660

 

I did not know about the "unit" option in GParted! Thank you sooo much!

 

But now I have yet another question: why is the first disk listed twice, once as hda and once as sda??? I see it happens to you as well...

Link to comment
Share on other sites
TomS. Newbie

TomS.

Posted
Posted
Ilciumba,

Would you also consider the output of parted's align, e.g. align opt 1, for each partition on each sample disk?

 

Wow, how right you were! I did as you suggested:

Thanks, luck sure helps.


Model: HGST HDN724040ALE640 (scsi)
Disk /dev/hda: 4000787030016B
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:

Number  Start        End             Size            File system     Name  Flags
1      1048576B     2551054335B     2550005760B     ext4                  raid
2      2551054336B  4698537983B     2147483648B     linux-swap(v1)        raid
5      4840079360B  4000681082879B  3995841003520B                        raid

 

So in fact partitions ARE 4k aligned!

1048576/4096 = 256

2551054336/4096 = 622816

4840079360/4096 = 1181660

 

I did not know about the "unit" option in GParted! Thank you sooo much!

 

What does parted's "align opt "n"" where n is each of the labels on the disk report? In particular does it agree that the partitions is aligned?

 

@berwhale, do you know why parted reports sector sizes of 512 bytes physical / 512 bytes logical for the HGST NAS drives when they are Advanced Format Disks, 4096 bytes physical / 512 logical?

 

But now I have yet another question: why is the first disk listed twice, once as hda and once as sda??? I see it happens to you as well...

 

A pure guess is all I can offer. A quick search indicates that it was a past practice for drivers to allocate /dev/hd(x) to parallel ATA drives and /dev/sd(x) to SCSI & SATA drives. Current practice is apparently to allocate all fixed disks as /dev/sd(x). I'm guessing that there are many dependencies on /dev/hda, so it is aliased to /dev/sda and retained for compatibility.

Link to comment
Share on other sites
berwhale Advanced Member

berwhale

Posted
Posted
@berwhale, do you know why parted reports sector sizes of 512 bytes physical / 512 bytes logical for the HGST NAS drives when they are Advanced Format Disks, 4096 bytes physical / 512 logical?

 

No, I don't. I ran 

hdparm -I /dev/sdX

on my system and it returned 512 bytes for logical and 4096 bytes for physical for all of the drives attached to my system.

 

I also found this interesting IBM article that goes into some depth on how 4K sectors are handled in Linux and the performance implications of misaligned partitions. The second half of the article mentions the libparted library (used by parted) and explains how to check partition alignment...

 

https://www.ibm.com/developerworks/libr ... tor-disks/

Link to comment
Share on other sites
AllGamer Super Member

AllGamer

Posted
Posted

As far as I'm concerned, if you are using the newer software, the alignment is done automatically behind the scene,

 

parted since the inception of the Advance Cluster drives came out, has defaulted to Megabyte alignment, instead of physical alignment as it used to be in the past before the Advanced drives.

 

Aligning the partition / format to the megabyte will automatically fit into the proper cluster size.

Link to comment
Share on other sites
ilciumbia Junior Member

ilciumbia

Posted
  • Author
Posted

What does parted's "align opt "n"" where n is each of the labels on the disk report? In particular does it agree that the partitions is aligned?

 

Er... it does not... ???

 

root@DiskStation:~>parted /dev/sda align opt 1

1 aligned

root@DiskStation:~>parted /dev/sda align opt 2

2 not aligned

root@DiskStation:~>parted /dev/sda align opt 5

5 not aligned

 

root@DiskStation:~>parted /dev/sdc align opt 1

1 aligned

root@DiskStation:~>parted /dev/sdc align opt 2

2 not aligned

root@DiskStation:~>parted /dev/sdc align opt 3

3 aligned

 

Ouch... I am lost again... :shock:

 

hdparm -I /dev/sda

 

reports:

 

Logical Sector size: 512 bytes

Physical Sector size: 4096 bytes

 

So my data partition on sda is in fact not aligned (not to mention the cache partition)?...

Link to comment
Share on other sites
  • 1 year later...
 Share
Go to topic listing
×
×
  • Create New...