mgrobins

Write Cache Not Functioning - not good

Recommended Posts

Hi,

 

I have noticed with my xpenology build that I cannot enable Disk Write Cache which is a pretty major problem for maintaining optimum speed and reducing HDD strain.

 

I am using a mixture of onboard SATA ports for my Supermicro motherboard (Xeon-D) and an LSWI 9211-8i HBA flashed in IT mode.

HDD - 6x Seagate 4TB Ironwolf NAS, 1x HGST 4TB NAS.

 

DSM version 6.1.1.15101 Update 2.

 

 

 

Anyone able to shed light on why this is an issue and how to work around it?

 

When I attempt to enable it I get a red text message below stating that "The Operation failed - please log into DSM and try again".

Share this post


Link to post
Share on other sites

I have the same on DSM 5.2-5644 Update 3. This happens because of LSWI 9211-8i. I have the LSI SAS 9207-8i. To check your write cache - ssh to your box and run:

hdparm -I /dev/sdX

Commands/features:
       Enabled Supported:
          *    SMART feature set
               Security Mode feature set
          *    Power Management feature set
          *    Write cache
          *    Look-ahead

 

Also you can check write cache via dmesg:

dmesg >/volume1/share_name/dmesg.txt

[    2.481547] sd 0:0:4:0: [sde] Write cache: enabled, read cache: enabled, supports DPO and FUA
[    2.484634] PM: Checking hibernation image partition /dev/md1
[    2.485628] sd 0:0:6:0: [sdg] Write cache: enabled, read cache: enabled, supports DPO and FUA
[    2.493347] sd 0:0:3:0: [sdd] Write Protect is off
[    2.493353] sd 0:0:3:0: [sdd] Mode Sense: 7f 00 10 08
[    2.493944] sd 0:0:0:0: [sda] Write Protect is off
[    2.493946] sd 0:0:0:0: [sda] Mode Sense: 7f 00 10 08
[    2.494739] sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, supports DPO and FUA
[    2.495497] sd 0:0:3:0: [sdd] Write cache: enabled, read cache: enabled, supports DPO and FUA

better to open dmesg.txt on Notepad++ or something similar.

 

My write speed to RAID-5 of 4x1Tb is 320Mb/s

Share this post


Link to post
Share on other sites

Thank you so much for the advice and direction :smile:.

 

Just a quick follow up. I could not get the first command to work - said it did not exist so I just did the dmesg.

 

From the text I have a couple of queries if you are able to help.... I'm not much of a Linux person I'm afraid though I do know PC design quite well.

 

 

what is this conflict? I think this is the chipset on my motherboard (model is in my sig). I am using the DS3617xs boot loader which is for a Xeon-D so I assumed it would be very similar.

 

[   13.640583] ACPI Warning: 0x0000000000000428-0x000000000000042F SystemIO conflicts with Region \GPE0 1 (20130328/utaddress-249)
[   13.640626] ACPI: If an ACPI driver is available for this device, you should use it instead of the native driver
[   13.640636] ACPI Warning: 0x0000000000000500-0x000000000000052F SystemIO conflicts with Region \GPIV 1 (20130328/utaddress-249)
[   13.640670] ACPI: If an ACPI driver is available for this device, you should use it instead of the native driver
[   13.641949] lpc_ich: Resource conflict(s) found affecting gpio_ich
[   13.654651] Loading iSCSI transport class v2.0-870.

 

 

Does the text indicate write cache is actually enabled or that the Linux kernel enables it and then DSM has disabled it later following boot or some such (which I heard was the case). Aside from throughput improvement it does reduce disk thrashing a lot so I think it's worth pursuing if it is not working (even if I need to get a different raid card perhaps).

 

From the section on write cache etc:

 

[   14.101669] ata1.00: ATA-9: HGST HDN726040ALE614, APGNW7JH, max UDMA/133
[   14.101736] ata1.00: 7814037168 sectors, multi 16: LBA48 NCQ (depth 31/32), AA
[   14.106561] ata3.00: ATA-10: ST4000VN008-2DR166, SC60, max UDMA/133
[   14.106592] ata3.00: 7814037168 sectors, multi 16: LBA48 NCQ (depth 31/32), AA
[   14.106721] ata1.00: configured for UDMA/133
[   14.107809] scsi 0:0:0:0: Direct-Access     HGST     HDN726040ALE614          APGN PQ: 0 ANSI: 5
[   14.107925] ata3.00: configured for UDMA/133
[   14.108090] ata4.00: ATA-10: ST4000VN008-2DR166, SC60, max UDMA/133
[   14.108096] ata4.00: 7814037168 sectors, multi 16: LBA48 NCQ (depth 31/32), AA
[   14.108152] ata6.00: ATA-10: ST4000VN008-2DR166, SC60, max UDMA/133
[   14.108183] ata6.00: 7814037168 sectors, multi 16: LBA48 NCQ (depth 31/32), AA
[   14.109225] ata4.00: configured for UDMA/133
[   14.109304] ata6.00: configured for UDMA/133
[   14.109605] sd 0:0:0:0: [sda] 7814037168 512-byte logical blocks: (4.00 TB/3.63 TiB)
[   14.109614] sd 0:0:0:0: [sda] 4096-byte physical blocks
[   14.109826] sd 0:0:0:0: [sda] Write Protect is off
[   14.109858] sd 0:0:0:0: [sda] Mode Sense: 00 3a 00 00
[   14.109945] sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[   14.116396] ata2.00: ATA-10: ST4000VN008-2DR166, SC60, max UDMA/133
[   14.116430] ata2.00: 7814037168 sectors, multi 16: LBA48 NCQ (depth 31/32), AA
[   14.117590] ata2.00: configured for UDMA/133
[   14.118852] scsi 1:0:0:0: Direct-Access     ATA      ST4000VN008-2DR166       SC60 PQ: 0 ANSI: 5
[   14.119706] ata5.00: ATA-10: ST4000VN008-2DR166, SC60, max UDMA/133
[   14.119739] ata5.00: 7814037168 sectors, multi 16: LBA48 NCQ (depth 31/32), AA
[   14.123230] ata5.00: configured for UDMA/133
[   14.125230] sd 1:0:0:0: [sdb] 7814037168 512-byte logical blocks: (4.00 TB/3.63 TiB)
[   14.125239] sd 1:0:0:0: [sdb] 4096-byte physical blocks
[   14.125441] sd 1:0:0:0: [sdb] Write Protect is off
[   14.125448] sd 1:0:0:0: [sdb] Mode Sense: 00 3a 00 00
[   14.125532] sd 1:0:0:0: [sdb] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[   14.158200]  sda: sda1 sda2 sda3
[   14.161110] sd 0:0:0:0: [sda] Attached SCSI disk
[   14.182204]  sdb: sdb1 sdb2 sdb3
[   14.195577] sd 1:0:0:0: [sdb] Attached SCSI disk
[   14.325330] scsi 2:0:0:0: Direct-Access     ATA      ST4000VN008-2DR166       SC60 PQ: 0 ANSI: 5
[   14.327096] sd 2:0:0:0: [sdc] 7814037168 512-byte logical blocks: (4.00 TB/3.63 TiB)
[   14.327106] sd 2:0:0:0: [sdc] 4096-byte physical blocks
[   14.327314] sd 2:0:0:0: [sdc] Write Protect is off
[   14.327345] sd 2:0:0:0: [sdc] Mode Sense: 00 3a 00 00
[   14.327432] sd 2:0:0:0: [sdc] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[   14.381884]  sdc: sdc1 sdc2 sdc3
[   14.384866] sd 2:0:0:0: [sdc] Attached SCSI disk
[   14.455142] scsi 3:0:0:0: Direct-Access     ATA      ST4000VN008-2DR166       SC60 PQ: 0 ANSI: 5
[   14.457016] sd 3:0:0:0: [sdd] 7814037168 512-byte logical blocks: (4.00 TB/3.63 TiB)
[   14.457024] sd 3:0:0:0: [sdd] 4096-byte physical blocks
[   14.457224] sd 3:0:0:0: [sdd] Write Protect is off
[   14.457232] sd 3:0:0:0: [sdd] Mode Sense: 00 3a 00 00
[   14.457445] sd 3:0:0:0: [sdd] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[   14.477877] scsi 4:0:0:0: Direct-Access     ATA      ST4000VN008-2DR166       SC60 PQ: 0 ANSI: 5
[   14.479456] sd 4:0:0:0: [sde] 7814037168 512-byte logical blocks: (4.00 TB/3.63 TiB)
[   14.479490] sd 4:0:0:0: [sde] 4096-byte physical blocks
[   14.479671] sd 4:0:0:0: [sde] Write Protect is off
[   14.479702] sd 4:0:0:0: [sde] Mode Sense: 00 3a 00 00
[   14.479788] sd 4:0:0:0: [sde] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[   14.496526] scsi 5:0:0:0: Direct-Access     ATA      ST4000VN008-2DR166       SC60 PQ: 0 ANSI: 5
[   14.499313] sd 5:0:0:0: [sdf] 7814037168 512-byte logical blocks: (4.00 TB/3.63 TiB)
[   14.499346] sd 5:0:0:0: [sdf] 4096-byte physical blocks
[   14.499722] sd 5:0:0:0: [sdf] Write Protect is off
[   14.499730] sd 5:0:0:0: [sdf] Mode Sense: 00 3a 00 00
[   14.499821] sd 5:0:0:0: [sdf] Write cache: enabled, read cache: enabled, doesn't suppo

 

I have 6 drives on onbvoard sata and one on my SAS HBI currently.... is this one below the SAS connected drive?

If so what are DPO and FUA as it supports them but the onboard does not....

 

 

[   18.613845] scsi 6:0:0:0: Direct-Access     ATA      ST4000VN008-2DR166       SC60 PQ: 0 ANSI: 6
[   18.618271] scsi 6:0:0:0: SATA: handle(0x0009), sas_addr(0x4433221103000000), phy(3), device_name(0x0000000000000000)
[   18.618306] scsi 6:0:0:0: SATA: enclosure_logical_id(0x500605b002c8adc5), slot(0)
[   18.618423] scsi 6:0:0:0: atapi(n), ncq(y), asyn_notify(n), smart(y), fua(y), sw_preserve(y)
[   18.622592] scsi 6:0:0:0: serial_number(            ZDH16R1H)
[   18.622627] scsi 6:0:0:0: qdepth(32), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1)
[   18.632140] sd 6:0:0:0: [sdg] 7814037168 512-byte logical blocks: (4.00 TB/3.63 TiB)
[   18.632151] sd 6:0:0:0: [sdg] 4096-byte physical blocks
[   18.939881] sd 6:0:0:0: [sdg] Write Protect is off
[   18.939890] sd 6:0:0:0: [sdg] Mode Sense: 7f 00 10 08
[   18.941917] sd 6:0:0:0: [sdg] Write cache: enabled, read cache: enabled, supports DPO and FUA

 

 

 

Some other errors or things I don't understand that may be explainable by others here :smile:

:

[   24.440607] broadwell_synobios: module license 'Synology Inc.' taints kernel.
[   24.440637] Disabling lock debugging due to kernel taint

 

[   24.522223] md: Autodetecting RAID arrays.
[   24.592601] md: invalid raid superblock magic on sda3
[   24.597744] md: sda3 does not have a valid v0.90 superblock, not importing!
[   24.677080] md: invalid raid superblock magic on sdb3
[   24.682189] md: sdb3 does not have a valid v0.90 superblock, not importing!
[   24.756138] md: invalid raid superblock magic on sdc3
[   24.761250] md: sdc3 does not have a valid v0.90 superblock, not importing!
[   24.821471] md: invalid raid superblock magic on sdd3
[   24.826583] md: sdd3 does not have a valid v0.90 superblock, not importing!
[   24.888822] md: invalid raid superblock magic on sde3
[   24.893907] md: sde3 does not have a valid v0.90 superblock, not importing!
[   24.980498] md: invalid raid superblock magic on sdf3
[   24.985586] md: sdf3 does not have a valid v0.90 superblock, not importing!
[   25.072161] md: invalid raid superblock magic on sdg3
[   25.077275] md: sdg3 does not have a valid v0.90 superblock, not importing!
[   25.077290] md: Scanned 21 and added 14 devices.

Share this post


Link to post
Share on other sites

hdparm -I /dev/sdX

Means that you shoud replace X to your device name.

hdparm -I /dev/sda
hdparm -I /dev/sdb
hdparm -I /dev/sdc

and so on

[   13.640583] ACPI Warning:

I have almost the same warning/error and my box works over 2 year without any problems. It can be a problem in BIOS firmware itself.

 

yes, you have write cache enabled on all drives:

[   14.109945] sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA

I did not find anything about DPO or FUA and I see the same on my LSI card(supports DPO and FUA). It can be related to LSI or/and SAS cards.

 

to check your speeds you can use:

hdparm -tT /dev/md3
/dev/md3:
Timing cached reads:   2366 MB in  2.00 seconds = 1183.36 MB/sec
Timing buffered disk reads: 954 MB in  3.00 seconds = 317.72 MB/sec

and write speed:

with write cache enabled:

time dd if=/dev/zero of=/volume2/RAID5/test.file bs=1M count=1000
1000+0 records in
1000+0 records out
real    0m 6.38s
user    0m 0.00s
sys     0m 2.97s

with write cache disabled:

time dd if=/dev/zero of=/volume2/RAID5/test.file bs=1M count=1000 conv=fsync
1000+0 records in
1000+0 records out
real    0m 16.47s
user    0m 0.00s
sys     0m 3.62s

then 1000Mb(1000 counts per 1Mb)/6.38 (second)= 157.23 MB/s

and only 60.71Mb/s without write cache.

 

Aside from throughput improvement it does reduce disk thrashing a lot

no, write cache in Linux works not the same way like in Windows.

 

sda3 does not have a valid v0.90 superblock, not importing!

I have the same. It is something connected to mdadm RAID autodetecting.

Share this post


Link to post
Share on other sites

I used hdparm to view device info ..... where or how would I enter the info to replace /dev/sdX with a-H for my drives and what if any difference would it make?

 

 

hdparm -Tt /dev/md1

/dev/md1:
Timing cached reads:   18298 MB in  2.00 seconds = 9158.76 MB/sec
Timing buffered disk reads: 632 MB in  3.01 seconds = 209.96 MB/sec

 

That's 7 disks in RAID6 with 8GB RAM and a Zeon-D proc.

 

time dd if=/dev/zero of=/volume1/test/test.file bs=1M count=1000

 

Can I just use any file for this? I just used a 25GB mkv file

 

 time dd if=/dev/zero of=/volume1/test/test.mkv bs=1M count=1000
1000+0 records in
1000+0 records out
1048576000 bytes (1.0 GB) copied, 0.486166 s, 2.2 GB/s

real    0m0.489s
user    0m0.000s
sys     0m0.489s

 

 

I ran with a count of 10,000 instead as I thought this might overcome any RAM caching and give me a proper result.

 

 time dd if=/dev/zero of=/volume1/test/test.mkv bs=1M count=10000
10000+0 records in
10000+0 records out
10485760000 bytes (10 GB) copied, 14.5246 s, 722 MB/s

real    0m15.674s
user    0m0.012s
sys     0m5.167s

 

I think the write caching is active. hdparm suggests it is at least at the drive level. My figures look a bit different to yours.... not sure if they are good or bad! I have one HGST 4TB and the rest are Seagate Ironwolf 4TB NAS drvies.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now