merve04 Posted January 1, 2020 #1 Posted January 1, 2020 (edited) Has anyone ever done this? I decided to convert my volume from SHR-1 to SHR-2 as I'm planning to add more HDD's to the volume and I feel after a while the more hdd's in the array, the greater the risk of failure. its been about 4 days now, storage manager reports 13.4%, mdstat show 26.7% on the first array, but the speed has been slow.. avg in the 2000's and 3000's KBps. Im using a LSI 9211-8i in it mode and some drives direct to mobo. Mobo is gigabyte B365m-ds3h. Loader 1.04 with latest dsm. There was a time i did a repair while still in shr-1, i was hitting well over 100000KBps sometimes hitting 200000, is changing the raid type just that difficult the compute? Edited January 1, 2020 by merve04 Quote
merve04 Posted January 4, 2020 Author #2 Posted January 4, 2020 I guess I’m the first risk taker, 152hours, dsm reporting 21.5%, Mdstat shows 43% on first array of 2. Quote
flyride Posted January 4, 2020 #3 Posted January 4, 2020 (edited) It's a lot of I/O and multiple stages as I recall. You can speed it up with MDADM tuning, or in DSM 6.2.x Storage Manager | Storage Pool | Configurations | Custom and change max and min to something silly like 2500MB (25000000). Put it back to Lower Impact when done. Edited January 14, 2020 by flyride Quote
merve04 Posted January 7, 2020 Author #5 Posted January 7, 2020 I will ask this, maybe some knows.. so there is X amount of data which had 1 parity, now changing to having 2 parity.. that's all great as its reading all the files and figuring stuff about, but due to the nature of how long this takes, I'm constantly adding more and more data in the GB to TB range while this is processing.. is that data written as double parity as I'm adding these new files? or is it caching somewhere to be processed once shr2 migration is completed? Quote
bearcat Posted January 7, 2020 #6 Posted January 7, 2020 Just out of curiosity: How many drives (and what sizes of disk's and volumes) did you have in total with your SHR-1 ? How many drives (and what sizes) did you add before starting the conversion to SHR-2? Have you tried not adding extra data to your Volume(s) during the conversion, to see what happens to the speed? Quote
flyride Posted January 7, 2020 #7 Posted January 7, 2020 (edited) 18 hours ago, merve04 said: I will ask this, maybe some knows.. so there is X amount of data which had 1 parity, now changing to having 2 parity.. that's all great as its reading all the files and figuring stuff about, but due to the nature of how long this takes, I'm constantly adding more and more data in the GB to TB range while this is processing.. is that data written as double parity as I'm adding these new files? or is it caching somewhere to be processed once shr2 migration is completed? This article by the author of md might help answer your questions. http://neil.brown.name/blog/20090817000931 This excerpt should help explain why your conversion speed is so slow: Quote For a reshape that does not change the number of devices, such as changing chunksize or layout, every write will be over-writing the old layout of that same data so after a crash there will definitely be a range of blocks that we cannot know whether they are in the old layout or the new layout or a bit of both. So we need to always have a backup of the range of blocks that are currently being reshaped. This is the most complex part of the new functionality in mdadm 3.1 [snip]. mdadm monitors the reshape, setting an upper bound of how far it can progress at any time and making sure the area that it is allow to rearrange has writes disabled and has been backed-up. This means that all the data is copied twice, once to the backup and once to the new layout on the array. This clearly means that such a reshape will go very slowly. But that is the price we have to pay for safety. It is like insurance. You might hate having to pay it, but you would hate it much more if you didn't and found that you needed it. The TL;DR version: If you convert from RAID5 to RAID6 while adding a disk, conversion takes one pass. An existing array set takes two passes. Edited January 7, 2020 by flyride Quote
merve04 Posted January 8, 2020 Author #8 Posted January 8, 2020 (edited) 13 hours ago, bearcat said: Just out of curiosity: How many drives (and what sizes of disk's and volumes) did you have in total with your SHR-1 ? How many drives (and what sizes) did you add before starting the conversion to SHR-2? Have you tried not adding extra data to your Volume(s) during the conversion, to see what happens to the speed? When I was still SHR-1, i had 3x 8TB and 4x 4TB, 1 volume just over 29TB I added a new 8TB to start the processed to SHR-2, once completed will be adding another 4TB and 8TB. I've added plenty of data to my volume, maybe +500GB since I started the process of changing to SHR-2, my question was, what happens to the new data added to the volume while a migration is happening? Is the new data being written to the volume including double parity? Forgot to mention, heck ya when I add huge batches of data, it kills the speed of the resync, just checked, downloading a bunch of movies at 30-40MB\s and my resync is down in the 500kbps range, usually if traffic is light its in the high 2000 to low 3000kbps Edited January 8, 2020 by merve04 1 Quote
bearcat Posted January 8, 2020 #9 Posted January 8, 2020 OK, I see Well, as explained above by flyride, there will be "some" extra reading/writing going on, might want to put those movies on hold, untill the conversion is done, to speed things up 🗓️ Quote
merve04 Posted January 14, 2020 Author #10 Posted January 14, 2020 Well 393 hours, first array is complete, now the 2nd and last one is processing, estimated 15-17 days probably before its completed. Quote
mata7 Posted February 6, 2020 #12 Posted February 6, 2020 lol 864 hours 71% , i am shiting my pants i just start a clean 85TB SHR-2, so i guest it will take a year to finish lol Quote
flyride Posted February 6, 2020 #13 Posted February 6, 2020 A clean install only requires a single write of the array, so it will be the same as a RAID5 or RAID1 using the same size member devices. Quote
mata7 Posted February 7, 2020 #14 Posted February 7, 2020 i got a message The consistency check of storage pool 1 (SHR) on Server has ended , but then the Parity consistency restarted and start all over again, i have no parity check scheduled setup Quote
merve04 Posted February 11, 2020 Author #15 Posted February 11, 2020 (edited) It finally finished at about 1068 hours. It was an extremely slow process avg 1200-1500kbps. im now expanding my volume with couple hard drives, first one in and at roughly 25% after 8 hours. Edited February 11, 2020 by merve04 1 Quote
bearcat Posted March 18, 2020 #16 Posted March 18, 2020 @merve04 So, how is it going, are your new volumes expanded. and in use yet? Quote
merve04 Posted March 21, 2020 Author #17 Posted March 21, 2020 oh for sure they are, were always in use during the conversion. 1 Quote
bearcat Posted April 15, 2020 #18 Posted April 15, 2020 @merve04 I might have missed it, but what type of drives are you using? I came across this article today, and it made me think twice... Quote
IG-88 Posted April 15, 2020 #19 Posted April 15, 2020 (edited) its not just wd, seagate does that too, they tried to hide the recording type and when you looked closer they invented n new name (tgmr) for it and you had to dig much deep to read that its just smr with some cmr as buffer, afair it was seagate barracuda at 1st (but these are desktop drives by the spec not nas drives) BUT they also sell Iron Wolf drives (nas) with "tgmr" so they are in the same boat as wd and there are a lot nas users in the last two years with such slow drives(?) https://www.seagate.com/www-content/product-content/ironwolf/en-us/docs/100807039b.pdf edit: seagate's TGMR stands for "Tunneling Giant Magneto-Resistive" and thechnology for the read/write of the head (used in the hdd for each plate) it seems it can be used for PMR and SMR so it depends on the hdd modell and how its pre formated and its firmware (PMR only or SMR with PMR zones as cache) the key word for seagate to look for might be "MTC Technology" or "Multi-Tier Caching Technology" on the barracuda webpage this tech is explicitly named for 2.5" drives (higher density for recording then 3.5") and "normal" barracuda drives https://www.seagate.com/internal-hard-drives/hdd/barracuda/ (lower section with 5 types of barracuda compared) "Multi-Tier Caching Technology" itself includes SMR as the lowest tech/tier as shown here https://www.seagate.com/files/www-content/product-content/barracuda-fam/barracuda-new/files/multi-tier-caching-technology-white-paper-2017.pdf the product manual for the ST4000DM004 does not mention it but it IS a SMR drive so its hard to guess what drives uses SMR for a 4TB drive it might be 2 plates / 4 heads for SMR and 3 plates / 6 heads for PMR drives barracuda pro drives might not use smr but as there is no indication of how to determine it can be changed at any time by seagate the only safe way to determine seem to flood all "caches" by hammering the drive with ~20% of its capacity and have a eagle eye on the data rate, if it plummets significantly below 100MB/s for some time or stops responding for some time (flushing the pmr cache to smr) it must me a smr drive seagate actually stated that there is no SMR used in iron wolf drives its also a good idea to look into the official compatibility chart for DSM from synology (example 918+ for 4TB seagate drives) https://www.synology.com/en-global/compatibility?search_by=products&model=DS918%2B&category=hdds_no_ssd_trim&filter_brand=Seagate&filter_size=4TB&p=1 no ST4000DM004 (2plates/4heads) there but ST4000DM005 (3plates/6heads) is ok also iron wolf ST4000VN008 (3plates/6heads) is there Edited April 20, 2020 by IG-88 1 Quote
merve04 Posted April 17, 2020 Author #20 Posted April 17, 2020 On 4/15/2020 at 3:19 PM, bearcat said: @merve04 I might have missed it, but what type of drives are you using? I came across this article today, and it made me think twice... They are all Segate Barracuda's, a mix of 4 and 8 TB drives. Quote
bearcat Posted April 17, 2020 #21 Posted April 17, 2020 As mentioned above: Seems like Seagate has been shipping some Barracudas with the same "problem". (SMR) Check out your models in case they are some of those. Quote
IG-88 Posted April 18, 2020 #22 Posted April 18, 2020 On 4/17/2020 at 4:57 AM, merve04 said: They are all Segate Barracuda's, a mix of 4 and 8 TB drives. could you list the exact models you have? (see my edited post above about smr/pmr disks from seagate) Quote
merve04 Posted April 19, 2020 Author #23 Posted April 19, 2020 5x 8TB all same model, 4x 4TB are the DM000 and one 4tb being a DM004 Quote
IG-88 Posted April 19, 2020 #24 Posted April 19, 2020 https://blocksandfiles.com/2020/04/15/seagate-2-4-and-8tb-barracuda-and-desktop-hdd-smr/ SMR: Barracuda 4TB – 5,400rpm – SATA 6gig – ST4000DM004 Barracuda 8TB – 5,400rpm – SATA 6gig – ST8000DM004 so that#s the reason for your extremely poor performace you are using SMR disks (all the 8TB disks and one 4TB disk) 1 Quote
merve04 Posted April 21, 2020 Author #25 Posted April 21, 2020 Well considering I had never done it, therefore have nothing to compare against it. When shr2 conversion was done, I expanded the volume twice with a 4 and 8 TB drives. It was a little bit slower than when I was on SHR1 but not a huge deal. Read write speeds on normal operations still maxes out based on saturation of my gigabit connection. Quote
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.