Xpenology and SAS Controller - HBA

[vasiliy_gr]

And one more possible reason for problems... It is SAS itself.

 

I mean the order of drive letters for sas in linux. As for the lsi 9211 with the latest IT firmware it does not correlate to hdd's positions in enclosures or to number-marks on sas cables. I suppose that it may be correlated to the sequence of adding drives to enclosure (for the first time for every hdd).

 

So there may be strange looking misbehave if drives are ordered in non-expecting order.

[akaz]

I am not exactly sure... But during several experiments with upgrades of DSM version (and so with resulting bad *portcfg values) I issued two types of problems in different hardware configurations (including those with up to 15 hdd-s in one array).

 

The worse one - 1 hdd from array is lost. Array is degraded. After changing internalportcfg to correct value the lost hdd becomes visible, but the only way to restore array to fully functional state is a full rebuild.

 

If there are two or more hdd-s that are invisible, array is non-functional. But after changing internalportcfg to correct value and reboot everything becomes fine (without need of rebuild).

 

In both situations system and swap partitions are degraded after changing internalportcfg value. But it is really not much of a problem.

 

Anyway I can't guarantee that it is definitely so in all the situations... And also there can be a problem of full data lost while migrating between different types of boot images (or/and different DSM versions).

 

I think when 2 disks are missing the raid is stopped.. so no changes are made.. where as 1 drive missing would just look like a drive failure so the raid carries on working.

 

so when you add the 2 disks back the raid is automatically turned back on?? or am i wrong,.

[snoopy78]

hello,

 

i use following settings...

 

1.) improve raid rebuild time (viewtopic.php?f=2&t=2079&p=10371&hilit=sysctl.conf#p10371)

sysctl.conf

sysctl -w dev.raid.speed_limit_min=300000

sysctl -w dev.raid.speed_limit_min=600000

 

2.) improve # disks

esataportcfg="0x000"

usbportcfg="0x05"

internalportcfg="0xffffff"

 

3.) additionally since last night i use my 2port 10G NIC as routing between both ports (my 10G switch only has 2x 10G sfp+ so i route my HP-Xpenology test/backup system via 10G over my SC836 chassis)

rc.local

echo 1 > /proc/sys/net/ipv4/conf/all/forwarding

 

i have this files prepared at my 1st volume (ssd single drive) so i copy them right after install to the right places

BR

snoopy78

[stanza]

@ snoopy78

 

you could try this for your item 3

 

http://xpenology.com/forum/viewtopic.php?f=2&t=2130

[snoopy78]

@ stanza

thx for your tip, next time when i play around i'll test

what you're suggesting is more like nat-pat... am i right?

(vpns, routing and all tunnel handling is beeing done by my dsr)

 

here is a part of my network

https://www.dropbox.com/s/tz2qnmo6a648y ... .48.25.jpg

 

the red connection are 10G

 

as you can see i do have a "real" vlan-routing capable router and i can redirect packets corretly

(actually i do have a little "dirty" network, as the iLO also provides LAN link in XPEnology, routing points to 10G @ SC836)

 

please correct me when i'm wrong

 

p.s. this is the atto bench to a single hdd in my hp-system, routed via sc836 and bypassed GW with direct route from client

Edited February 28, 2014 by Guest

[vasiliy_gr]

the red connection are 10G

Hi, snoopy78!

 

May be you can tell the full story - what is needed to build 10G network? I'm absolutely nube in 10G... So everything - controllers, routers, cables - is not clear at all.

 

I do understand that it looks like a little bit off-topic question. But with xpenology and especially sas-based configurations we see real throughoutput up to 10G... So I believe it may be interesting not only for me. May be it shall be a special topic for 10G network hardware environment for xpenology.

 

If you can share your knowledge (or if anyone else can help) I will be thankful much.

[snoopy78]

the red connection are 10G

Hi, snoopy78!

 

May be you can tell the full story - what is needed to build 10G network? I'm absolutely nube in 10G... So everything - controllers, routers, cables - is not clear at all.

 

I do understand that it looks like a little bit off-topic question. But with xpenology and especially sas-based configurations we see real throughoutput up to 10G... So I believe it may be interesting not only for me. May be it shall be a special topic for 10G network hardware environment for xpenology.

 

If you can share your knowledge (or if anyone else can help) I will be thankful much.

 

ok, let's say like that... i'm working in enterprise IT industry as an network-tech guy so naturally i'm interested in what's possible and how to achieve this

also i try to learn something by myself with such an setup and test some of our newest products in this way

 

so at first you need time, money and a wife/family who will accept your job/hobby

secondly you need to have an idea what you want to do.... and you must be willing to spent time and money on it

10G is not cheap, it's affordable.. but still...it cost's money

 

there are many way's to get 10G (40G) up and running (like Infiniband, FCoE or 10 GBe) and i ONLY focus on 10G Ethernet as this also covers my job

 

so what you 1st need is a Client and Storage which can actually handle such speed (even my setup can't fully make use of this),

this means you need

- fast HDD's / SSD's

- fast HBA for SATA or BETTER SAS drives (for me sas drives are way to expensive and i'm satisfied with my reachable speed)

- fast cpu

 

then it comes to the networking part

currently most of the 10G NICs are using SFP+ or DAC (directly attaced SFP+ cable), new intel ones now switch to Cat7 RJ45, but afaik the latency is higher then SFP+/DAC

so now you have to think what cables and distances you need.. usually at home MMF is enough, even DAC (1m, 2m, 5m, 10m) should be fine

SO let's say you buy the brand new DGS-1510-20 (which offers 2 10G SFP+) ports then you already spend for this about 200-250 €

then you need a 10G nic, here is ebay your best friend.. i was luck to obtain some qlogic nics for about 100€ each nic (so it makes again totally ~ 200 €)

then you need a DAC or SFP+.. for a 10G MMF (850nm) SFP+ you should pay around 40 € and the LWL Cable ~ 10 - 15€ (so it makes another 180-200 €)

if you use DAC then you can be lucky at ebay and get some for around 30-50 € eacht (so totally 100 €)

 

so now you can around calculate the prices...

 

BUT what's most important...

do you have the data for the speed?

10G usually makes sense if you need to transfer laaaarge amount of files fast... but do you need it at home?

 

hope this helps for now... i need now to catch my little girl..

[vasiliy_gr]

snoopy78, many thanks for quick reply!

 

You gave quite enough keywords to start search. I really understood now half of the text (about wife, money and if I really need 10G...). But really its total price does not look fencing. But I'm afraid that here, in Russia, it may be 10ths times higher... But you are right - we have ebay and also our chinese friends with their online shops.

 

As for necessity of 10G at home (yes - you are right, I'm building at-home nas environment)... You see, after several experiments I managed to build three nas-es insteed of one needed. And two of these three has internal transfer rate higher than 500 MB/s. So i'm sad now for not utilizing this high speed gained by accident. So I need 10G network between of this zoo of nas-es and my workstation.

 

So snoopy78 - thank you for interesting info and keywords to work on (I'll copy your message locally and will work on it item-by-item). Everyone else - sorry for offtopic.

[jimpa]

OK

 

with 7 drives on H200 only

 

upgrade from 3810 beta5 to 3827 beta7 works fine (using migration)

 

all users / settings / files / programs are still there

 

I guess because

 

3810 beta5 defaults are within 3827 beta7 defaults

 

esataportcfg="0x1000" beta5

esataportcfg="0xff000" <<

usbportcfg="0xffe000"beta5

usbportconfig="0xf00000" <<

internalportcfg="0xfff" beta5

internalportcfg="0xfff" <<

 

So if you have under 12 drives / ports in TOTAL in use.... you might be good for the upgrade.

If you have OVER 12 Drives.... Backup and restore or take the risk and edit synoinfo.conf file ASAP after upgrading as currently it will wipe out your synoinfo.conf changes

 

.

 

I have the tip after several month of making a big system with lots of disks SHR-2.

This is the best parameters added to the rebuild speed parameters.

0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 esataportcfg=0x00 (0 ESATA ports)

0111 1111 1111 0000 0000 0000 0000 0000 0000 0000 0000 usbportcfg=0x7FF00000000 (11 USB ports)

0000 0000 0000 0111 1111 1111 1111 1111 1111 1111 1111 internalportcfg=0x7FFFFFFF (31 Internal ports)

 

It works fine to reboot also several times so far with 4.3 3827 Beta7.pat

Also maxdisks=20 is needed

Important to edit through putty both the synoinfo.conf files then directly type reboot in console and first after that re-login through the webpage.

Else the synoinfo.conf file resets to default again and again and again....etc. Really frustrating.

Sometimes even get need to migrate system to reach it. But no harm found on files or any settings...

So make the synoinfo.conf parameters as high as possible please.

Hope to soon see a new build with these settings by default ...perhaps I can also share info how to come around the SSD problem once mounted them faulty to an existing raid )

I testing this right now but all tests take Days.

Cu soon

/Jimmy

[stanza]

What I would like to know is

 

On the REAL DS3612xs which is 12 disk bays

We know the default synoinfo.conf file settings

 

So

 

What happens / how does it change automatically when you plug in the optional DX1211 . Which is anther 12 bays

 

We need to find someone with the above, check the files.... Then plug in the expansion and check the files again

 

Tell us what moves/ changes

 

.

[stanza]

the red connection are 10G

Hi, snoopy78!

 

May be you can tell the full story - what is needed to build 10G network? I'm absolutely nube in 10G... So everything - controllers, routers, cables - is not clear at all.

 

I do understand that it looks like a little bit off-topic question. But with xpenology and especially sas-based configurations we see real throughoutput up to 10G... So I believe it may be interesting not only for me. May be it shall be a special topic for 10G network hardware environment for xpenology.

 

If you can share your knowledge (or if anyone else can help) I will be thankful much.

 

You can also try out bonding several 1g LAN ports together to bump up your speed between Xpenologies and or say virtualization servers.

 

I will write up a new thread of how to do it if you like.

 

.

[snoopy78]

new threat would be fine as others may want to know too...

 

Ok...bonding...afaik syno/xpenology uses 802.3ad for bonding...so you need an 802.3ad capable switch for that.

 

so the IEEE standard 802.3ad (lacp) is session based.

This means there is an algorithm which is balancing the sessions onto the bond's physical nics.

So for each client the max. speed is still 1 GBit. Bonding only provides fault tolerance and more capacity if there are sevetal clients accessing the "server"...so for each connection no speed gain...in my net plan you can see a ds-1812+ which is connected by a 802.3ad bond...and still always only 1gbit for each session...

 

for virtualization..you need 802.3ad bonding and on top of that 802.1Q but still...yout v-switch must be capable then.... Lacp and vlan together with the correct balancing algoriththm.....

Ok...vlans you need only if your virtualized machines needs to be separated from each other..

[stanza]

or you can edit some files (not supported of course) and enable RR mode to increase thruput

 

eg 4 x 1Gb ports to acheive rougly 450Mb/s

 

ok I will start a new thread for this

 

.

[vasiliy_gr]

or you can edit some files (not supported of course) and enable RR mode to increase thruput

 

eg 4 x 1Gb ports to acheive rougly 450Mb/s

 

ok I will start a new thread for this

 

.

Thank you, stanza, and again thank you, snoopy78.

 

A new thread is a good idea. As for bonding and LACP - I have done it already. I'm using a pair of Netgear GS108T for it. And two intel nic-s on nas and on the main workstation. And really it is not much as fault-tolerance mode. It only helps in situation of massive backup not to stop other network activity on nas.

 

As for RR mode I tried to use it (between two linux-es) several years ago with no luck. Network speed was not doubled. But may be I configured something wrongly.

[stanza]

Guide is up

 

http://xpenology.com/forum/viewtopic.php?f=2&t=2476

 

.

[jimpa]

I got a problem that I cannot expand my volume capacity. Even after adding 3 more disks and tried expand again.

Only get back to this page all the time. Any hints?

[vasiliy_gr]

I got a problem that I cannot expand my volume capacity. Even after adding 3 more disks and tried expand again.

Only get back to this page all the time. Any hints?

I had similar problem with expanding volume of 7 4TB hdd-s in SHR with the 8th one. After much googling I decided that it is a problem of current DSM version (as it was also reported on original synology forums). Especially the version of e2fsprogs package with resize2fs in it which does not support FS resize above 16TB. It can make volume above 16TB, but not resize above this value. Here is its version info:

XPEnology> resize2fs
resize2fs 1.42.6 (21-Sep-2012)
Usage: resize2fs [-d debug_flags] [-f] [-F] [-M] [-P] [-p] device [new_size]

It is for DSM 4.3, latest releases. Did not checked om DSM 5.0 (as it happened 3 months ago).

 

So... What shall be done?.. I nearly cracked my head thinking how to install newer version of e2fsprogs on xpenology (as I am linux user but not a system-level linux programmer). Also I was not absolutely sure that the problem is exactly with this. But at last I founf a rather simple solution.

 

I looked on my desktop linux (suse 13.1):

vasil@vgserver:~> resize2fs
resize2fs 1.42.9 (28-Dec-2013)
Usage: resize2fs [-d debug_flags] [-f] [-F] [-M] [-P] [-p] device [new_size]

This version should have no problems with 16TB boundary. (3 months ago real version was a little lower but I give the one I have right now with updates.)

 

So... I connected an unused ssd to my xpenology and installed minimal suse system from dvd. Then from suse cli I found my SHR to be sure that everything ok (also I looked on it through yast). And the last action looked something like this:

resize2fs -fpF /dev/vg1000/lv

I do not remember now if I changed something in this line (it was 3 months ago) but in my offline notes for this issue I have this string.

 

It worked several minutes. After completing I checked new volume size - it was really expanded. So i disconnected suse's ssd, reverted boot order to usb-flash, restarted DSM and - mission accomlished. It took less than an hour for everything - connection, installation, operation, disconnection, celebration...

 

Certainly you may use not suse, but any linux distro with suitable version of e2fsprogs. Or you may try to compile it youself (if you know how you can do it). Also you should realize that my solution is for a problem when your SHR is already expanded but ext4 above it is not. If you have another problem - this solution may be not suitable!!!

[gnoboot]

hmmm, let me create a new e2fsprogs spk:)

[jimpa]

hmmm, let me create a new e2fsprogs spk:)

 

 

Yes please I am a windows user so I need help here to get a package I can apply to 4.3 3811 beta5.pat or beta7.pat installation.

THANKS vasiliy_gr for a fantastic explanation of actually you having exact the same problem.

I wait now for someone to help me with a package of code I can apply and in detial desribe how to fix this.

I do now even know the linux commands

 

Thanks again

/Jimpa

[jimpa]

What is the easiest way to try to expand the ext4 volume like the problem described above with DSM 5?

Today I have 4.3 3827 beta7.

First I guess I need to get my hands on DSM5 and update ..can't even find the beta 5.

Will all settings and data stil be intact?

I can not loose the data.

 

I need to try and see if the resize2fs program is updated and working with bigger volumes.

 

/Jimpa

[gnoboot]

I got a problem that I cannot expand my volume capacity. Even after adding 3 more disks and tried expand again.

Only get back to this page all the time. Any hints?

I had similar problem with expanding volume of 7 4TB hdd-s in SHR with the 8th one. After much googling I decided that it is a problem of current DSM version (as it was also reported on original synology forums). Especially the version of e2fsprogs package with resize2fs in it which does not support FS resize above 16TB. It can make volume above 16TB, but not resize above this value. Here is its version info:

XPEnology> resize2fs
resize2fs 1.42.6 (21-Sep-2012)
Usage: resize2fs [-d debug_flags] [-f] [-F] [-M] [-P] [-p] device [new_size]

It is for DSM 4.3, latest releases. Did not checked om DSM 5.0 (as it happened 3 months ago).

 

This version already supports online resize above 16TB. I think there's no need to update it. But a patch is needed to enable this feature in ext4 filesystem.

[jimpa]

This version already supports online resize above 16TB. I think there's no need to update it. But a patch is needed to enable this feature in ext4 filesystem.

 

Ok so how do I fix my expanding proble. due to that I am not a linux person

Patch...mm how can someone help me out here?

Do I need to just apply beta5 and cross my fingers that all data will be intact ?

[gnoboot]

Upgrading to 5.0 will not also work.

[jimpa]

What do you mean?

 

Does it work with DSM 5.0 to expand EXT4 over 16tb or not?

Your reply was really obscure.

Waiting Before doing the big install.

[gnoboot]

My build and trantor's is still using the same dsgpl 4.3 kernel . So even if you install 5.0 it will never work because kernel doesn't support online resizing. You need at least kernel 3.3 and above to make it work.

gnoboot> cat /etc/VERSION
majorversion="5"
minorversion="0"
buildphase="beta"
buildnumber="4418"
smallfixnumber="0"
builddate="2014/01/20"
gnoboot> resize2fs
resize2fs 1.42.6 (21-Sep-2012)
Usage: resize2fs [-d debug_flags] [-f] [-F] [-M] [-P] [-p] device [new_size]