• Announcements

    • Polanskiman

      DSM 6.2-23739 - WARNING   05/23/2018

      This is a MAJOR update of DSM. DO NOT UPDATE TO DSM 6.2 with Jun's loader 1.02b or earlier. Your box will be bricked.  You have been warned.   https://www.synology.com/en-global/releaseNote/DS3615xs
    • Polanskiman

      Email server issues - FIXED   06/14/2018

      We have been experiencing email server issues. We are working on it. Bear with us. Thank you. ------------- Problem has been fixed. You might receive duplicate emails. Sorry for that.
test4321

10Gbe setup - will this work with 6.1?

Recommended Posts

Lat night I changed the test for proof of concept  (Ubuntu 16.04 includes the Kernel Sources in /usr/src) and it works. With the the newer single port card and the scripted Mellanox OFED install. I configured the connetcx3 as root in the terminal. This document was great.

 

https://community.mellanox.com/docs/DOC-2431

 

I'm changing back to connectx2 (dual port) and DS3617 today.

 

 

 

 

Share this post


Link to post
Share on other sites
3 hours ago, RacerX said:

Lat night I changed the test for proof of concept  (Ubuntu 16.04 includes the Kernel Sources in /usr/src) and it works. With the the newer single port card and the scripted Mellanox OFED install. I configured the connetcx3 as root in the terminal. This document was great.

 

in what way does this help with the dsm/xpenology beside knowing the card does work?

the driver you have on dsm (form synology) does support the card in question and found the card when the driver was loaded, i'd say that rules out driver problems like the plain mlx4*.ko file for the hardware

Share this post


Link to post
Share on other sites
Posted (edited)

I just bought the connectx3 card last week so I wanted to make sure it works (it does with my limited test).  I know more about the connect2 cards since I've had them for a long time.  I tried changing grub to 3 macs. but now i get error 13 every time I try to install it.

 

Thanks

Edited by RacerX
spelling

Share this post


Link to post
Share on other sites

does the vid/pid still matches the usb device?

how does grub.cfg look now (did you insert numbers for the mac's)

Share this post


Link to post
Share on other sites

I changed the macs last number sequence to be 1, then 2, then 3 and saved as text but for some reason error 13 over and over ...yuk

Share this post


Link to post
Share on other sites

the revert the change and test if error 13 disappears (should not depend on the nic changes)

Share this post


Link to post
Share on other sites

try installing dsm using the 'default' mac for a single nic, and once you are 'up' then edit grub for the the correct addresses and quantity  

Share this post


Link to post
Share on other sites

Thanks for the help.  I have to return the hardware that I had this weekend to test.

 

Share this post


Link to post
Share on other sites
Posted (edited)

Got all the devices in today. 

 

Connected the cards.

 

Having a couple of issues:

 

1) Windows 7 Driver - non-existent. I have downloaded these two:

 

http://driverdownloads.qlogic.com/QLogicDriverDownloads_UI/SearchByProduct.aspx?ProductCategory=322&Product=1214&Os=194

 

-  Driver package for Windows Server 2012 R2 (x64)

- Adapter Software Installer for Windows

 

Installation of the device  through Device Manager fails too:

 

9umpw84WkcSLQk46GCKFec5BA9S1xwBCXedNc6Gc

 

^^^ these adapters installed but with an error: 

 

L9XuQLL73fH6W_C_eNdNoU7aT03y-MY39po0WP20

 

Code 39 (havent looked into it yet)

 

 

Fibre Channel Controller

 

The bigger problem is the Fibre Channel Controller - I cant install the drivers without briking Win7 - complete bluescreen. Only restore point gets the system back to normal.

 

RMV8zyjY6xojVRSmJyGSEjU1tsRKVpEoUkdlgn4t

 

Any ideas on how to get this device to work on WIN 7 PRO?

 

2) The NIC itself loads in before boot devices.


I get an error when the screen is loading:

 

Adapter 1/1/1 (Brocade-1020): Link initialization failed. Disabling BIOS

Adapter 1/0/0 (Brocade-1020): Link initialization failed. Disabling BIOS

 

I enabled both of the NICs through options / settings:

 

xCF1jeQuP8c5wmerbU-1ds78gS4N-goEn-1JV2c6

 

I'm not sure what I'm supposed to enable here. Any ideas?

 

Thanks!

 

 

Edited by test4321

Share this post


Link to post
Share on other sites
19 hours ago, test4321 said:

2) The NIC itself loads in before boot devices.


I get an error when the screen is loading:

 

Adapter 1/1/1 (Brocade-1020): Link initialization failed. Disabling BIOS

Adapter 1/0/0 (Brocade-1020): Link initialization failed. Disabling BIOS

 

I enabled both of the NICs through options / settings:

 

I'm not sure what I'm supposed to enable here. Any ideas?

 

Thanks!

 

I think I figured out issue #2:

 

This is some kind of boot over LAN setting which I dont need to use. I'll turn it off. 

 

So I guess the only thing is the WIN7 issue

Share this post


Link to post
Share on other sites
Posted (edited)
On 3/28/2018 at 12:32 PM, test4321 said:

Fibre Channel Controller

???

 

i'd assume you use a DAC cable or 10GSR SFP+ module for 10G ethernet

FC is different and not used with dsm

 

also if you cant find a win7 driver then win server 2008r2 would be the better choice (same code base as win7)

http://driverdownloads.qlogic.com/QLogicDriverDownloads_UI/SearchByProduct.aspx?ProductCategory=322&Product=1214&Os=190

 

win7 seems to be supported

https://forums.servethehome.com/index.php?threads/brocade-1020-cna-10gbe-pcie-cards.2955/

 

 

Edited by IG-88
  • Like 1

Share this post


Link to post
Share on other sites
Posted (edited)
11 hours ago, IG-88 said:

???

 

i'd assume you use a DAC cable or 10GSR SFP+ module for 10G ethernet

FC is different and not used with dsm

 

also if you cant find a win7 driver then win server 2008r2 would be the better choice (same code base as win7)

http://driverdownloads.qlogic.com/QLogicDriverDownloads_UI/SearchByProduct.aspx?ProductCategory=322&Product=1214&Os=190

 

win7 seems to be supported

https://forums.servethehome.com/index.php?threads/brocade-1020-cna-10gbe-pcie-cards.2955/

 

 

 

Ah thanks! The 2008 R2 version worked!

 

After that I had an issue with adding extra.lzma - ended up redoing the USB stick. Not the smoothest of the processes but it works!

 

Speeds are about 180-300mb - probably limited by my hard drives on NAS. 

 

Thanks for the help @IG-88, @quicknick, @Polanskiman!!!!!

 

The last part I have to figure out is this Brocam BIOS hijacking my PC's bios - its weird it blocks the motherboard BIOS even if I disable it in MB BIOS from loading the card at all. 


PS: sorry guys I didnt reply, I was out for a bit.

 

2018-03-31 02_31_49-Copying 1 item (6.54 GB).jpg

Edited by test4321

Share this post


Link to post
Share on other sites
Posted (edited)

not sure what your disk configuration is but even with 12 disks i "only" get ~500MB/s, there is more (900MB/s) for about a few second when writing data to nas goes to ram (source pc for this has nvme ssd so the satat3 limit of ~550MB/s can be broken)

if you want to test network speed without possible limits of disks on both sides involved you can try netio there are versions for windows and linux (netio-x86_64) you just copy the binary to you system(s) and run it on both sides to measure

even if you get a nice number, it does not help you much getting more speed out of the system, for more write speed you can have 2 ssd's as write cache for a volume in dsm (in bare metal there is the sata3 limit atm) and for reading (uncached) data there is just more disks or going all ssd drives (having 6 ssd's in raid config might be able to reach the 1000MB/s?)

there is also the approach to use nvme with esxi and virtual dsm like @flyride did, but i guess thats a very unique scenario

 

 

Edited by IG-88
  • Like 1

Share this post


Link to post
Share on other sites
14 hours ago, IG-88 said:

not sure what your disk configuration is but even with 12 disks i "only" get ~500MB/s, there is more (900MB/s) for about a few second when writing data to nas goes to ram (source pc for this has nvme ssd so the satat3 limit of ~550MB/s can be broken)

if you want to test network speed without possible limits of disks on both sides involved you can try netio there are versions for windows and linux (netio-x86_64) you just copy the binary to you system(s) and run it on both sides to measure

even if you get a nice number, it does not help you much getting more speed out of the system, for more write speed you can have 2 ssd's as write cache for a volume in dsm (in bare metal there is the sata3 limit atm) and for reading (uncached) data there is just more disks or going all ssd drives (having 6 ssd's in raid config might be able to reach the 1000MB/s?)

there is also the approach to use nvme with esxi and virtual dsm like @flyride did, but i guess thats a very unique scenario

 

 


No I'm ok with it. 220mb on average is good!!! 

 

I'll probably stuff 2 SSDs there when I get a better MB. But thats probably for next year project.

 

Thanks for all the help!

Share this post


Link to post
Share on other sites

@test4321 Please update the picutres in your post. Links are broken. Please upload them directly in your post instead of linking them from a remote server. Thank you.

Share this post


Link to post
Share on other sites

You can also get Intel X520-DA cards dirt cheap from China. If you don't mind waiting a few weeks for them to arrive, then they're literally 2-4 times cheaper than getting them locally.

 

I've got two, one in my desktop and the other in my Xpenology box. Both work out of the box and I get around 400-450MB/s throughput copying files from a block-level formatted iSCSI connection (1x SSD drive). I also get just over 2x the performance out of my 3-drive SHR array (7200RPM drives) going from 110MB/s to around 240MB/s.

 

I highly recommend the Intel cards as they "just work" with no configuration whatsoever.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now