Jump to content
XPEnology Community

10Gbe setup - will this work with 6.1?


test4321

Recommended Posts

Hey guys,

 

I'm moving my machine to some better hardware:

 

i5-6400T

H110I PRO AC

Crucial 8GB Single DDR4

 

Now for networking I want to go 10GBe SFP+

 

I am looking at:

 

2X - Mellanox ConnectX-2

https://www.ebay.com/itm/391459428428

 

1X - Cisco Genuine SFP-H10GB-CU3M SFP+ 

https://www.ebay.com/itm/201986388845

 

^^^ will these work out of the box with 6.1 DS3615xs version? 

 

Thanks!

 

Link to comment
Share on other sites

  • 3 weeks later...

I have used a range of 10GB NIC's, optical transceivers as well as DAC cables, and 4 different switches with 10g.

I don't think that the mellanox connectX-2 (I tested one in the past) have any sort of built in driver support in XPenology. I have used broadcom in the past, and now use intel 82599 based cards.

 

If you just want to built a point to point 10GBE link, I had great luck with the IBM BNX2 broadcom cards their included transceiver, and a standard OM3 LC/LC fiber patch. 

 

I also use the 3615xs version

 

 

 

  • Like 1
Link to comment
Share on other sites

10 hours ago, mervincm said:

I have used a range of 10GB NIC's, optical transceivers as well as DAC cables, and 4 different switches with 10g.

I don't think that the mellanox connectX-2 (I tested one in the past) have any sort of built in driver support in XPenology. I have used broadcom in the past, and now use intel 82599 based cards.

 

If you just want to built a point to point 10GBE link, I had great luck with the IBM BNX2 broadcom cards their included transceiver, and a standard OM3 LC/LC fiber patch. 

 

I also use the 3615xs version

 

 

 

 

Thanks! Thats my next step is to find those on ebay for a good price. 

 

Which wire did you run between? Are you running device to device or device to switch?

Link to comment
Share on other sites

you can check whats supported as additional drivers (link below) in the network driver section

 

ootb (without additional drivers, aka build into dsm itself) the range is very limited

you can check that directly on the synology page, like this for 3615 (the synology branded cards in this example have a tehuti chip, so that one is supported too, i use one with rj-45 that was branded by qnap, tn40xx is the linux driver)

https://www.synology.com/en-global/compatibility?search_by=products&model=DS3615xs&category=network_interface_cards&p=1

or a pcie list and devices to every card

https://www.synology.com/en-us/compatibility?search_by=category&category=network_interface_cards&p=1

so if you are looking for cheap mellanox on ebay it should be ConnectX-3 or X-4

 

the cheapest new 10G cards supported with additional drivers are Aquantia AQC-100/107/108 based cards (afaik)

 

 

  • Like 1
Link to comment
Share on other sites

  • 3 weeks later...

Thanks guys! I got the box setup going to be thinking about which way I wanna go with the network cards.

 

1st way - built in drivers but expensive cards.

 

2nd way - cheap network cards, but a lot of time spent on creating drivers and tinkering.

 

I'll write up a post on what I did.

 

Thanks again!

Link to comment
Share on other sites

19 minutes ago, test4321 said:

Thanks guys! I got the box setup going to be thinking about which way I wanna go with the network cards.

 

1st way - built in drivers but expensive cards.

 

2nd way - cheap network cards, but a lot of time spent on creating drivers and tinkering.

 

I'll write up a post on what I did.

 

Thanks again!

Cheap route: 10Gbe's are the Brocade (QLogic) BR-1020's and it will always be supported by me because it is what I use.

 

Expensive route: Can't go wrong with  most Intel cards because they are easily supported, and most already are

  • Like 1
Link to comment
Share on other sites

2 minutes ago, quicknick said:

Cheap route: 10Gbe's are the Brocade (QLogic) BR-1020's and it will always be supported by me because it is what I use.

 

Expensive route: Can't go wrong with  most Intel cards because they are easily supported, and most already are

 

So Brocade drivers are already in the standard loader 1.02 A/B?

Link to comment
Share on other sites

Just now, quicknick said:

It will be released in my loader slated to be released tomorrow.

 

Sweet thank you! 

 

I'll get on searching for the hardware then! 

 

One more question:

 

do you have suggestions on direct connection cable(PC to PC  - no switch)? is it any different than just normal cables?

 

Will something like this work:

 

https://www.ebay.com/itm/201986388845

 

??

Link to comment
Share on other sites

1 hour ago, test4321 said:
  Sweet thank you! 

 

I'll get on searching for the hardware then! 

 

One more question:

 

do you have suggestions on direct connection cable(PC to PC  - no switch)? is it any different than just normal cables?

 

Will something like this work:

 

https://www.ebay.com/itm/201986388845

 

??

 

just get the brocade twinax cables and do pc to pc. forget getting a switch unless you got money to spend. also don't use cisco twinax cables with the brocade 1020. 

Sent from my SM-N950U using Tapatalk

  • Like 1
Link to comment
Share on other sites

On 2/20/2018 at 7:30 PM, quicknick said:

just get the brocade twinax cables and do pc to pc. forget getting a switch unless you got money to spend. also don't use cisco twinax cables with the brocade 1020. 

Sent from my SM-N950U using Tapatalk

 

Alright, ordered:

 

https://www.ebay.ca/itm/372110856130

 

and

 

https://www.ebay.ca/itm/302545665700

 

No pressure on releasing that new loader ;)

 

Thanks again!

Link to comment
Share on other sites

I suggest getting a switch. it will cost you 1 more cable, but it makes your networking so much easier. managing 2 networks (1 Data, and 1 Storage) is definitely possible, but it's a mess that you don't have to deal with for under 200$.  I have tried both.

 

You can buy a brand new switch from Mikrotik with a couple 10 gig ports for under 150$ (CSS326-24G-2S+RM)

 

 

 

 

  • Like 1
Link to comment
Share on other sites

23 minutes ago, mervincm said:

I suggest getting a switch. it will cost you 1 more cable, but it makes your networking so much easier. managing 2 networks (1 Data, and 1 Storage) is definitely possible, but it's a mess that you don't have to deal with for under 200$.  I have tried both.

 

You can buy a brand new switch from Mikrotik with a couple 10 gig ports for under 150$ (CSS326-24G-2S+RM)

 

 

 

 

 

Thanks, but I dont like microtic interface...I worked with Microtic router and it was like pulling teeth without lidocaine :)

 

I'll probably invest into UNIFI switch / router, but not right now. 

 

 

Edited by test4321
Link to comment
Share on other sites

I suggest getting a switch. it will cost you 1 more cable, but it makes your networking so much easier. managing 2 networks (1 Data, and 1 Storage) is definitely possible, but it's a mess that you don't have to deal with for under 200$.  I have tried both.
 
You can buy a brand new switch from Mikrotik with a couple 10 gig ports for under 150$ (CSS326-24G-2S+RM)
 
 
 
 
It all depends on the use case. I deduced that test4321 just needed connection from 2 nodes. if it was different, then switches would bw the way to go.

Even with ESXi hosts or kvm hypervisors traffic can communicate between each other because switching happens on the hypervisor even with multiple portgroups.

Or you can get really complicate with VMware and do VXLans, but that is going into the weeds.

I keep my traffic east-west and everything north of my fws. ie workstations/laptops/wifi , live in a 1 Gbe world. everything fw, ids ips and below is 10Gbe and all virtual.

Sent from my SM-N950U using Tapatalk

  • Like 1
Link to comment
Share on other sites

I agree with the MicroTik interface comment for the most part; that being said, for a flat switch, it's really nothing to do. Additionally, this model runs switch-OS, not the router OS you are used to. much much simpler. I have 4 10 gig switches in my home lab (Ubiquiti UNIFI 16x10gig, Microtik CRS-226 24x1, 2x10, quanta l4b, 48x1GB and 2x10G, and a DLINK 24x1G, 4x10G) so I have seen a range of less common options (no cisco/HPE/juniper) At this time I only really use the D-Link Enterprise line switch.

 

a switch allows you to have a single IP address and a single name/DNS entry for your NAS. It's a nice to have, :) not a need to have.

 

I do 10GBE main PC to physical NAS, and to vSphere Host. I also do multichannel SMB3 over multiple NICs to some secondary PC's and a second NAS (real Synology box) physical routing/firewall/IDS make do with 1 GBE because my WAN link is limited.

Edited by mervincm
Link to comment
Share on other sites

3 minutes ago, mervincm said:

I agree with the MicroTik interface comment for the most part; that being said, for a flat switch, it's really nothing to do. Additionally, this model runs switch-OS, not the router OS you are used to. much much simpler. I have 4 10 gig switches in my home lab (Ubiquiti UNIFI 16x10gig, Microtik CRS-226 24x1, 2x10, quanta l4b, 48x1GB and 2x10G, and a DLINK 24x1G, 4x10G) so I have seen a range of less common options (no cisco/HPE/juniper) At this time I only really use the D-Link Enterprise line switch.

 

a switch allows you to have a single IP address and a single name/DNS entry for your NAS. It's a nice to have, :) not a need to have.

 

 

 

I dont have that many needs for my backups. Point to point should be good enough.

 

So for now, no need for a switch. 

Link to comment
Share on other sites

10 minutes ago, quicknick said:

It all depends on the use case. I deduced that test4321 just needed connection from 2 nodes. if it was different, then switches would bw the way to go.

Even with ESXi hosts or kvm hypervisors traffic can communicate between each other because switching happens on the hypervisor even with multiple portgroups.

Or you can get really complicate with VMware and do VXLans, but that is going into the weeds.

I keep my traffic east-west and everything north of my fws. ie workstations/laptops/wifi , live in a 1 Gbe world. everything fw, ids ips and below is 10Gbe and all virtual.

Sent from my SM-N950U using Tapatalk
 

 

Yeah my setup is super simple. I only have 2 PCs in my bedroom - one PC, one NAS(right next to each other). I sometimes need to do 3-4TB backups - which is a pain at 100mb speeds. So 10GBe is the way to go.

 

Current spindle drive speeds are around 150mb-250mb. SSD's getting cheaper too (some 1TB drives are going below $300 mark) so sitting at 100mb / s has no point.

 

 

 

Link to comment
Share on other sites

Found compatibility list:

 

https://community.brocade.com/dtscp75322/attachments/dtscp75322/FibreChannel/248/1/External Brocade Qual and Support Transceiver Matrix.pdf

 

Brocade 1020 and 58-1000023-01 (10G-SFPP-TWX-0501) seem to be compatible. 

 

PS: The more I look into fiber networking the more I'm confused about it's terms and compatibility issues. Why would they not get a standard going? Seems like fiber transceivers are made specifically just to lock customers into getting just that specific company product and only it.

Link to comment
Share on other sites

Found compatibility list:
 
https://community.brocade.com/dtscp75322/attachments/dtscp75322/FibreChannel/248/1/External Brocade Qual and Support Transceiver Matrix.pdf
 
Brocade 1020 and 58-1000023-01 (10G-SFPP-TWX-0501) seem to be compatible. 
 
PS: The more I look into fiber networking the more I'm confused about it's terms and compatibility issues. Why would they not get a standard going? Seems like fiber transceivers are made specifically just to lock customers into getting just that specific company product and only it.
You are correct. It is about money and lock in.

Sent from my SM-N950U using Tapatalk

Link to comment
Share on other sites

btw. Brocade 1020 is part of the driver list for my extra.lzma, but color code is yellow so no one requested it yet (i try to find someone to test so i can see at least one positive  when inserting a driver)

"bna: Brocade 1010/1020 10Gb Ethernet"

 

so as long as we can compile drivers (not shure about 6.2, that seems to bring signed drivers into play) it should be possible

 

if you go for independent from external drivers you can check what *,ko files are in dsm (the hardware list provided by synology lists only specific cards but usually the chip is supported by the kernel module so all cards with that chip should work)

it you look for cheap then tehuti based (tn40xx.ko) cards are a option (10GBase-T and SFP+ available)

my route was to use a 2 port card on the nas ( ASUS PEB-10G/57840-2T, bnx2x.ko based, driver is part of dsm) and two tehuti based cards for backup destination (also xpenology, tn40xx.ko is part of dsm)  and to desktop pc, all three have also a 1G connection on the main switch, 10G is just used ip based or with a host file for name resolution

but thats for 2 years ago, now 4 or 8 port 10GBase-T switches start to get affordable (but 10GBase-T has higher latency then SFP+)

 

  • Like 1
Link to comment
Share on other sites

2 hours ago, IG-88 said:

btw. Brocade 1020 is part of the driver list for my extra.lzma, but color code is yellow so no one requested it yet (i try to find someone to test so i can see at least one positive  when inserting a driver)

"bna: Brocade 1010/1020 10Gb Ethernet"

 

so as long as we can compile drivers (not shure about 6.2, that seems to bring signed drivers into play) it should be possible

 

if you go for independent from external drivers you can check what *,ko files are in dsm (the hardware list provided by synology lists only specific cards but usually the chip is supported by the kernel module so all cards with that chip should work)

it you look for cheap then tehuti based (tn40xx.ko) cards are a option (10GBase-T and SFP+ available)

my route was to use a 2 port card on the nas ( ASUS PEB-10G/57840-2T, bnx2x.ko based, driver is part of dsm) and two tehuti based cards for backup destination (also xpenology, tn40xx.ko is part of dsm)  and to desktop pc, all three have also a 1G connection on the main switch, 10G is just used ip based or with a host file for name resolution

but thats for 2 years ago, now 4 or 8 port 10GBase-T switches start to get affordable (but 10GBase-T has higher latency then SFP+)

 

 

I will be able to test around March 10-15th. My items already shipped out.

 

I dont know much about plugging in drivers into a loader though. (I'll look into it this weekend)

Can you point me into direction on how to do that? 

 

Otherwise we can wait for Quicknicks loader :)

Edited by test4321
Link to comment
Share on other sites

if quicknicks loader comes out there will be no need for my extra.lzma anymore i guess and if a driver from his collection does not work we can easily use the source i used for my extra.lzma and fix it
 
Not true. your extra.lzma may still be used. I will not be able to keep up on driver uodates like you have .

Ihaven't had time to compare my drivers vs yours, but any user could replace my extra.lzma with yours and still get all the benefits of my loader because my ramdisk is always last.

drivers, firmwares and /etc/modules.conf will always remain in extra.lzma. anyone can replace those portions as they see fit.

Personally, I wanted to rename to drivers.lzma, but for compatability between Juns loadrr and mine I left as extra.lzma.

Sent from my SM-N950U using Tapatalk

  • Like 1
Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...