dfortier

Infiniband install help

Recommended Posts

Hello,

I'm im having some trouble getting infiniband to work on xpenology, and hoping someone has some advice. Ive tried two different cards on two different machines, the results are the same. Nothing seen in the gui and nothing seen via ifconfig. Both were update to the latest patch found here. The cards are Mellanox MHQH29B-XTR (connect-x2), Mellanox MHGA28-XTC (connect-x). id rather use the connect-x because otherwise it will be collecting dust, but working on either would be great.

 

i download the drivers from the link in this post: viewtopic.php?f=2&t=6070

 

this will be my nas device:

Supermicro x8dt3-ln4f

dual xeon e5520

16gb ecc

mix of 1tb and 3tb drives

has quad on board nics, i thought this may be an issue if. is xpenology limited to only four nic cards?

 

so i started messing around on a test machine, to get a "proof of concept"

 

test machine:

gigabyte 990fxa-ud3

amd fx 8320

8gb ram

60gb ssd for test load os

onboard lan off, using intel dual port 1gb

 

Process:

Standard load process works like a charm, patches install w/o a hitch (thanks for all the hard work contributors)

reboot between patches

used winscp to install the ko files (/lib/modules) and modified the /etc/rc and /etc.default/rc per the readme in zip file.

reboot after driver install

 

lsmod results:

 

Xpenology> lsmod | grep mlx

mlx4_ib 104330 0

ib_sa 19338 4 mlx4_ib,rdma_cm,ib_ipoib,ib_cm

ib_mad 34788 4 mlx4_ib,ib_cm,ib_sa,ib_mthca

ib_core 47092 10 mlx4_ib,svcrdma,xprtrdma,rdma_cm,iw_cm,ib_ipoib,ib_cm,ib_sa,ib_mthca,ib_mad

mlx4_en 67584 0

mlx4_core 169852 2 mlx4_ib,mlx4_en

newbie to linux, but looks ok?

 

ifconfig results:

 

Xpenology> ifconfig

eth0 Link encap:Ethernet HWaddr xxxxxxxxxxxxxx

inet addr:192.168.40.231 Bcast:192.168.40.255 Mask:255.255.255.0

inet6 addr: fe80::92e2:baff:fe18:d448/64 Scope:Link

UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1

RX packets:1617 errors:0 dropped:0 overruns:0 frame:0

TX packets:774 errors:0 dropped:0 overruns:0 carrier:0

collisions:0 txqueuelen:1000

RX bytes:638456 (623.4 KiB) TX bytes:340936 (332.9 KiB)

 

eth1 Link encap:Ethernet HWaddr xxxxxxxxxxxxxxxx

inet addr:169.254.242.227 Bcast:169.254.255.255 Mask:255.255.0.0

UP BROADCAST MULTICAST MTU:1500 Metric:1

RX packets:0 errors:0 dropped:0 overruns:0 frame:0

TX packets:0 errors:0 dropped:0 overruns:0 carrier:0

collisions:0 txqueuelen:1000

RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)

 

lo Link encap:Local Loopback

inet addr:127.0.0.1 Mask:255.0.0.0

inet6 addr: ::1/128 Scope:Host

UP LOOPBACK RUNNING MTU:65536 Metric:1

RX packets:58 errors:0 dropped:0 overruns:0 frame:0

TX packets:58 errors:0 dropped:0 overruns:0 carrier:0

collisions:0 txqueuelen:0

RX bytes:6165 (6.0 KiB) TX bytes:6165 (6.0 KiB)

 

so i have a few questions:

is xpenology limited to only four nic cards, if this is the case than i may need a new nas board....

anyone see a flaw in my process?

are the cards im using simply too old, i didnt see a card mentioned in the successful test system in the thread i was reading.

 

Any input would be most appreciated

Share this post


Link to post
Share on other sites
In the syslinux.cfg located on your usb stick you can edit the network interface number :

netif_num=4 to netif_num=X

 

Other users with mellanox have to switch port type form ib to eth:

https://community.mellanox.com/docs/DOC-1131

look the inbox driver part

 

thanks for the info. unfortunately i have an older Connect-x2 card and it doesnt support eth mode to my knowledge.

Share this post


Link to post
Share on other sites