mervincm

rebranded LSI controllers like the H200 (I have an HPE version) in IT mode are the default best option from my opinion. Inexpensive, TONS of them around, lots of support available from home-labbers. Most add 8 6G SAS/SATA ports with no complications around SMART etc.

I agree with the MicroTik interface comment for the most part; that being said, for a flat switch, it's really nothing to do. Additionally, this model runs switch-OS, not the router OS you are used to. much much simpler. I have 4 10 gig switches in my home lab (Ubiquiti UNIFI 16x10gig, Microtik CRS-226 24x1, 2x10, quanta l4b, 48x1GB and 2x10G, and a DLINK 24x1G, 4x10G) so I have seen a range of less common options (no cisco/HPE/juniper) At this time I only really use the D-Link Enterprise line switch. a switch allows you to have a single IP address and a single name/DNS entry for your NAS. It's a nice to have, not a need to have. I do 10GBE main PC to physical NAS, and to vSphere Host. I also do multichannel SMB3 over multiple NICs to some secondary PC's and a second NAS (real Synology box) physical routing/firewall/IDS make do with 1 GBE because my WAN link is limited.

I suggest getting a switch. it will cost you 1 more cable, but it makes your networking so much easier. managing 2 networks (1 Data, and 1 Storage) is definitely possible, but it's a mess that you don't have to deal with for under 200$. I have tried both. You can buy a brand new switch from Mikrotik with a couple 10 gig ports for under 150$ (CSS326-24G-2S+RM)

Seems like the right approach to me. Disable sequential caching till NVME support is available. It should not be too long, as models like the 918+ have nvme caching features. Once that is in Xpenology, you can LACP your 2 links from switch to NAS, and one day be pushing for 2 gigabytes per second transfers!

I have a lenovo Xpenology system with a 10gig intel 559 based card, xeon 1226v3 CPU, 32GB RAM, and 6HDD (WD RED) RAID5, LSI IT mode HBA (6g SATA/SAS) an well as a real synology 1815+ w 5HDD RAID5. I have not tried adding in SSD cache as my testing (back in 5.x days) showed that adding them in impacted my sequential performance in a negative way. my Win10 system is a 6700k w 32 GB ram, Intel SSD750 and a samsung 960 evo, and on the same 10gigE intel card. I can do some tests, so you have something to compare to.

Safe to assume you are using a 10Gbe network here? When you created your SSD cache you created a 2 SSD Read/Write and only for sequential transfer correct? Atom CPU (a great atom but still an atom) might be impacting performance, did you monitor CPU while this is happening? 3HDD RAID5 supplies a steady 500-600MB/s? that's not real, must be caching involved. the most you can hope for in writing to a 3 disk raid 5 array is 2x a single disk, and I doubt you have HDD disks that can go that fast my really old agility 3 SSDs five 120 write performance in the DSM write benchmark in my xeon system. maybe you have an issue with backplane? BTW very cool motherboard! I would love one!

I have used a range of 10GB NIC's, optical transceivers as well as DAC cables, and 4 different switches with 10g. I don't think that the mellanox connectX-2 (I tested one in the past) have any sort of built in driver support in XPenology. I have used broadcom in the past, and now use intel 82599 based cards. If you just want to built a point to point 10GBE link, I had great luck with the IBM BNX2 broadcom cards their included transceiver, and a standard OM3 LC/LC fiber patch. I also use the 3615xs version

This can be a number of things, some are not easily fixed. HBA, SATA/SAS Backplane, port multiplier, OS. The 6/i looks to be a bad choice for xpenology for two reasons, no IT mode firmware and no support for over 2TB disks. if you are sure you don't have a backplane limitation, this might be as easy to fix as purchasing a replacement SAS/SATA HBA. There are other forums that are more suited to server grade parts in the home lab you might try there for specific advice.

I started with ESXi, but am now running it bare metal. ESXi was fine, but I didn't need any extra VM hosting. You can do that now inside DSM with virtual machine manager anyway. I wanted to give DSM local access ALL of the HDD for smart testing. I wanted push button power button support, USB support One item you should consider is that bare metal install means that you need DSM to include your driver support whereas with esxi you need vSphere to include your driver support. This might be a deal breaker either direction, depends what you have.

update: it's working. moved the USB boot drive from my test server to the main Xpenology NAS (exact same hardware) and it is stable running at 10gbps on the Intel card (broadcom still non functional.) It seems that something in .1 or u4 fixed the problem. Still no luck with the broadcom, but I would rather use the intel anyway.

Did a test today. I have a real Synology 1813+ and its setup (today) with a single SHR volume, 8 HDD + BTRFS FS. setup a 4gig link with link aggregation with its 4 NICS. removed all packages and turned off all reports, indexing etc. Created a best case scenario (without SSDs.) Directly from the 1813+, I did a rsync copy pulling huge files from my xpenology. at the same time I did a CIFS copy (huge files) from a workstation to the 1813+. I was able to get CUMULATIVE averages over 1gbps and CUMULATIVE peaks at about 1.6 gbps. If you had more clients looking the write to the 1813+ at the same time, it might have gone even higher. This is a typical scenario where link aggregation can help.

10 gigabit networking does not have to be expensive. IMO a starter setup is just 2 NICS (USED can be as low as 20-30$ each used) (not all work with xpenology at this time) 2 "cables" (cables for cheap 10 gig networking usually means 2 optical SFP+ transceivers and a fiber patch cable the length required, OR an SFP+ DAC cable) each cable is about 20$-75$ 1 switch with at least 2 SFP+ ports, new from 200$-400$, used from 100$ its doable for not too much.

I have played with read and read/write caches. I found the read cache made no noticeable improvement, but never did it seem to get it the way. The Read/write cache actually made random recent reads noticeably better, but also negatively impacted by sequential writes. I took them both and build a dedicated SSD volume. MUCH MUCH better. it was big enough to run my apps and small VMs from, worked great.

Before you spend more time on link aggregation, are you sure you understand what advantage it offers? If you are trying to get more that 1gbps between your NAS and a single client, IT DOESN'T WORK THAT WAY. No matter how many NICS in your PC or NAS, no matter how you create your bonded channel, 1 client maxes at 1gbps. What you can hope to achieve (and its hardly easy) is have multiple clients simultaneous aggregate to over 1gbps. And even if you can get the networking done right, you will still need to have the disks to keep up. When you have simultaneous devices accessing your NAS, thats a someone random activity, and you need ALOT of hard drives to exceed 1gbps of random activity, unless you create an all SSD volume. even if you get everything working how many minutes a week is everything gonna line up perfectly? there are two solutions that do work to give you over 1gbps somewhat often. a-10gbps NICS and a 10gbps switch - used cards and switches are very affordable - this is what I do b-multichannel SMB 3 - right now only works on windows systems, available in linux based systems using the latest samba systems if enable EXPERIMENTAL features.

I should have added that I was able to get both the broadcom 10gbe and the intel 10 gbe NICs working with 5.x, ran it that way for months if not over a year. I have upgraded from 6.1 to 6.1.1u4 and there is a significant improvement, if not quite fixed? is it possible syno tweaked a driver or something? It is MUCH better, at least with the Intel Card I have been testing with so far. I can read from the NAS at about 300MB/sec (OK for a 4 Disk R5 BTRFS) I can only write to the NAS at about 105MB/sec (not great) The only thing I did differently was testing BTRFS this time, I used ETX4 for previous testing. I want to let it settle for a while, make sure it is stable, then I will try the broadcom card and then both again with the new ram drive (assuming that this ram drive is NOT the same as you had it your 6.02 tutorial, as I used that one in yesterdays testing)