there are a lot of nice looking components that will things go south, you need to look very careful for specs and chips on cards
like this one, hyper duo ? sounds like marvell, might only have one at best two pcie lanes and can only do pcie 2.0
i looked it up and its a 88SE9230 so its 2 lanes pcie 2.0 resulting in max 1000MB/s for the 4 drives, so not optimal
the M.2 is pcie 3.0 x4 and there are M.2 to pcie 4x slot adapters with flex cable, so the m.2 can become a pcie 4x slot
https://www.amazon.com/ADT-LINK-Extension-Express-Extender-R42SF/dp/B08DRDT47K
(i bought two, the shorter one above works so far but i did not test much as its in my backup system with 10 disks and kind of crammed)
the pcie 16 slot looks like as it has 16 lanes and is pcie 3.0
maybe a jmb585 based card that supports pcie 3.0 (even with just 2 lanes its way better then the old marvell cards with just pcie 2.0
read this thread about some options
https://xpenology.com/forum/topic/35882-new-sataahci-cards-with-more-then-4-ports-and-no-sata-multiplexer/
i fully agree with @flyride ssd cache might not be worth the risk and cost, 16GB or 32GB ram do some full 10G speed buffering too
might be better to have a ssd volume for special purpose like vm/docker
using the 16x slot for a two or four m.2 ssd seems not a option here as the cheap adapters need bifurication to work and the manual does not mention that
as flyride suggested try 10G nic first and see how it performs and then decide how to go further
ssd cache makes no sense with 1G nic you won't see any difference your 8 disks should easily saturate that nic
i also use a matx but one the has lower depth just 10mm more the m-itx ()but its wider and has 4 pcie slots), getting a new housing might be cheaper and easier the a m-itx with 10G onboard and enough sata options to have 12 or more ports (i know that as my system has 12 ports and 10G and i had m-itx before)