OK, I've spent a day on this so I figure I'd share in the hopes that the issue can be fixed and/or I can help someone else with my workaround...
1. I am running an ASRrock C236 WSI mobo that has two builtin 1Gb intel nics (1xi210 and 1xi219) and I added in an intel x550-t1 pcie nic to run on my 10gbe network.
2. I wasn't using the onboard nics, but they were enabled in the bios (which saved me in the end).
3. I properly set up the mac1,mac2,mac3 entries in the grub.cfg with the appropriate mac addresses
when first booting loader to upgrade to 6.0.2 from 5.2, I could see the 10gbe nic requesting an IP through DHCP and my server sending one back... but then it kept asking over and over again, ignoring the responses. so I plugged a cable into the first onboard nic, and voila... worked like a champ.
using the onboard i219 nic, the upgrade ran smoothly.
after booting into DSM, I went to try to figure out what was going on with the network... this is when I realised the i210 and the x550t1 nics seemed to be able to send data out, but nothing would come back. DSM could see that the nics were connected to the network, but couldn't communicate... so it couldn't get an IP address via DHCP. Even when I manually assigned IP addresses, I could not ping that address from another machine.
I ssh'd into the NAS, and started looking at the modules.
I noticed that the i219 used the e1000e driver, while the i210 and x550 used the igb and ixgbe drivers, both of which depended on the dca.ko module (direct cache access) - not sure if this is relevant
Anyway, i found that if I removed and reloaded the ixgbe module, i could then use ifconfig to bring up the network adapter and i could appropriately communicate with the network.
I tried several other workarounds/attempted fixes, but nothing would work... so eventually I just made a script and would unload and reload the ixgbe module and called it from the /etc/rc file just before it configured the network.
that did the trick.
So, why is the igb and the ixgbe drivers not working properly at boot? I have no clue. If anyone has any ideas, please let me know. If you like me to try anything to troubleshoot, I'm happy to help.
Good luck all.