Not being skilled enough to fix this myself I figured let's give this a go;
Have been using XPEnology for quite some time now on VMWare Fusion and ESXi5.1. The issue that keeps coming back is network performance using the vmxnet3 driver.
I am not using RDM but regular vmdk files. A file copy from a 4 drive RAID5 set within the XPEnology build is gives about 130Mb/s. The local PCI-e SSD for the primary volume easily reaches 250Mb/s both read and write from and to the same volume (benchmark on OSX gives a little over 700Mb/s for read and write tests so this is about right). Not bad for a non-RDM volume.
Considering disk performance is not the issue I did some network tests. As others mentioned, e1000 gives the best, most stable and reliable connection. With e1000 I can see speeds between 60 and 80Mb/s in both directions using SMB or NFS. The e1000 adapter uses only a single queue and is not able to harvest the true performance of multiple cores in the system I guess.
The vmxnet3 adapter however is capable of multiqueueing and so it does once started. I am using 2 vcpus on Fusion (core i7) with 2Gb of ram for the vm. When I run ethtool -S eth0 I can see 2 receive and 2 transmit queues. However, rx queue 1 only gives zeros on all counters where both tx queues do seem to be used.
With vmxnet3 is only get between 30 and 60Mb/s and link performance is fluctuating very much. The result is probably less than 25Mb/s for a large file transfer and this only gets worse when you start multiple copy tasks.
I have tried adding and removing CPU's but i have not seen much difference in performance using anywhere from 1 to 8 vcpu's. You do see 8 tx and 8 rx queues with 8 vcpus but again, the last 7 remain 0 for all rx counters whereas the 8 tx queues all seem to be used.
To get some reference for this issue on the same hardware. I used Ubuntu x64 with vmware tools installed. The 10Gb vmxnet3 gives me a VM to host networking performance of more than 230Mb/s. Same thing goes for an old windows7 vm with RSS activated.
Still very happy with all the hard work everyone is doing. XPEnology is great. Whatever the outcome, many thanks for what you've achieved thus far !