Thanks for your reply!
I'd like to clarify, I wouldn't like to arguing, just trying to understand the "why".
Trying to understand why not beneficial to use a controller designed for this purpose with its own chip and memory and burning resources for SW raid.
All my listed options above based on the idea used the controller to manage the RAID5 array.
In case if I miss the controller based HW RAID5 (or 6)
-can add many datastores and vdisks to the Xpenology VM - as I understand it is not the best way
-passthrough the controller - I think it could be impossible as the DSM itself can not set up to this hardver as bare metal, due the driver issue
-passthrough disks - yes, I'd use RDM for this
I did the last one already, but as I configured the RAID5 on HW level the ESXi saw it as a RAID5 block and the DSM saw it as a single disk and was able to create a volume on it.
It is a big question for me if I'll able to expand the volume size within the DSM after I expand the RAID5 block on HW level (adding physical drives to the block).
But again, I would not like to arguing, instead learn the best possible way to create, manage and later expand the storage pool.