Xpenology on Esxi vs baremetal Xpenology with virtual machine manager


Recommended Posts

I want to build new NAS that will have xpenology and extra virtual systems. I want to use Intel 8gen (with igpu), 2x nvme, 6x hdd in a SHR poool (or raid5/6)+ extra disks for backups, downloads and temporary files. First option is ESXI on nvme + xpenology VM with hba and all single disks passed-through. Second option is baremetal and synology virtual machine manager. Probably DS918 platform would be the best to have transcode and nvme cache support but is it possible to have it on ESXI? Maybe better option will be baremetal in which it should work without major problems, but is there a way to backup system partition just in case? Does synology virtual machine manager have option off pci pass-through? Which option will be the best for me?

Link to post
Share on other sites
  • 2 weeks later...

Yes it is posible to have xpenology on ESXI , as a matter of fact i hae it running on a ESXi 7.0 host 

Runs quite well, but stay away from the  vmware tools for package i see high memory usage with it installed contatnly at 75-80% of the 16GB i have alloated to it

 

Link to post
Share on other sites
Posted (edited)

So I am trying to pass-trough pcie sata/sas controller( flashed into it mode). Dont know what I am doing wrong but there is no way to pass controller with 6 disks attached. Sometimes it shows 5 disks in DSM, sometimes one, sometimes 4 (depends on when I add another sata controller, scsi controller, etc). Tried with 2 controllers and 3 pcie slots on my mobo without luck. Can someone explain how it should be done (I did not think there will be so many problems with it...)

Edited by ruffpl
Link to post
Share on other sites

The default configuration for the loader is to support (1) virtual SATA controller to support the boot loader VMDK, and one additional SATA controller.

 

If you are following the ESXi install guide, it suggests creating a second virtual SATA controller to support an example virtual disk for a Storage Pool/Volume.  If you add a passthrough controller, this will cause you some issues as you may run out of disk slots (12 total: with 8 default dedicated to the virtual SATA #2 only allow you 4 on the passthrough).

 

If you don't need virtual disk support and are only installing DSM to disks attached to your passthrough controller, delete your second virtual SATA controller and things should generally work ok.  If you need all three controllers online (to support virtual disks along with physical simultaneously), you will need to modify grub.cfg parameters SataPortMap and DiskIdxMap to match your individual situation, in order to get all your disks online within the 12 slots allowed. https://xpenology.com/forum/topic/35937-configuring-sataportmap

 

Alternatively, you can increase the number of slots allowed by modifying /etc.defaults/synoinfo.conf, and change the maxdisks value to something >12 and <=24 (DiskIdxMap may still need to be modified to push the loader device out of range). But that will be overwritten anytime DSM is installed (including updates, although that may be a moot point at this time).

 

If you are using SAS disks, disk to port mappings work less predictably, but increasing the maxdisks value tends to be effective.

Edited by flyride
Link to post
Share on other sites
Posted (edited)

I need all the data and synology applications on SHR all connected to pcie sata controller, just like on baremetal instalation + nvme I have connected to mainboard for cache ( I need ESXI for other systems and other devices pass-through that is way I can not go baremetal)/ I will use ds918 for that( I can optionally use virtual 16gb/ just need an option of snapshots to revert update, etc by esxi)

What do I need exactly to make it run? Just synoboot +vmdk with correct serial and mac adresses in grub.cfg + one virtual sata set to 0:0? I want to use pcie sas controller with 2x4 sata cables (6hdds for now) + extra nvme disk passed-trough. Should I make some changes in grub.cfg to get all sata ports working or should I do it some other way?  I dont understand this " SataPortMap and DiskIdxMap " part. Could You please explain how it should be done?

Edited by ruffpl
Link to post
Share on other sites

I also run ESXi and passthrough my on-board SATA controller to DSM VM.  I have a NVMe drive that is used for ESXi datastore and all the other VM's.

How is the scratch storage/other VM datastores attached on your planned system?

 

1 hour ago, ruffpl said:

What do I need exactly to make it run?

 

You can use the default serial number in the loader.  No need to change unless you are going to run multiple DSM instances at the same time.  The MAC is only critical if you are going to use Wake-On-LAN - unlikely with a VM.

 

Set up your loader VMDK on SATA Controller 0 (0:0) and nothing else on that controller.  Don't add another virtual SATA controller or virtual disk.  Passthrough your PCIe controller and you ought to see all your physical disks on which to install.  Worry about the NVMe disk afterward.

Link to post
Share on other sites
Posted (edited)

I got ds 918 loader and vdmk (found somewher over internet) on sata 0:0. Controller is passed-through and I stil dont see all disks. I had passed the controller to Win10 pe, removed all partitions and format all drives (they are all visible on other vm systems also). When I try to pass-through to Xpenology there is only maxiumum 4 drives (one 6tb drive is out/got some errors, now I will have to use 5 drives). I can find synology over the net and install system on drives passed by controller but I can never get all drives to work (sometimes it shows 3, sometimes 1, sometimes 4....). I would like to have them in use like on baremetal/no virtual drives, etc.

Edited by ruffpl
Link to post
Share on other sites
3 minutes ago, ruffpl said:

I got ds 918 loader and vdmk (found somewher over internet) on sata 0:0

 

Sorry what does this mean?  You should be using the loader file from the official link?  Are you using one vmdk or two?

 

4 minutes ago, ruffpl said:

When I try to pass-through to Xpenology there is only maxiumum 4 drives (one 6tb drive is out/got some errors, now I will have to use 5 drives). I can find synology over the net and install system on drives passed by controller but I can never get all drives to work (sometimes it shows 3, sometimes 1, sometimes 4....)

 

Post some relevant screenshots (overview, HDD/SDD) from Storage Manager.

Link to post
Share on other sites
Posted (edited)

I am using 1.04b synoboot downloaded by Xpenology tool ( I also tried this one with vdmdk/ I think I downloaded them from here). This is what I got (should I change the line 37 inside grub cfg?)

 

2021-04-22 22.35.00 192.168.1.100 7cf522763d31.png

2021-04-22 22.34.17 192.168.1.101 cdcb02582e83.png

2021-04-22 22.33.57 192.168.1.101 e6fd75b75ec6.png

2.png

Edited by ruffpl
Link to post
Share on other sites

Ok, a few things:

 

First, SAS support is better on DS3617xs than DS918+.  You should consider moving to 1.03b and DS3617xs for the best results.

 

It looks like you decided to uncomment the grub command set sata_args='SataPortMap=4'

 

If you stay on DS918+, I would try the following: set sata_args='SataPortMap=1 DiskIdxMap=1000'

 

The SataPortMap argument tells DSM to only use 1 slot (the loader) from the first controller, then DiskIdxMap assigns the first controller to slot 17 (effectively hiding the loader), and the second controller (hopefully your passthrough) to slots starting at 00 (/dev/sda).

 

If for some reason you decide to switch to DS3617xs, the DiskIdxMap parameter should be 0C00 instead of 1000.

 

SAS controllers sometimes behave weird with this setup, but let's try it first and see what you get.

  • Like 1
Link to post
Share on other sites

So I am trying again. I wanted to convert my synoboot.img by Starwind V2V converter instead of using the one I had downloaded but I get error "warning this format is not compatible with esxi vmdk format". After creating the vdmk file I can not add existing hard drive in ESXI so I can not create the boot disk... Any idea ?

Link to post
Share on other sites
Posted (edited)

YES! Made it! All drives attached to sas controller are visible in Storage Manager.

 

What I have done was to convert my individual ( thanks to flyride ) synoboot.img by the old version of StarWind V2V Converter (v8.165). After that I had to upload both files .vmdk and flat.vmdk with synoboot.img so ESXI could see it and apply it as sata 0:0 disk. System got recovered from 3 hdds that was visible last time and now it all looks promising.

Now its time for nvme, gpu for transcoding, quad ethernet card and dvbt tuner to work with it.

 

Ps. Can I add another 2 disks to the controller with  " set sata_args='SataPortMap=1 DiskIdxMap=1000'  " if I will create SHR volume now and would like to add extra drives to it later?

disks.PNG

Edited by ruffpl
Link to post
Share on other sites
  • 2 weeks later...
Posted (edited)

I added a spare ssd drive for testing cache and it showed up without problem (dont want to play with nvme it will be to complicated )

 

On 4/23/2021 at 2:15 AM, flyride said:

 

 

If you stay on DS918+, I would try the following: set sata_args='SataPortMap=1 DiskIdxMap=1000'

 

The SataPortMap argument tells DSM to only use 1 slot (the loader) from the first controller, then DiskIdxMap assigns the first controller to slot 17 (effectively hiding the loader), and the second controller (hopefully your passthrough) to slots starting at 00 (/dev/sda).

 

 

 

Could You tell me how to make the loader hidden but get a extra virtual sata hdd seen by storage manager? I want to have it attached to dsm (just as a copy of system partition like on other drives). If I would like to update the system I could temporary remove the passed sas controller, create snapshot of the virtual disk and then make an update ( If there will be a problem I could just restore it and after that attach the controller back to it)

Edited by ruffpl
Link to post
Share on other sites

Any disk that is "Initialized" has the system partition on it.  Just add a VMDK and put a Basic Storage Pool on it.

 

The way I would do this is add in a second SATA Controller (SATA 0:0 is still the loader, SATA1:0 is the new VMDK).  Then you need to understand the PCI slot order of each controller.  It should be SATA0 with the lowest slot #, SATA1 with the next lowest, and finally your passthrough controller with the highest.  If it differs from that, you will need to reorder the SataPortMap and DiskIdxMap parameters accordingly.

 

Assuming that the slot order is SATA0, SATA1, passthrough, set SataPortMap=11 and DiskIdxMap=100001

 

SataPortMap setting explained: one loader disk on SATA0, one VMDK on SATA1, no limits on passthrough (defaults to 9 I think)

DiskIdxMap setting explained: loader disk assigned to /dev/sdq, VMDK assigned to /dev/sda, passthrough disks starting with /dev/sdb

Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.