snailium Posted April 30, 2021 Share #1 Posted April 30, 2021 I know there are tutorials for installing DSM on ESXi as "virtualization" by creating virtual SATA drive. I ran into DSM upgrade problem (Error 13) using that method. Finally, I've dicovered a way to let ESXi create an emulated USB storage for the loader (the BAREMETAL way)! 1. Get the loader. Either take Jun's loader 1.04b and do some customization, or take the modified loader here. 1a. Modify VID/PID grub.cfg If you are using the modified loader, the VID/PID has already been fixed. Skip to the next step. If you are using original 1.04b loader, follow the baremetal guide to mount the second partition in the loader, edit grub.cfg using the following VID/PID set. # VMware virtual USB mass storage set vid=0x0e0f set pid=0x0005 DO NOT remove any boot options, ONLY update VID/PID (and SN/MAC if you have them). Then, use StarWind V2V Converter to covert the .bin to .vmdk. (Note: it will generate a .vmdk and a -flat.vmdk file. Both files are required) 2. Create VM in ESXi While creating VM, add a USB 3.0 controller In the VM options, select OS type "Other 4.x or later Linux (64-bit)". In the boot option, select EFI firmware, uncheck Secure boot, and select Force BIOS setup. After VM is created, DO NOT power it up for now. 3. Attach loader as emulated USB storage. Go to ESXI datastore. First, update synoboot.vmdk + synoboot.bin to the VM folder (synoboot.vmdk + synoboot-flat.vmdk if you follow Step 1a). Then, download the VM's .vmx file (VM configuration file). Edit it with a text editor. Make sure the following line present in the .vmx file. usb_xhci.present = "TRUE" Then, add the following line at the bottom of the file. usb_xhci:3.present = "TRUE" usb_xhci:3.deviceType = "disk" usb_xhci:3.fileName = "synoboot.vmdk" usb_xhci:3.readonly = "FALSE" Re-upload the .vmx file to datastore. Now it is ready to power up the VM. 4. Setup boot device As soon as the VM is powered on, it should go into the BIOS settings. Navigate to Enter setup -> Configure boot option -> Enable or disable boot option. Leave "EFI USB device" as ENABLED and set the rest options to DISABLED. Commit change and exit, and reboot the VM. Now the DSM is running in ESXi in the same way as a "baremetal" installation. No more Error 13 while upgrade DSM! 1 1 Quote Link to comment Share on other sites More sharing options...
tvon Posted May 3, 2021 Share #2 Posted May 3, 2021 I gave this a shot and it seemed to work but after going through the find.synology tool, it said the VM had no HD attached. Quote Link to comment Share on other sites More sharing options...
Wurstwasser Posted May 7, 2021 Share #3 Posted May 7, 2021 This actually works great. I am using a passed through Dell H200 HBA flashed to LSI IT-Mode. Also i am using a virtual SATA-Controller from ESXi with one disk in addition. The only thing i noticed is that you have to use common_args instead of sata_args to define your port mappings. The variable sata_args is ignored it seems. My mapping is this: Bay 1-8 Dell H200 Bay 9 VMware Disk Quote Link to comment Share on other sites More sharing options...
snailium Posted May 7, 2021 Author Share #4 Posted May 7, 2021 On 5/3/2021 at 6:21 PM, tvon said: I gave this a shot and it seemed to work but after going through the find.synology tool, it said the VM had no HD attached. If you have configured virtual disk, you'd better using a virtual SATA controller. I'm actually using PCIe passthrough to attach my LSI 9211-8i to the VM, so the VM natively see 10 disks. Quote Link to comment Share on other sites More sharing options...
Grevlingkroken Posted May 26, 2021 Share #5 Posted May 26, 2021 Hi, Just trying to get this working on ESXi 7, but no success. I see you have "Then, use StarWind V2V Converter to covert the .bin to .vmdk" - are you referring to the synoboot.img file from the "modified loader" or any other bin-file that I have missed? Thanks! Quote Link to comment Share on other sites More sharing options...
Marawan Posted May 28, 2021 Share #6 Posted May 28, 2021 This tutorial is meant for DS918+ (Since you're using loader v1.04b and UEFI boot) Have you tried this for DS3615xs/DS3617xs with loader 1.03b in CSM mode? Quote Link to comment Share on other sites More sharing options...
snailium Posted May 31, 2021 Author Share #7 Posted May 31, 2021 On 5/26/2021 at 6:54 AM, Grevlingkroken said: Hi, Just trying to get this working on ESXi 7, but no success. I see you have "Then, use StarWind V2V Converter to covert the .bin to .vmdk" - are you referring to the synoboot.img file from the "modified loader" or any other bin-file that I have missed? Thanks! The ".bin" file was a typo, but it seems I cannot edit the post. Here the ".bin" means synoboot.img you just patched in Step 1a. On 5/28/2021 at 10:46 AM, Marawan said: This tutorial is meant for DS918+ (Since you're using loader v1.04b and UEFI boot) Have you tried this for DS3615xs/DS3617xs with loader 1.03b in CSM mode? I don't think ESXi VM can boot from virtual USB storage in BIOS mode. It has to be in EFI mode. Also, there is no CSM in ESXi's EFI mode. I haven't tried v1.03b loader, but I don't think it will work. Quote Link to comment Share on other sites More sharing options...
flyride Posted May 31, 2021 Share #8 Posted May 31, 2021 8 hours ago, snailium said: I don't think ESXi VM can boot from virtual USB storage in BIOS mode. It has to be in EFI mode. Also, there is no CSM in ESXi's EFI mode. I haven't tried v1.03b loader, but I don't think it will work. Not true (at least with VM built in 6.7), set BIOS mode here: This particular VM is running 1.03b happily with ESXi 7. I would be very surprised if ESXi 7 VM's did not have BIOS option, but if that happens, create the VM with an earlier ESXi version. Quote Link to comment Share on other sites More sharing options...
KeyMag Posted June 4, 2021 Share #9 Posted June 4, 2021 Hi! Can you help with DS3617xs? When i use your method all loader start correct but network doesn't work. ESXi 6.7 Quote Link to comment Share on other sites More sharing options...
snailium Posted June 17, 2021 Author Share #10 Posted June 17, 2021 On 6/4/2021 at 9:25 AM, KeyMag said: Hi! Can you help with DS3617xs? When i use your method all loader start correct but network doesn't work. ESXi 6.7 It sounds the network driver problem. Did you choose "e1000e" for the virtual network adapter? Or maybe try IG-88's extension for additional network driver? Quote Link to comment Share on other sites More sharing options...
snailium Posted June 17, 2021 Author Share #11 Posted June 17, 2021 On 5/31/2021 at 11:00 AM, flyride said: Not true (at least with VM built in 6.7), set BIOS mode here: This particular VM is running 1.03b happily with ESXi 7. I would be very surprised if ESXi 7 VM's did not have BIOS option, but if that happens, create the VM with an earlier ESXi version. It is not the boot option. I mean if you use BIOS mode, there is no support on booting from USB drive. At least, I didn't have it work that way. Maybe you can find some other solution to make the VM boot from USB using BIOS mode. Quote Link to comment Share on other sites More sharing options...
Pantigon Posted August 13, 2021 Share #12 Posted August 13, 2021 (edited) # VMware virtual USB mass storage set vid=0x0e0f set pid=0x0005 May I ask how to get this parameter? i think the vid and pid parameters have been changed on ESXi 7.0 so i can't use the loder file you created. Here is the content of the file .vmx .encoding = "UTF-8" config.version = "8" virtualHW.version = "14" nvram = "XPEnology918.nvram" pciBridge0.present = "TRUE" svga.present = "TRUE" pciBridge4.present = "TRUE" pciBridge4.virtualDev = "pcieRootPort" pciBridge4.functions = "8" pciBridge5.present = "TRUE" pciBridge5.virtualDev = "pcieRootPort" pciBridge5.functions = "8" pciBridge6.present = "TRUE" pciBridge6.virtualDev = "pcieRootPort" pciBridge6.functions = "8" pciBridge7.present = "TRUE" pciBridge7.virtualDev = "pcieRootPort" pciBridge7.functions = "8" vmci0.present = "TRUE" hpet0.present = "TRUE" floppy0.present = "FALSE" RemoteDisplay.maxConnections = "-1" numvcpus = "4" memSize = "4096" bios.bootRetry.delay = "10" firmware = "efi" powerType.powerOff = "default" powerType.suspend = "soft" powerType.reset = "default" tools.upgrade.policy = "manual" sched.cpu.units = "mhz" sched.cpu.affinity = "all" sched.cpu.latencySensitivity = "normal" vm.createDate = "1628805320629596" sata0.present = "TRUE" usb.present = "TRUE" ehci.present = "TRUE" usb_xhci.present = "TRUE" svga.autodetect = "TRUE" serial0.fileType = "file" serial0.fileName = "serial.out" serial0.yieldOnMsrRead = "TRUE" serial0.present = "TRUE" sata0:1.fileName = "XPEnology918.vmdk" sched.sata0:1.shares = "normal" sched.sata0:1.throughputCap = "off" sata0:1.present = "TRUE" ethernet0.virtualDev = "e1000e" ethernet0.networkName = "VM Network" ethernet0.addressType = "static" ethernet0.address = "00:11:32:12:34:56" ethernet0.wakeOnPcktRcv = "FALSE" ethernet0.present = "TRUE" displayName = "XPEnology918" guestOS = "other4xlinux-64" bios.forceSetupOnce = "TRUE" toolScripts.afterPowerOn = "TRUE" toolScripts.afterResume = "TRUE" toolScripts.beforeSuspend = "TRUE" toolScripts.beforePowerOff = "TRUE" tools.syncTime = "FALSE" uuid.bios = "56 4d 3b af 2c 82 0a 64-b0 ed 5e 70 14 f4 99 39" uuid.location = "56 4d 3b af 2c 82 0a 64-b0 ed 5e 70 14 f4 99 39" vc.uuid = "52 21 7d 5a 86 30 d7 c8-a0 ec 4b ee 8b b9 93 dc" sched.cpu.min = "0" sched.cpu.shares = "normal" sched.mem.min = "0" sched.mem.minSize = "0" sched.mem.shares = "normal" usb_xhci:3.present = "TRUE" usb_xhci:3.deviceType = "disk" usb_xhci:3.fileName = "synoboot.vmdk" usb_xhci:3.readonly = "FALSE" Edited August 13, 2021 by Pantigon Quote Link to comment Share on other sites More sharing options...
Marawan Posted August 14, 2021 Share #13 Posted August 14, 2021 (edited) @Pantigon You are looking at the wrong file, You should first read the tutorial Tutorial: Install/Migrate DSM 5.2 to 6.1.x (Jun's loader) That tutorial will tell in details everything you need to get started. Also OP referred you to that tutorial in step 1a above. Edited August 14, 2021 by Marawan Quote Link to comment Share on other sites More sharing options...
Pantigon Posted August 14, 2021 Share #14 Posted August 14, 2021 (edited) @Marawan I used his mod loader file HERE. The problem is that I boot ok but XPEN doesn't seem to get the IP when installing on ESXi 7, but ESXi 6.7 is ok. Are you running it on ESXi 7? Edited August 14, 2021 by Pantigon Quote Link to comment Share on other sites More sharing options...
Marawan Posted August 15, 2021 Share #15 Posted August 15, 2021 7 hours ago, Pantigon said: @Marawan I used his mod loader file HERE. The problem is that I boot ok but XPEN doesn't seem to get the IP when installing on ESXi 7, but ESXi 6.7 is ok. Are you running it on ESXi 7? I use 3615x VM in ESXi 7.02u with passthrough of a NIC and a SAS HBA . I passed both Intel and Realtek types of NICs and both were recognized properly. Have you set your card mac correctly in grub.cfg? set mac1=? May be your problem is that you are reusing an ESXi 6.7 VM in 7 ??!, Try create a new fresh one in 7 and see what happens. Try experiment with a different model with a different loader and see what happens. 1 Quote Link to comment Share on other sites More sharing options...
snailium Posted August 15, 2021 Author Share #16 Posted August 15, 2021 9 hours ago, Pantigon said: @Marawan I used his mod loader file HERE. The problem is that I boot ok but XPEN doesn't seem to get the IP when installing on ESXi 7, but ESXi 6.7 is ok. Are you running it on ESXi 7? Which network adapter are you using? You should use e1000e, not vmxnet3, because DSM doesn't have driver for vmxnet3. 1 Quote Link to comment Share on other sites More sharing options...
Marawan Posted August 15, 2021 Share #17 Posted August 15, 2021 4 hours ago, snailium said: Which network adapter are you using? You should use e1000e, not vmxnet3, because DSM doesn't have driver for vmxnet3. As I said I don't use virtual network adapter e1000et/vmxnet3,, I passthrough the physical PCIe network adapter to Synology VM. That means the VM sees the adapter as is not as e1000et/vmxnet3. You need to enable passthrough in ESXi, go to Host->Manage->Hardware->PCI Devices After you enable the passthrough for the network adapter you need to add it your VM. From VM settings choose Add other device->PCI device and select the network adapter you just enabled passthrough for it. You will not be able to share a passthrough device with other VMs, Only one VM can use it at a time. Quote Link to comment Share on other sites More sharing options...
Pantigon Posted August 15, 2021 Share #18 Posted August 15, 2021 10 hours ago, snailium said: Which network adapter are you using? You should use e1000e, not vmxnet3, because DSM doesn't have driver for vmxnet3. I using e1000e. But it doesn't seem to get the IP. But with your instructions, I run on ESXi 6.7 and it's ok Quote Link to comment Share on other sites More sharing options...
Pantigon Posted August 15, 2021 Share #19 Posted August 15, 2021 (edited) 12 hours ago, Marawan said: I use 3615x VM in ESXi 7.02u with passthrough of a NIC and a SAS HBA . I passed both Intel and Realtek types of NICs and both were recognized properly. Have you set your card mac correctly in grub.cfg? set mac1=? May be your problem is that you are reusing an ESXi 6.7 VM in 7 ??!, Try create a new fresh one in 7 and see what happens. Try experiment with a different model with a different loader and see what happens. you are booting DS3617 as virtual USB synoboot.vmdk or sata synoboot.vmdk? Of course I fixed the card's MAC. I booting DS3167 as sata synoboot.vmdk is ok but virtual USB synoboot.vmdk isn't IP received. And of course my ESXi 6.7 and ESXi 7 on 2 different physical servers. Edited August 15, 2021 by Pantigon Quote Link to comment Share on other sites More sharing options...
snailium Posted August 17, 2021 Author Share #20 Posted August 17, 2021 On 8/15/2021 at 8:40 AM, Pantigon said: I using e1000e. But it doesn't seem to get the IP. But with your instructions, I run on ESXi 6.7 and it's ok I would recommend adding a serial port in the VM options, dump the output to a log file. After booting your VM, keep checking the serial log file for hints (you can access the log file from datastore). It may tell you which driver has issue. P.S. double check "secure boot" option, it needs to remain disabled. 1 1 Quote Link to comment Share on other sites More sharing options...
Pantigon Posted August 17, 2021 Share #21 Posted August 17, 2021 2 hours ago, snailium said: I would recommend adding a serial port in the VM options, dump the output to a log file. After booting your VM, keep checking the serial log file for hints (you can access the log file from datastore). It may tell you which driver has issue. P.S. double check "secure boot" option, it needs to remain disabled. Here is the configuration on ESXi 7.0 U2 Boot ok, but not getting IP.... P/s: Same settings, I run it on ESXi 6.7 server, it works 😀 Quote Link to comment Share on other sites More sharing options...
snailium Posted August 18, 2021 Author Share #22 Posted August 18, 2021 On 8/17/2021 at 4:32 AM, Pantigon said: Here is the configuration on ESXi 7.0 U2 Boot ok, but not getting IP.... P/s: Same settings, I run it on ESXi 6.7 server, it works 😀 Can you share the "serial.out" file under your VM directory? Quote Link to comment Share on other sites More sharing options...
Pantigon Posted August 18, 2021 Share #23 Posted August 18, 2021 5 hours ago, snailium said: Can you share the "serial.out" file under your VM directory? Here is the file you need. serial.out Quote Link to comment Share on other sites More sharing options...
snailium Posted August 18, 2021 Author Share #24 Posted August 18, 2021 It seems grub (bootloader) didn't find any valid boot entry. So, DSM is not loaded. Can you please also share "vmware.log"? Quote Link to comment Share on other sites More sharing options...
Pantigon Posted August 19, 2021 Share #25 Posted August 19, 2021 16 hours ago, snailium said: It seems grub (bootloader) didn't find any valid boot entry. So, DSM is not loaded. Can you please also share "vmware.log"? Here is the file you need. vmware.log Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.