For those who are stuck on file corrupted error (13) during DSM installation step with ESXi, I would like to share my workarounds. There are two things I've missed.
1. Choose SATA mode instead of USB!
USB boot is for baremetal (physical usb thumb drive) or virtual USB emulation for hypervisor. The one we used to choose in the previous Jun's loader is just introduced in redpill by the latest update. So try with SATA mode passing synoboot_satadom=2 in grub.cfg.
2. Check your vmdk
For long time, I have used a pair of files as bootloader, raw disk image file and dummy vmdk file which is pointing actual image file. This was the mistake. To obtain valid vmdk file, I first convert built img file using "qemu-img convert -f raw -O vmdk redpill-DS918+_6.2.4-25556_b1632162635.img boot.vmdk" in ubuntu shell, then again convert "vmkfstools -i boot.vmdk -d thin bootin.vmdk" in ESXi shell. Please let me know better way to do this.
After these two things are resolved, it finishes DSM installation process and finally shows login screen! Thanks!