ibexcentral Posted March 26, 2016 Share #1 Posted March 26, 2016 Hi, I am about to build a VMWare ESXi 5.5 (for VM's) / NAS running XPENology (DSM5.2) and need some advice on the best / standard setup for perfomance and safe storage. I want DSM volumes to be separate from ESXi diskstore so that I can map a network drive from a laptop on the same network and store files on DSM. I was thinking that I would use 2 x 4TB WD Red HDD for DSM volume and have a backup process to sync with the last 1 x 4TB WD Red HDD. Hardware: Asrock AM1H-ITX mini-itx board AMD 5350 CPU 16GB RAM 3 x 4TB WD Red HDD 1 x SDD? (See below) What is the best setup to achieve what I need? Is it possible to have a SDD that has two partitions the first paritition for ESXi only and the second for a datastore to contain a VM for running XPENology / DSM? The other HDD used as DSM volumes. Does ESXi and EXPnology / DSM operate efficiently this way or is there another configruation? that is boot ESXi from USB and use small SDD as the intial datastore to contain VM for running XPEnology / DSM? Any advice would be appreciated. Link to comment Share on other sites More sharing options...
billat29 Posted March 26, 2016 Share #2 Posted March 26, 2016 I think the missing piece is what else you are going to use your ESXi server for. The other VMs you intend to create may affect how you configure your server. ESXi will quite happily boot from a USB drive and that will save you trying to partition your SSD. I am assuming that you will connect your HDDs via RDM and so, as far as DSM is concerned, the datastore doesn't need to be on anything particularly fast as the virtual machine folder will only contain configuration and the xpenology virtual disk (or .iso file). This is only read at boot time, the DSM system itself will be on your HDDs. So I have: USB with ESXi 1 small HDD with a datastore containing the DSM VM and others 3 x HDD for DSM. Link to comment Share on other sites More sharing options...
rhinospray Posted March 26, 2016 Share #3 Posted March 26, 2016 When you install ESXi to any device it will create a handful of small partitions for itself and (except on USB devices) another partition using the rest of the disk, formatted as a datastore. If you are going to dedicate a HD for the datastore, the only practical benefit to install ESXi on a USB drive is that you will save the few GBs it needs for install and a slightly faster boot compared to a normal HD. My advice in your case, is to install ESXi and its datastore in that dedicated HD, as HD are generally more reliable than USB drives. As for the type of disk you need, SSD or not will not make a real difference for ESXi and XPEnoboot, as both are fully loaded in memory and small, so, performance wise, we are talking of an initial delay of a few seconds at startup. I made a test once and I THINK (was time ago) it was less than 2 seconds of difference. However, the other virtual machines that you intend to start will greatly benefit from a SSD as their virtual disks will normally be allocated in the datastore. Last, any reason for ESXi 5.5 when you have v6 freely available? Link to comment Share on other sites More sharing options...
Schnapps Posted March 26, 2016 Share #4 Posted March 26, 2016 1. Why go with ESX 5.5 when 6.0 is out and stable? 2. You have two options in regards to how you present the HDDs to ESX RDM - Raw Device Mapping (or passthrough, skipping ESX, and going directly to the VM) VMFS based - where you present the disks to ESX and format them as VMFS and then create a THICK VMDK disk on each Datastore (basically HDD). ATTENTION: THICK and not THIN. I suggest THICK LAZY ZEROED 3. There are advantages and disadvantages for each method of presenting the disks. In case of RDM, DSM can't monitor the HDDs as no SMART data can be passed-through the hypervisor but you can always take your disks and insert them into a real Synology system or even XPENO and will work. In case of VMFS based disks, SMART will work but you can't "transplant" the HDDs to another DSM. In theory, it will be slower due to the fact that data is being passed through another filesystem, but I can you or anybody else in this forum can even measure the difference 4. Take my advice and use 4 HDDs in a SHR1 volume. You will thank me later. If not, when one of the HDDs will break, you will lose all your data. SHR1 with 4 disks will protect you from a disk failure (any one of them). In 3 years of XPENO use, I lost 2 HDDs but not a single bit of data due to it. 5. You can install ESX on an USB stick. Based on VMware's best practices, a 16GB stick would be recommended for the ESX. The stick will be used just at the boot until the OS (ESX) will be loaded in RAM. 6. The SSD is pretty useless if you install ESX on the USB drive, unless you use as a datastore for some additional VMs that you want to be a bit speedy, like a Windows machine that you can connect via RDP 7. If you don't use an USB drive, you could install ESX on the SSD and the remaining free space from the SSD after installing ESX could be use as a datastore for VMs like DSM and Windows or the Xpenoboot boot disk 8. My DSM runs under Hyper-V for 3 years. Hyper-V because on J1900 ESX cannot be installed (or better said, I didn't know how to make it work at that time) Choose wise Young Padawan! Link to comment Share on other sites More sharing options...
berwhale Posted March 26, 2016 Share #5 Posted March 26, 2016 There's a third option to present HDDs to a VM - you can use DirectPath I/O to pass-through the whole SATA adapter to the VM. The VM will then have complete control over the adapter and any disks attached to it. However, you do need a CPU and motherboard that support DirectPath I/O. I'm currently getting ready to move the 4x 3TB drives from my old server to the T20 in my signature using this method. Link to comment Share on other sites More sharing options...
Schnapps Posted March 26, 2016 Share #6 Posted March 26, 2016 There's a third option to present HDDs to a VM - you can use DirectPath I/O to pass-through the whole SATA adapter to the VM. The VM will then have complete control over the adapter and any disks attached to it. However, you do need a CPU and motherboard that support DirectPath I/O. I was sticking to his config and forgot about pass-through RAID card. Good that you mentioned. Link to comment Share on other sites More sharing options...
sWORDs Posted March 28, 2016 Share #7 Posted March 28, 2016 (edited) edit: double post Edited March 28, 2016 by Guest Link to comment Share on other sites More sharing options...
sWORDs Posted March 28, 2016 Share #8 Posted March 28, 2016 Thick disks aren't really faster then thin anymore (even much slower in 4K writes, especially with flash) also I'd recommend use the correct virtual nic (VMXNET3) and virtual controller (PVSCSI). Passthrough will give you best performance (throughput and lower CPU) and best power savings as the disks can spindown. Best way of doing XPENology on ESXi 6: 1) Boot ESXi from other controller (SATA/PCIe/USB) if using passthrough. SATA/PCIe recommended as you can use that as datastore for other VM's and the XPENology boot vmdk. 2) Passthrough extra SATA controller or the onboard (requires map file edit, nothing else can be on that controller) if using passthrough. 3) Modify image to block IDE and set NIC mac. 4) Modify VM to use VMXNET, PVSCSI (when not using passthrough) and set MAC. How to do step 2 for onboard Intel controller: 1) Login to ssh console and run 'esxcli storage core adapter list' 2) In the description between parenthesis you'll see your pci id, something like: (0000:00:11.4) Intel Corporation Wellsburg AHCI Controller 3) Run lspci -n |grep 0000:00:11.4 4) After the last colon you'll see your pid, something like 0000:00:11.4 Class 0106: 8086:8d62 [vmhba0] 5) Now put '8086 [PID] d3d0 false' at the bottom of /etc/vmware/passthru.map, my full file: # passthrough attributes for devices # file format: vendor-id device-id resetMethod fptShareable # vendor/device id: xxxx (in hex) (ffff can be used for wildchar match) # reset methods: flr, d3d0, link, bridge, default # fptShareable: true/default, false # Intel 82598 10Gig cards can be reset with d3d0 8086 10b6 d3d0 default 8086 10c6 d3d0 default 8086 10c7 d3d0 default 8086 10c8 d3d0 default 8086 10dd d3d0 default # Broadcom 57710/57711/57712 10Gig cards are not shareable 14e4 164e default false 14e4 164f default false 14e4 1650 default false 14e4 1662 link false # Qlogic 8Gb FC card can not be shared 1077 2532 default false # LSILogic 1068 based SAS controllers 1000 0056 d3d0 default 1000 0058 d3d0 default # NVIDIA 10de ffff bridge false # Intel Wellsburg AHCI 8086 8d62 d3d0 false 8086 8d02 d3d0 false 6) Reboot, put device in passthrough, reboot and attach to VM. For step 3 see 'Modify the bootimage' here: https://idmedia.no/projects/xpenology/installing-or-upgrading-to-xpenology-5-2-or-later/ In your case I'd use an extra SATA controller (LSI based) as you don't have an Intel controller. Link to comment Share on other sites More sharing options...
supernoman Posted March 28, 2016 Share #9 Posted March 28, 2016 BTW, I have that CPU in my old NAS. It's great for NAS but would probably be pretty bad/slow as an ESXi CPU running multiple VMs. Link to comment Share on other sites More sharing options...
Recommended Posts