Tuatara
-
Posts
60 -
Joined
-
Last visited
Posts posted by Tuatara
-
-
Hi everyone and especially thanks to Tuatara!
I only made the guide - others made the modifications, so thanks go to all who made it possible.
Unfortunately this is the trade-off. The PVSCSI driver does not support SMART communications. If you absolutely need SMART, then you must go the VMDirectPathIO route and map your drives that way. But, since you're using RDM, you can easily switch between VMDirectPathIO and RDM for your datastores. Easy-as!So if i well understand, I should switch the RDM paravirtual to VMDirectPathIO - which shouldn't affect performances but will allow HDD to rest with SMART. And so reduce power consumption and hopefully extend their lifetime.
I'm a total newbie with ESXI and actually don't know how to do that...
If you absolutely need SMART, then you will need to go VMDirectPathIO. This requires VT-d extensions within ESXi, which permits you to map the physical hardware into the Virtual Machine. i.e. you map the physical PCI device into the VM Configuration.
AFAIK drive power-down is still handled by the PVSCSI driver, but I'm not sure though as I don't power them down - I'm using WD REDs (RAID/NAS Drives). If you use WD GREENs, they should auto-power down when not accessed and in that case you have no need for SMART - except to find out drive parameter status (lifetime) or temperature.
first question: will all the settings and installed packages be running without having to reinstall everything if i switch to VMDirectPathIO ?Yes, because in both cases the Synology would see the Hard Drive as a physical hard drive (raw information) if you are using a RDM. If you're using a VMDK (Virtual Disk), then that sitting on top of the VMFS Filesystem like a single file would in a FAT, FAT32, NTFS, EXT3, EXT4, etc. filesystem, and that needs to be virtualised by ESXi.
So when switching from RDM to VMDirectPathIO - you still have the physical disk contents being exactly the same.
second question: would you have some documentation to share or would like to update your idiot's guide to help us setting that VMDirectPathIO?If your ESXi supports PCI passthrough, then you'll have the option to assign PCI Cards to your VM. This requires a number of hardware devices to be compatible, (processor, motherboard, bios, pci card, etc.). If and only if all those are compatible, and the PCI card you want to pass through to the Synology is recognised by it (and it's hardware drivers), then you're good to go. I had to change out a fair few hardware controllers before I found one which was stable and worked in VMDirectPathIO mode. My advice, use RDM unless you really, really need SMART monitoring.
-
OK there is almost certainly a bug in this version where your storage device does not persist a reboot.
The VMDK is still attached but if you go into Storage Manager, there are no volumes.
I'm not sure what's happening for you, but the RDMs I'm using are all fine and survive reboots without any issues. Potentially there is something in your ESXi installation at issue. Are you using any 3rd party drivers on your ESXi whitebox? Anything special in your configuration or hardware?
-
If you can successfully install ESXi on the machine (USB keys are perfect for this) ...
Not sure about the SD Slot, but you probably can map that to another VM through VMDirectPathIO.
You can install ESXi to SD card (>=2GB) instead of USB. Its there specifically for embedded installs.
Learn something every day. I wasn't sure if ESXi supported SD cards - good to know!
P.S> I use these USB drives for all my ESXi installations. They're almost invisible, and you won't have anyone pulling them out accidentally.
-
Details are also included about creating RDM VMDK disk images instead of VMFS based virtual disks.
Hello Tuatara,
thanks for your guide.
No worries ... glad it's helping.
I try to install dsm following your recommendations, but before you start the job, I would like to know if my hardware is compatible with your guideIf you can successfully install ESXi on the machine (USB keys are perfect for this), and all physical devices are recognized (CPU, RAM, Perc controller, etc.) then yes - you will be fine.
Making the RDM files - yes - just follow the instructions but substitute your drives obviously!
BTW: That would be one *kick-ass* machine! Not sure about the SD Slot, but you probably can map that to another VM through VMDirectPathIO.
-
Any modifications done via SSH (e.g. enabling ipkg) do not persist a reboot
All my Synology apps (e.g. Audio Station) do not persist a reboot
Just done a clean install and things seem OK now.
I'm a little concerned about why my storage drive disappeared (that's what caused all my apps and stuff to disappear). The drive was visible in DSM but there were no mounted volumes and it wanted to format the disk to create them...
How did you shut down the VM or perform the reboot? Do you have the drives mapped as RDM or as VMDK? Depending on how you shut down the host (don't do it through vSphere!) there may be cached files which are not written to disk by the time the VM is powered off. Think of it as "pulling the power" on the machine.
-
The SCSI controller seems to always change to LSI Logic when the HDD is removed and re-added...
I think the documentation needs updating to reflect this.
Unfortunately this is a "known" bug ... something I just automatically do. It has something to do with adding the virtual hardware to the machine ... sometimes the virtualized drivers "stick", but sometimes they don't. No idea why. I guess I could add that into the "rev 2" of the Guide ... sometime.
-
Thanks a lot for the idiot's guide !
Following it, i've been able to install dsm 4.2 ; everything works perfectly!
You're welcome. Glad it's been working for everyone.
I created 2 RDM for my 2 physical HD's.Using SCSI in paravirtual and HD as Mapped Raw LUN
Could you eventually confirm it's the best way?
From my testing the difference between using a VMDirectPathIO mapped controller and a Paravirtual RDM is negligible. I was unable to find any real difference in performance, and would estimate the performance difference is under 1% - well within my experimental error in testing. Stick with RDM, since you can then use ANY disk controller supported under ESXi without any driver changes/additions required for the Synology. It would also be much more stable IMHO.
On DSM, in Storage management, i see the 2 HD's without SMART :sDoes anyone knows a solution to allow SMART?
Unfortunately this is the trade-off. The PVSCSI driver does not support SMART communications. If you absolutely need SMART, then you must go the VMDirectPathIO route and map your drives that way. But, since you're using RDM, you can easily switch between VMDirectPathIO and RDM for your datastores. Easy-as!
-
Unfortunatly it's not working. Getting the same messages in the log:
Jun 6 09:49:38 s2s_monitor: synotify_add_watch error: Function not implemented path:[/volume1/Test2] Jun 6 09:49:38 s2s_monitor: synotify_add_watch error: Function not implemented path:[/volume1/Test2/synctest]
[update]
After testing above I did anoher test.
I setup the sync the other way around. So ds412+ to sync to XPEnology.
Created the sync set and set it to automatically sync when there is a change.
This works without problems.
Ok ... this means that the synotify_add_watch error has been removed from the ESXi patches. If you're happy having the sync operate the other way, then you have a solution.
I'll see if I can find the function call/stub/omission in jukolaut's code updates, but I don't have the time right now to do so. Maybe when the 3211 version GPL code is available.
-
I have question if I can run DSM on VM workstation whether it implies can be run on ESXi?
Ummm ... I don't understand the question.
This entire thread is about running DSM in ESXi. By re-reading the thread, you'll find all the information you need to run DSM on ESXi. You need a different set of installation files for ESXi than what is needed for VMWare Workstation or VirtualBox or Hyper-V or for a physical machine (HP N40, etc.)
For ESXi, read (and follow exactly) the "Idiot's Guide" to install on ESXi.
Idiot's Guide to DSM 4.2 and ESXi 5.1
For other VMWare or VirtualBox installations, check other forum threads. The best one would probably be "XPEnology for dummies LINK Update 02/02/2013".
-
Guys is someone experiencing the same problem as I have?
I installed the build as per Idiot's Guide. Running fine but I do have one problem.
I setup Shared Folder Sync on my XPEnology and it syncs to my real Synology DS412+ at home.
I have set the configuration of Shared Folder Sync to sync files whenever the folder changes but nothing happens.
The initial sync works, so the sync itself is working, but after the initial sync it does not see anything that gets changed.
I checked the logs on my XPEnology and saw this error:
Jun 5 22:34:09 s2s_monitor: synotify_add_watch error: Function not implemented path:[/volume1/photo/Foto Sync]
Is this a problem with this build or with XPEnology?
Or is it something else?
"synotify_add_watch" with a path sounds suspiciously like the "inotify_add_watch" function ( Reference ) in Linux. It's possible that Synology has extended this function to support their special disk configurations (SHR?), and this support is not available (or not present) in the PVSCSI drivers being used for VMWare/ESXi. In which case, it will take some reverse engineering to resolve.
Alternatively, there could be an error in the DSM Shared Folder Sync functionality. The first thing that comes to mind is the [space] in the directory name. It's possible that there is an error in the code, which has been corrected/updated in the newer DSM version. For reference, check out my post above:
Synology DiskStation DS3612xs Release Notes Version: DSM 4.2-3211 (2013/04/18) - What's New Improved stability when syncing files between two DiskStations with Shared Folder Sync.
My first attempt at a fix, would be to remove the [space] in the directory name and try the Shared Folder Sync configuration/setup again. Gut feel says that this has a high chance of working.
-
I have seen dsm 4.2 3211 was released. Does anyone know how to upgrade the 3202 version ?
First off, don't try to upgrade through DSM itself - you will overwrite all modifications which were made to 3202 (kernel, drivers, etc.) which were done in order to get it to run on ESXi.
The changes for DSM 4.2 3211 sound significant enough (don't they always), that an update will probably be coming ... from someone ... sometime. Patience. ( DSM 4.2 3211 Release Notes )
Synology have also not as yet released the GPL sources to Sourceforge ( Synology GPL Source ). Since jukolaut has posted his modifications, it would only be a matter of time before the next version is created.
Remember ... this DSM code isn't a "free NAS software solution" - it's hacked/modified code, "unreliable" due to that fact, coming with no support, warranty, or remedies. If your system hiccups, crashes, fails, loses all your data, goes up in flames, etc. ... you are on your own ... entirely on your own. This code was originally intended to run only on dedicated Synology hardware. Synology are proving themselves to be a fantastic Open Source company - publishing the GPL portions of their code in a very timely fashion. This dedication to F/OSS is not to be abused.
For me, the ability to virtualize DSM has permitted me to spec business requirements, create a virtualized environment to suit the assumptions, check load & usage, and then deploy the needs-appropriate, fully supported, actual Synology hardware. Benefit === incalculable.
Synology DiskStation DS3612xs Release Notes Version: DSM 4.2-3211 (2013/04/18) - What's New Fixed an issue where users could not connect to the DiskStation via FTP over SSL. Fixed an issue which kept the DiskStation from connecting to SNMP (Simple Network Management Protocol) UPS devices. Fixed an issue where domain users could not obtain an emergency code for 2-step verification. Improved the compatibility of file sharing via SMB/CIFS with Mac computers. Improved the stability of generating Disk Usage Reports. Improved the stability of iSCSI LUN. Improved the compatibility when backing up data between different versions of DSM. Improved stability when syncing files between two DiskStations with Shared Folder Sync. Improved the stability when backing up data to another DiskStation with Time Backup. Other issue fixes and enhancements.
-
... I've packaged the kernel source code ... includes the source to the fake synobios module (in drivers/synobios/synobios.c) and other modifications ...
You rock. 'Nuff sed.
Cheers & Thank you.
-
Thanks Tuatara,
I finally upgraded to Esxi 5.1 and it works fine.
One more question : is it possible to upgrade to recent version of DSM ( DSM 4.2 -3202 towards DSM 4.2 -3211 ) ??
Thanks a lot
Good to hear. As for the upgrade, I don't believe this is possible without replacing the kernel & changed files before importing the Synology .pat file. Using the actual files will remove the PVSCSI driver/etc. and the VM will no longer boot. Stick with 3202 until an updated version is released. I'm hoping jukolaut will return to the forum and provide the sources for his modifications.
-
Unfortunately no change, the SHR rebuild is done, this is the only VM currently running, CPU is very low, and I'm still loosing around 1 packet every 5 (see below, copy/paste is in French but could be understandable).
French, German, English ... all good here.
From the dump it looks like exactly 1 packet every 5. This looks very suspicious.
Have you got the "Denial of service (DoS) protection" enabled on the DiskStation? If so, then the DiskStation is deliberately dropping the packets to prevent a flood. This would be by design.
-
Hi johndoe,
I don't believe the current build can be installed on ESXi 4.1, as the PVSCSI driver has been built against ESXi 5.x. I can confirm that it will successfully install on ESXi 5.0 (with or without updates), ESXi 5.1 (with or without updates). Unfortunately I do not have an ESXi 4.1 installation available to test this on.
My recommendation - give it a try. It only takes about 15 minutes to get it all set up. If the VM boots, then there is a good chance everything will work. Failing that ... upgrade the box to ESXi 5.x and make use of the increased functionality/stability which the major version jump offers.
Regards,
Tuatara
-
I've a strange behavior with my xpenology VM running on ESXi 5.1 (on a Microserver N40L) in my lab.
When pinging the IP address of this VM from my LAN, and this VM _only_, I've around 25 to 30% of packets loss (all the other VMs share the same ethernet interface, and only the IP address associated to the xpenology VM has this problem).
Except the packet loss, everything else seems to run as expected, no real problem, pretty good performance (between 60 to 70 MB per second using FTP, even if the volume is currently expending from 2 to 5 virtual disks in SHR mode), but this sounds strange.
Does anyone else notice this ?
My bet is that the CPU shares for the Synology instance are limiting the overall performance. If the CPU (virtual) is maxed out due to the SHR rebuild, then there will be little (if any) horsepower available for anything else. It's just too busy to service all requests.
Now, the reason it's not dropping anything else is due to how TCP is designed. A ping is a UDP packet - it is sent with no expectation of a reply. A FTP connection (or NFS, Samba, etc.) are TCP connections, which by design expect a reply. If there was no reply, the packet is re-sent. This guarantees the communications stream. So basically, the Linux OS on the Synology is dropping the Ping UDP since it is of the lowest priority and can be dropped without incident/issue.
I'm certain that once the rebuild is done, you'll notice your transfer speeds approach 100MB/s (Gb LAN speed) and there will be no dropped pings. Alternatively, allocate a greater share of the virtual resources to the Synology VM - more CPU will help this issue.
-
Is there a bootstrap available ?
I've checked http://packages.quadrat4.de/, but this model is not supported ?
From this german forum, it's apparently fine to use the Atom bootstrap. You would then need to do the process manually (Info Here).
The other "possible" option is that we change the model which the XPEnology thinks it is - something smaller ... but then "drives would be lost".
-
Thanks, I forgot it added a new controller when adding the data disk ...
The disk is now found, however, "an error has occured" when installing the image
EDIT: fixed ! data disk was too small ...
Fantastic - great to hear it's now sorted.
For everyone else - yes, the guide is proven and tested. Follow it, and it will work every time.
-
Hi jukolaut, can you upload the modified sources? I'm also busy getting open-vm-tools compiled and packaged through the IPKG Optware repository.
Thanks!
Sweet! If you do get it compiled, having open-vm-tools installable through IPKG would be perfect! I'm going to give it a go shortly ... setting up the development environment soon.
jukolaut - if you can provide your modified sources - I've already downloaded the Synology GPL sources and will look into the modifications required myself. I'll continue jukolaut's work and remove the remaining SYNOIO_GET_xxxx ioctl calls which are being logged and see if I can "simply" integrate a few other VM friendly elements.
-
Guys!
Following the guide "to the letter" means choosing all the same options!Post or PM me your VMX file (about 3Kb in size) ... I'm sure I'll then easily spot the configuration difference.
I did choose that option ...
I'll send it tonight when I get home.
I'm having no problems with otherVM's though ...
vmx File: link
Hey devastator, you'll be devastated to hear ... you've got the "lsilogic" controller in your configuration, not the "pvscsi" controller. You will need to power down the VM, and then change the SCSI controller type. Exactly the same issue as Portnawak had. Do this, and it should fix your problem.
Additionally, you have not reserved the memory for the Synology OS. I'd recommend reserving the memory (dedicated), as a real Synology box does not have virtualised virtual memory. I have no idea how much performance will/would suffer when the ESXi kernel swaps out the memory for the VM, and then you make a iSCSI/NFS/SAMBA request and it has to swap all the memory back in.
Other drive related matters - you additionally have added in a CD-ROM, and left the floppy drive enabled, but not connected - so I "think" that's ok ... but a Synology box doesn't have either device.
To further improve your VM, you can change the network card to use the network address "00:11:32:08:D6:2A" which is a real Synology MAC address, and the one used internally/recognised by the Synology itself. Not sure if this will "clear up" any ARP/Network issues which may arise.
Hope that helps and gets your XPEnology up and running.
-
Guys!
Following the guide "to the letter" means choosing all the same options!I did choose that option ...
Post or PM me your VMX file (about 3Kb in size) ... I'm sure I'll then easily spot the configuration difference.
Otherwise, the only other option is that the machine you're running ESXi on has some other failure (hardware, or virtualization failure).
-
Followed the guide to the letter, but when connecting to the Synology, it says it can not find any hard drive ...
Any thoughts about this? I'm using a VMFS disk.
Had the same problem until I changed the SCSI controler type to Paravirtual, and then, it worked (ESXi 5.1 on a Microserver N40L, and a VMFS disk as well - just testing...)
Guys!
Following the guide "to the letter" means choosing all the same options! -
Thanks for the guide
I've followed it to the letter (the only thing I hadn't done first time around was use a PV adaptor.
The thing boots and is recognised by the assistant but when I connect I get the background come up but after a minute or so I just get an "Unable to connect" error message.
Seems to be using a lot of CPU to say it isn't doing anything.
Did you install DSM, and the timeout is in the manager?
Or are you timing out pre-install of the pat file?
What ESXi hardware are you running this on? Intel / AMD, memory, disk, motherboard, etc.
Are there any other VMs on the same machine?
Are you using a VMFS Disk or a RDM Disk? You may have faulty hardware (fails to virtualise).
Follow the guide, use a VMFS Disk first to test that it installs and works, then delete/disconnect the disk and redo it properly with the desired Drive. All else being the same, the guide setup will work, since it is purely virtual.
-
As requested, I've created an Idiot's Guide document which details the installation steps and options required in order to install DSM 4.2 on ESXi 5.1. A complete screen-shot example of all configuration steps is provided, with a fully virtual configuration for testing purposes. Details are also included about creating RDM VMDK disk images instead of VMFS based virtual disks.
Thanks again go to jukolaut and odie82544 for making DSM 4.2 on ESXi 5.1 possible.
http://depositfiles.com/files/virzefc1a[/url]
[SOLVED] Install on vmware ESXI
in DSM 5.2 and earlier (Legacy)
Posted
I didn't have anything to do with the original patches, but due to the nature of my work I am very interested in keeping the ESXi hardware virtualization of Synology DSM 4.x up to date and current. I'll have a look at jukolaut's work - some century when I have "free time" [what's that?
] and see what is involved in updating the kernel/patches and making a new PAT file. No promises ... but being interested will keep this on the front of my 'To Do List'.
If anyone else is keen - AFAIK all the details are available on the site - for skillset - Linux kernel building, Binary Hex File editing, and some basic C code to stub out/handle driver calls ... and you should be good to go.