frank_zero

[SOLVED] Install on vmware ESXI

Recommended Posts

Hi,

 

I've a strange behavior with my xpenology VM running on ESXi 5.1 (on a Microserver N40L) in my lab.

 

When pinging the IP address of this VM from my LAN, and this VM _only_, I've around 25 to 30% of packets loss (all the other VMs share the same ethernet interface, and only the IP address associated to the xpenology VM has this problem).

 

Except the packet loss, everything else seems to run as expected, no real problem, pretty good performance (between 60 to 70 MB per second using FTP, even if the volume is currently expending from 2 to 5 virtual disks in SHR mode), but this sounds strange.

 

Does anyone else notice this ?

Share this post


Link to post
Share on other sites
I've a strange behavior with my xpenology VM running on ESXi 5.1 (on a Microserver N40L) in my lab.

 

When pinging the IP address of this VM from my LAN, and this VM _only_, I've around 25 to 30% of packets loss (all the other VMs share the same ethernet interface, and only the IP address associated to the xpenology VM has this problem).

 

Except the packet loss, everything else seems to run as expected, no real problem, pretty good performance (between 60 to 70 MB per second using FTP, even if the volume is currently expending from 2 to 5 virtual disks in SHR mode), but this sounds strange.

 

Does anyone else notice this ?

 

My bet is that the CPU shares for the Synology instance are limiting the overall performance. If the CPU (virtual) is maxed out due to the SHR rebuild, then there will be little (if any) horsepower available for anything else. It's just too busy to service all requests.

 

Now, the reason it's not dropping anything else is due to how TCP is designed. A ping is a UDP packet - it is sent with no expectation of a reply. A FTP connection (or NFS, Samba, etc.) are TCP connections, which by design expect a reply. If there was no reply, the packet is re-sent. This guarantees the communications stream. So basically, the Linux OS on the Synology is dropping the Ping UDP since it is of the lowest priority and can be dropped without incident/issue.

 

I'm certain that once the rebuild is done, you'll notice your transfer speeds approach 100MB/s (Gb LAN speed) and there will be no dropped pings. Alternatively, allocate a greater share of the virtual resources to the Synology VM - more CPU will help this issue.

Share this post


Link to post
Share on other sites

My bet is that the CPU shares for the Synology instance are limiting the overall performance. If the CPU (virtual) is maxed out due to the SHR rebuild, then there will be little (if any) horsepower available for anything else. It's just too busy to service all requests.

 

Now, the reason it's not dropping anything else is due to how TCP is designed. A ping is a UDP packet - it is sent with no expectation of a reply. A FTP connection (or NFS, Samba, etc.) are TCP connections, which by design expect a reply. If there was no reply, the packet is re-sent. This guarantees the communications stream. So basically, the Linux OS on the Synology is dropping the Ping UDP since it is of the lowest priority and can be dropped without incident/issue.

 

I'm certain that once the rebuild is done, you'll notice your transfer speeds approach 100MB/s (Gb LAN speed) and there will be no dropped pings. Alternatively, allocate a greater share of the virtual resources to the Synology VM - more CPU will help this issue.

 

Unfortunately no change, the SHR rebuild is done, this is the only VM currently running, CPU is very low, and I'm still loosing around 1 packet every 5 (see below, copy/paste is in French but could be understandable).

 

When I try to reboot the VM, as soon as the network starts, it's OK during several few seconds, and probably after one of the services starts (still have to identify which one), same problem occurs again.

 

Does anyone else here run this with ESXi 5.1 on a Microserver N40L, and could let me know if I'm really the only one ?

 

Thanks in advance.

 

Délai d'attente de la demande dépassé.

Réponse de 192.168.1.21 : octets=32 temps<1ms TTL=64

Réponse de 192.168.1.21 : octets=32 temps<1ms TTL=64

Réponse de 192.168.1.21 : octets=32 temps<1ms TTL=64

Réponse de 192.168.1.21 : octets=32 temps<1ms TTL=64

Délai d'attente de la demande dépassé.

Réponse de 192.168.1.21 : octets=32 temps=2 ms TTL=64

Réponse de 192.168.1.21 : octets=32 temps=4 ms TTL=64

Réponse de 192.168.1.21 : octets=32 temps=2 ms TTL=64

Réponse de 192.168.1.21 : octets=32 temps=1 ms TTL=64

Délai d'attente de la demande dépassé.

Réponse de 192.168.1.21 : octets=32 temps=1 ms TTL=64

Réponse de 192.168.1.21 : octets=32 temps=7 ms TTL=64

Réponse de 192.168.1.21 : octets=32 temps<1ms TTL=64

Réponse de 192.168.1.21 : octets=32 temps<1ms TTL=64

Délai d'attente de la demande dépassé.

Réponse de 192.168.1.21 : octets=32 temps=2 ms TTL=64

Réponse de 192.168.1.21 : octets=32 temps<1ms TTL=64

Réponse de 192.168.1.21 : octets=32 temps=17 ms TTL=64

Réponse de 192.168.1.21 : octets=32 temps<1ms TTL=64

Délai d'attente de la demande dépassé.

Réponse de 192.168.1.21 : octets=32 temps=1 ms TTL=64

Réponse de 192.168.1.21 : octets=32 temps=3 ms TTL=64

Réponse de 192.168.1.21 : octets=32 temps=1 ms TTL=64

Réponse de 192.168.1.21 : octets=32 temps=3 ms TTL=64

Délai d'attente de la demande dépassé.

Réponse de 192.168.1.21 : octets=32 temps=1 ms TTL=64

Réponse de 192.168.1.21 : octets=32 temps<1ms TTL=64

Réponse de 192.168.1.21 : octets=32 temps<1ms TTL=64

Réponse de 192.168.1.21 : octets=32 temps<1ms TTL=64

Délai d'attente de la demande dépassé.

Réponse de 192.168.1.21 : octets=32 temps=1 ms TTL=64

Réponse de 192.168.1.21 : octets=32 temps=2 ms TTL=64

Réponse de 192.168.1.21 : octets=32 temps<1ms TTL=64

Réponse de 192.168.1.21 : octets=32 temps=1 ms TTL=64

Délai d'attente de la demande dépassé.

Réponse de 192.168.1.21 : octets=32 temps=1 ms TTL=64

Réponse de 192.168.1.21 : octets=32 temps=1 ms TTL=64

Réponse de 192.168.1.21 : octets=32 temps=1 ms TTL=64

Réponse de 192.168.1.21 : octets=32 temps<1ms TTL=64

Délai d'attente de la demande dépassé.

Réponse de 192.168.1.21 : octets=32 temps=3 ms TTL=64

Réponse de 192.168.1.21 : octets=32 temps=2 ms TTL=64

Réponse de 192.168.1.21 : octets=32 temps=1 ms TTL=64

Réponse de 192.168.1.21 : octets=32 temps=1 ms TTL=64

Délai d'attente de la demande dépassé.

Share this post


Link to post
Share on other sites
Unfortunately no change, the SHR rebuild is done, this is the only VM currently running, CPU is very low, and I'm still loosing around 1 packet every 5 (see below, copy/paste is in French but could be understandable).

 

French, German, English ... all good here. :smile:

 

From the dump it looks like exactly 1 packet every 5. This looks very suspicious.

 

Have you got the "Denial of service (DoS) protection" enabled on the DiskStation? If so, then the DiskStation is deliberately dropping the packets to prevent a flood. This would be by design.

Share this post


Link to post
Share on other sites
Unfortunately no change, the SHR rebuild is done, this is the only VM currently running, CPU is very low, and I'm still loosing around 1 packet every 5 (see below, copy/paste is in French but could be understandable).

 

French, German, English ... all good here. :smile:

 

From the dump it looks like exactly 1 packet every 5. This looks very suspicious.

 

Have you got the "Denial of service (DoS) protection" enabled on the DiskStation? If so, then the DiskStation is deliberately dropping the packets to prevent a flood. This would be by design.

Real diskstation doesn't do it with or without DoS enabled

 

Pinging 192.168.250.10 with 32 bytes of data:

Reply from 192.168.250.10: bytes=32 time<1ms TTL=64

Reply from 192.168.250.10: bytes=32 time<1ms TTL=64

Reply from 192.168.250.10: bytes=32 time<1ms TTL=64

Reply from 192.168.250.10: bytes=32 time<1ms TTL=64

Reply from 192.168.250.10: bytes=32 time<1ms TTL=64

Reply from 192.168.250.10: bytes=32 time<1ms TTL=64

Reply from 192.168.250.10: bytes=32 time<1ms TTL=64

Reply from 192.168.250.10: bytes=32 time<1ms TTL=64

Reply from 192.168.250.10: bytes=32 time<1ms TTL=64

Reply from 192.168.250.10: bytes=32 time<1ms TTL=64

Reply from 192.168.250.10: bytes=32 time<1ms TTL=64

Reply from 192.168.250.10: bytes=32 time<1ms TTL=64

Reply from 192.168.250.10: bytes=32 time<1ms TTL=64

Reply from 192.168.250.10: bytes=32 time<1ms TTL=64

Reply from 192.168.250.10: bytes=32 time<1ms TTL=64

Reply from 192.168.250.10: bytes=32 time<1ms TTL=64

Reply from 192.168.250.10: bytes=32 time<1ms TTL=64

Reply from 192.168.250.10: bytes=32 time<1ms TTL=64

Reply from 192.168.250.10: bytes=32 time<1ms TTL=64

 

Ping statistics for 192.168.250.10:

Packets: Sent = 19, Received = 19, Lost = 0 (0% loss),

Approximate round trip times in milli-seconds:

Minimum = 0ms, Maximum = 0ms, Average = 0ms

Some activity from ESXi is interfering with network.

Share this post


Link to post
Share on other sites
From the dump it looks like exactly 1 packet every 5. This looks very suspicious.

Have you got the "Denial of service (DoS) protection" enabled on the DiskStation? If so, then the DiskStation is deliberately dropping the packets to prevent a flood. This would be by design.

 

Very suspicious indeed, yes, 1 packet every 5.

 

I've tried _many_ things, by stopping almost _all_ services, changing the IP address, making sure DOS protection was not enabled, moving the Microserver to another switch, changing the virtual NIC within ESXi as well as the physical NIC (I've got an Intel dual Gig also in the Microserver) and ... no change.

 

I finaly gave up, deleted the Xpenology VM, created it again (with 4 VMDK for the time being and not yet 5 but that should not change anything), and now it rocks ! :smile: (I'm afraid that I'll never understand what happened...)

 

Thanks for the tips anyway.

Share this post


Link to post
Share on other sites

hi again, sorry for my absence! Noticed that my esxi install image has been downloaded over 26000 times, I thought it would interest about 10 people in the world :razz:

 

As my system running that is stable I'm not that keen on doing new versions but that might change when synology releases a new major version. Unfortunately it seems that upgrading an exising installation will be difficult.

 

I've packaged the kernel source code here: http://yadi.sk/d/rqFVaQhb4yFtu The package includes the source to the fake synobios module (in drivers/synobios/synobios.c) and other modifications described in this post.

 

happy hacking!

Share this post


Link to post
Share on other sites
... I've packaged the kernel source code ... includes the source to the fake synobios module (in drivers/synobios/synobios.c) and other modifications ...

You rock. 'Nuff sed.

 

Cheers & Thank you.

Share this post


Link to post
Share on other sites

Hello,

 

I would like to first thank you because i just built an HTPC on ESXi 5.1 and mounted dsm 4.2 3202 on it.

It works like a charm :smile:

 

I have seen dsm 4.2 3211 was released. Does anyone know how to upgrade the 3202 version ?

 

Thanks.

Share this post


Link to post
Share on other sites
I have seen dsm 4.2 3211 was released. Does anyone know how to upgrade the 3202 version ?

First off, don't try to upgrade through DSM itself - you will overwrite all modifications which were made to 3202 (kernel, drivers, etc.) which were done in order to get it to run on ESXi.

 

The changes for DSM 4.2 3211 sound significant enough (don't they always), that an update will probably be coming ... from someone ... sometime. Patience. ( DSM 4.2 3211 Release Notes )

 

Synology have also not as yet released the GPL sources to Sourceforge ( Synology GPL Source ). Since jukolaut has posted his modifications, it would only be a matter of time before the next version is created.

 

Remember ... this DSM code isn't a "free NAS software solution" - it's hacked/modified code, "unreliable" due to that fact, coming with no support, warranty, or remedies. If your system hiccups, crashes, fails, loses all your data, goes up in flames, etc. ... you are on your own ... entirely on your own. This code was originally intended to run only on dedicated Synology hardware. Synology are proving themselves to be a fantastic Open Source company - publishing the GPL portions of their code in a very timely fashion. This dedication to F/OSS is not to be abused.

 

For me, the ability to virtualize DSM has permitted me to spec business requirements, create a virtualized environment to suit the assumptions, check load & usage, and then deploy the needs-appropriate, fully supported, actual Synology hardware. Benefit === incalculable.

 

Synology DiskStation DS3612xs Release Notes
Version: DSM 4.2-3211
(2013/04/18) - What's New

Fixed an issue where users could not connect to the DiskStation via FTP over SSL.
Fixed an issue which kept the DiskStation from connecting to SNMP (Simple Network Management Protocol) UPS devices.
Fixed an issue where domain users could not obtain an emergency code for 2-step verification.
Improved the compatibility of file sharing via SMB/CIFS with Mac computers.
Improved the stability of generating Disk Usage Reports.
Improved the stability of iSCSI LUN.
Improved the compatibility when backing up data between different versions of DSM.
Improved stability when syncing files between two DiskStations with Shared Folder Sync.
Improved the stability when backing up data to another DiskStation with Time Backup.
Other issue fixes and enhancements.

Share this post


Link to post
Share on other sites
As requested, I've created an Idiot's Guide document which details the installation steps and options required in order to install DSM 4.2 on ESXi 5.1. A complete screen-shot example of all configuration steps is provided, with a fully virtual configuration for testing purposes. Details are also included about creating RDM VMDK disk images instead of VMFS based virtual disks.

 

Thanks again go to jukolaut and odie82544 for making DSM 4.2 on ESXi 5.1 possible.

 

Idiot's Guide to DSM 4.2 and ESXi 5.1.docx - http://depositfiles.com/files/virzefc1a

 

Thanks for the guide!

I have been running my build on ESXi without any problems thanks to your guide! :mrgreen:

Share this post


Link to post
Share on other sites
Ok, do you need sources or binaries?

 

The binaries are here(v2): http://yadi.sk/d/fTRkFMyU3D8Yc

 

This is based on odie82544's DS3612xs_3202-Repack images, so thanks for those! Only the kernel, modules and linuxrc.syno are modified so far.

 

The archive contains both the .pat file for installation and .vmdk file for boot.

 

Create the virtual machine to esxi and add hardware:

  • IDE controller with single harddisk (vmdk file in archive). Boot from this.
  • PVSCSI controller and raw harddisks or vmdk files for data. I've tested with 3 disks so far.
  • VMXNET3 network. Only single interface in bridged mode is tested so far. MAC address can be anything.

 

Follow the installation instructions elsewhere in this forum to install the pat file.

 

I wouldn't use this for anything else than evaluation yet.

 

Great work, Jukolaut, thank you very much for sharing it.

Share this post


Link to post
Share on other sites

Guys is someone experiencing the same problem as I have?

 

I installed the build as per Idiot's Guide. Running fine but I do have one problem.

I setup Shared Folder Sync on my XPEnology and it syncs to my real Synology DS412+ at home.

I have set the configuration of Shared Folder Sync to sync files whenever the folder changes but nothing happens.

The initial sync works, so the sync itself is working, but after the initial sync it does not see anything that gets changed.

 

I checked the logs on my XPEnology and saw this error:

Jun  5 22:34:09 s2s_monitor: synotify_add_watch error: Function not implemented  path:[/volume1/photo/Foto Sync]

 

Is this a problem with this build or with XPEnology?

Or is it something else?

Share this post


Link to post
Share on other sites
Guys is someone experiencing the same problem as I have?

 

I installed the build as per Idiot's Guide. Running fine but I do have one problem.

I setup Shared Folder Sync on my XPEnology and it syncs to my real Synology DS412+ at home.

I have set the configuration of Shared Folder Sync to sync files whenever the folder changes but nothing happens.

The initial sync works, so the sync itself is working, but after the initial sync it does not see anything that gets changed.

 

I checked the logs on my XPEnology and saw this error:

Jun  5 22:34:09 s2s_monitor: synotify_add_watch error: Function not implemented  path:[/volume1/photo/Foto Sync]

 

Is this a problem with this build or with XPEnology?

Or is it something else?

 

"synotify_add_watch" with a path sounds suspiciously like the "inotify_add_watch" function ( Reference ) in Linux. It's possible that Synology has extended this function to support their special disk configurations (SHR?), and this support is not available (or not present) in the PVSCSI drivers being used for VMWare/ESXi. In which case, it will take some reverse engineering to resolve.

 

Alternatively, there could be an error in the DSM Shared Folder Sync functionality. The first thing that comes to mind is the [space] in the directory name. It's possible that there is an error in the code, which has been corrected/updated in the newer DSM version. For reference, check out my post above:

 

Synology DiskStation DS3612xs Release Notes
Version: DSM 4.2-3211
(2013/04/18) - What's New

Improved stability when syncing files between two DiskStations with Shared Folder Sync.

 

My first attempt at a fix, would be to remove the [space] in the directory name and try the Shared Folder Sync configuration/setup again. Gut feel says that this has a high chance of working.

Share this post


Link to post
Share on other sites

I have question if I can run DSM on VM workstation whether it implies can be run on ESXi?

 

thank you.

Share this post


Link to post
Share on other sites
I have question if I can run DSM on VM workstation whether it implies can be run on ESXi?

 

Ummm ... I don't understand the question.

 

This entire thread is about running DSM in ESXi. By re-reading the thread, you'll find all the information you need to run DSM on ESXi. You need a different set of installation files for ESXi than what is needed for VMWare Workstation or VirtualBox or Hyper-V or for a physical machine (HP N40, etc.)

 

For ESXi, read (and follow exactly) the "Idiot's Guide" to install on ESXi.

Idiot's Guide to DSM 4.2 and ESXi 5.1

 

For other VMWare or VirtualBox installations, check other forum threads. The best one would probably be "XPEnology for dummies LINK Update 02/02/2013".

XPEnology for dummies LINK Update 02/02/2013

Share this post


Link to post
Share on other sites
Guys is someone experiencing the same problem as I have?

 

I installed the build as per Idiot's Guide. Running fine but I do have one problem.

I setup Shared Folder Sync on my XPEnology and it syncs to my real Synology DS412+ at home.

I have set the configuration of Shared Folder Sync to sync files whenever the folder changes but nothing happens.

The initial sync works, so the sync itself is working, but after the initial sync it does not see anything that gets changed.

 

I checked the logs on my XPEnology and saw this error:

Jun  5 22:34:09 s2s_monitor: synotify_add_watch error: Function not implemented  path:[/volume1/photo/Foto Sync]

 

Is this a problem with this build or with XPEnology?

Or is it something else?

 

"synotify_add_watch" with a path sounds suspiciously like the "inotify_add_watch" function ( Reference ) in Linux. It's possible that Synology has extended this function to support their special disk configurations (SHR?), and this support is not available (or not present) in the PVSCSI drivers being used for VMWare/ESXi. In which case, it will take some reverse engineering to resolve.

 

Alternatively, there could be an error in the DSM Shared Folder Sync functionality. The first thing that comes to mind is the [space] in the directory name. It's possible that there is an error in the code, which has been corrected/updated in the newer DSM version. For reference, check out my post above:

 

Synology DiskStation DS3612xs Release Notes
Version: DSM 4.2-3211
(2013/04/18) - What's New

Improved stability when syncing files between two DiskStations with Shared Folder Sync.

 

My first attempt at a fix, would be to remove the [space] in the directory name and try the Shared Folder Sync configuration/setup again. Gut feel says that this has a high chance of working.

 

Thanks mate for your reply!

I'll give it a try. Although the folder I'm trying to sync is not with spaces. The name of the Sync Set is called Foto Sync (with a space).

But never the less I'm going to give it a try and report back here.

 

cheers

Share this post


Link to post
Share on other sites

Unfortunatly it's not working. Getting the same messages in the log:

 

Jun  6 09:49:38 s2s_monitor: synotify_add_watch error: Function not implemented  path:[/volume1/Test2]
Jun  6 09:49:38 s2s_monitor: synotify_add_watch error: Function not implemented  path:[/volume1/Test2/synctest]

 

 

[update]

After testing above I did anoher test.

I setup the sync the other way around. So ds412+ to sync to XPEnology.

Created the sync set and set it to automatically sync when there is a change.

This works without problems.

Share this post


Link to post
Share on other sites
Unfortunatly it's not working. Getting the same messages in the log:

 

Jun  6 09:49:38 s2s_monitor: synotify_add_watch error: Function not implemented  path:[/volume1/Test2]
Jun  6 09:49:38 s2s_monitor: synotify_add_watch error: Function not implemented  path:[/volume1/Test2/synctest]

 

[update]

After testing above I did anoher test.

I setup the sync the other way around. So ds412+ to sync to XPEnology.

Created the sync set and set it to automatically sync when there is a change.

This works without problems.

 

Ok ... this means that the synotify_add_watch error has been removed from the ESXi patches. If you're happy having the sync operate the other way, then you have a solution.

 

I'll see if I can find the function call/stub/omission in jukolaut's code updates, but I don't have the time right now to do so. Maybe when the 3211 version GPL code is available. :smile:

Share this post


Link to post
Share on other sites

To be honest I only the sync the other way around for testing purposes.

I understand you don't have all the time of the world. :grin:

I'll just wait for it.

 

For now manual syncing (Full Sync) works. So that's someting. :lol:

 

Thanks!

Share this post


Link to post
Share on other sites
I have question if I can run DSM on VM workstation whether it implies can be run on ESXi?

 

Ummm ... I don't understand the question.

 

This entire thread is about running DSM in ESXi. By re-reading the thread, you'll find all the information you need to run DSM on ESXi. You need a different set of installation files for ESXi than what is needed for VMWare Workstation or VirtualBox or Hyper-V or for a physical machine (HP N40, etc.)

 

For ESXi, read (and follow exactly) the "Idiot's Guide" to install on ESXi.

Idiot's Guide to DSM 4.2 and ESXi 5.1

 

For other VMWare or VirtualBox installations, check other forum threads. The best one would probably be "XPEnology for dummies LINK Update 02/02/2013".

XPEnology for dummies LINK Update 02/02/2013

 

 

It is because I have finished set up DSM 3211 on VM Workstation 8 under Windows 2008 and wanna export this VM to ESXi 5.1. Am I just click to export in VM Workstation or fully install from the beginning under ESXi?

Share this post


Link to post
Share on other sites

DSM 4.2 build 3211 is not yet running under ESXi.

So you won't be able to export it.

 

At the moment the most recent version for ESXi = build 3202.

 

Tuatara made an Idiot's Guide (for dummies) to install it under ESXi 5.0/5.1

Which can be found here: viewtopic.php?f=2&t=558&start=40#p1632

Share this post


Link to post
Share on other sites
DSM 4.2 build 3211 is not yet running under ESXi.

So you won't be able to export it.

 

At the moment the most recent version for ESXi = build 3202.

 

Tuatara made an Idiot's Guide (for dummies) to install it under ESXi 5.0/5.1

Which can be found here: viewtopic.php?f=2&t=558&start=40#p1632

 

I have first installed 3211 in VB, then copy VMware version 3202's "zImage" file to VB. Use VM workstation to run it. Success. :grin::grin:

Share this post


Link to post
Share on other sites

I am talking about VMware ESXi not VMware Workstation.

Different product. :grin:

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now