Portnawak Posted May 15, 2013 Share #76 Posted May 15, 2013 Hi, I've a strange behavior with my xpenology VM running on ESXi 5.1 (on a Microserver N40L) in my lab. When pinging the IP address of this VM from my LAN, and this VM _only_, I've around 25 to 30% of packets loss (all the other VMs share the same ethernet interface, and only the IP address associated to the xpenology VM has this problem). Except the packet loss, everything else seems to run as expected, no real problem, pretty good performance (between 60 to 70 MB per second using FTP, even if the volume is currently expending from 2 to 5 virtual disks in SHR mode), but this sounds strange. Does anyone else notice this ? Quote Link to comment Share on other sites More sharing options...
Tuatara Posted May 15, 2013 Share #77 Posted May 15, 2013 I've a strange behavior with my xpenology VM running on ESXi 5.1 (on a Microserver N40L) in my lab. When pinging the IP address of this VM from my LAN, and this VM _only_, I've around 25 to 30% of packets loss (all the other VMs share the same ethernet interface, and only the IP address associated to the xpenology VM has this problem). Except the packet loss, everything else seems to run as expected, no real problem, pretty good performance (between 60 to 70 MB per second using FTP, even if the volume is currently expending from 2 to 5 virtual disks in SHR mode), but this sounds strange. Does anyone else notice this ? My bet is that the CPU shares for the Synology instance are limiting the overall performance. If the CPU (virtual) is maxed out due to the SHR rebuild, then there will be little (if any) horsepower available for anything else. It's just too busy to service all requests. Now, the reason it's not dropping anything else is due to how TCP is designed. A ping is a UDP packet - it is sent with no expectation of a reply. A FTP connection (or NFS, Samba, etc.) are TCP connections, which by design expect a reply. If there was no reply, the packet is re-sent. This guarantees the communications stream. So basically, the Linux OS on the Synology is dropping the Ping UDP since it is of the lowest priority and can be dropped without incident/issue. I'm certain that once the rebuild is done, you'll notice your transfer speeds approach 100MB/s (Gb LAN speed) and there will be no dropped pings. Alternatively, allocate a greater share of the virtual resources to the Synology VM - more CPU will help this issue. Quote Link to comment Share on other sites More sharing options...
Portnawak Posted May 16, 2013 Share #78 Posted May 16, 2013 My bet is that the CPU shares for the Synology instance are limiting the overall performance. If the CPU (virtual) is maxed out due to the SHR rebuild, then there will be little (if any) horsepower available for anything else. It's just too busy to service all requests. Now, the reason it's not dropping anything else is due to how TCP is designed. A ping is a UDP packet - it is sent with no expectation of a reply. A FTP connection (or NFS, Samba, etc.) are TCP connections, which by design expect a reply. If there was no reply, the packet is re-sent. This guarantees the communications stream. So basically, the Linux OS on the Synology is dropping the Ping UDP since it is of the lowest priority and can be dropped without incident/issue. I'm certain that once the rebuild is done, you'll notice your transfer speeds approach 100MB/s (Gb LAN speed) and there will be no dropped pings. Alternatively, allocate a greater share of the virtual resources to the Synology VM - more CPU will help this issue. Unfortunately no change, the SHR rebuild is done, this is the only VM currently running, CPU is very low, and I'm still loosing around 1 packet every 5 (see below, copy/paste is in French but could be understandable). When I try to reboot the VM, as soon as the network starts, it's OK during several few seconds, and probably after one of the services starts (still have to identify which one), same problem occurs again. Does anyone else here run this with ESXi 5.1 on a Microserver N40L, and could let me know if I'm really the only one ? Thanks in advance. Délai d'attente de la demande dépassé. Réponse de 192.168.1.21 : octets=32 temps<1ms TTL=64 Réponse de 192.168.1.21 : octets=32 temps<1ms TTL=64 Réponse de 192.168.1.21 : octets=32 temps<1ms TTL=64 Réponse de 192.168.1.21 : octets=32 temps<1ms TTL=64 Délai d'attente de la demande dépassé. Réponse de 192.168.1.21 : octets=32 temps=2 ms TTL=64 Réponse de 192.168.1.21 : octets=32 temps=4 ms TTL=64 Réponse de 192.168.1.21 : octets=32 temps=2 ms TTL=64 Réponse de 192.168.1.21 : octets=32 temps=1 ms TTL=64 Délai d'attente de la demande dépassé. Réponse de 192.168.1.21 : octets=32 temps=1 ms TTL=64 Réponse de 192.168.1.21 : octets=32 temps=7 ms TTL=64 Réponse de 192.168.1.21 : octets=32 temps<1ms TTL=64 Réponse de 192.168.1.21 : octets=32 temps<1ms TTL=64 Délai d'attente de la demande dépassé. Réponse de 192.168.1.21 : octets=32 temps=2 ms TTL=64 Réponse de 192.168.1.21 : octets=32 temps<1ms TTL=64 Réponse de 192.168.1.21 : octets=32 temps=17 ms TTL=64 Réponse de 192.168.1.21 : octets=32 temps<1ms TTL=64 Délai d'attente de la demande dépassé. Réponse de 192.168.1.21 : octets=32 temps=1 ms TTL=64 Réponse de 192.168.1.21 : octets=32 temps=3 ms TTL=64 Réponse de 192.168.1.21 : octets=32 temps=1 ms TTL=64 Réponse de 192.168.1.21 : octets=32 temps=3 ms TTL=64 Délai d'attente de la demande dépassé. Réponse de 192.168.1.21 : octets=32 temps=1 ms TTL=64 Réponse de 192.168.1.21 : octets=32 temps<1ms TTL=64 Réponse de 192.168.1.21 : octets=32 temps<1ms TTL=64 Réponse de 192.168.1.21 : octets=32 temps<1ms TTL=64 Délai d'attente de la demande dépassé. Réponse de 192.168.1.21 : octets=32 temps=1 ms TTL=64 Réponse de 192.168.1.21 : octets=32 temps=2 ms TTL=64 Réponse de 192.168.1.21 : octets=32 temps<1ms TTL=64 Réponse de 192.168.1.21 : octets=32 temps=1 ms TTL=64 Délai d'attente de la demande dépassé. Réponse de 192.168.1.21 : octets=32 temps=1 ms TTL=64 Réponse de 192.168.1.21 : octets=32 temps=1 ms TTL=64 Réponse de 192.168.1.21 : octets=32 temps=1 ms TTL=64 Réponse de 192.168.1.21 : octets=32 temps<1ms TTL=64 Délai d'attente de la demande dépassé. Réponse de 192.168.1.21 : octets=32 temps=3 ms TTL=64 Réponse de 192.168.1.21 : octets=32 temps=2 ms TTL=64 Réponse de 192.168.1.21 : octets=32 temps=1 ms TTL=64 Réponse de 192.168.1.21 : octets=32 temps=1 ms TTL=64 Délai d'attente de la demande dépassé. Quote Link to comment Share on other sites More sharing options...
Tuatara Posted May 17, 2013 Share #79 Posted May 17, 2013 Unfortunately no change, the SHR rebuild is done, this is the only VM currently running, CPU is very low, and I'm still loosing around 1 packet every 5 (see below, copy/paste is in French but could be understandable). French, German, English ... all good here. From the dump it looks like exactly 1 packet every 5. This looks very suspicious. Have you got the "Denial of service (DoS) protection" enabled on the DiskStation? If so, then the DiskStation is deliberately dropping the packets to prevent a flood. This would be by design. Quote Link to comment Share on other sites More sharing options...
XPEH Posted May 17, 2013 Share #80 Posted May 17, 2013 Unfortunately no change, the SHR rebuild is done, this is the only VM currently running, CPU is very low, and I'm still loosing around 1 packet every 5 (see below, copy/paste is in French but could be understandable). French, German, English ... all good here. From the dump it looks like exactly 1 packet every 5. This looks very suspicious. Have you got the "Denial of service (DoS) protection" enabled on the DiskStation? If so, then the DiskStation is deliberately dropping the packets to prevent a flood. This would be by design. Real diskstation doesn't do it with or without DoS enabled Pinging 192.168.250.10 with 32 bytes of data: Reply from 192.168.250.10: bytes=32 time<1ms TTL=64 Reply from 192.168.250.10: bytes=32 time<1ms TTL=64 Reply from 192.168.250.10: bytes=32 time<1ms TTL=64 Reply from 192.168.250.10: bytes=32 time<1ms TTL=64 Reply from 192.168.250.10: bytes=32 time<1ms TTL=64 Reply from 192.168.250.10: bytes=32 time<1ms TTL=64 Reply from 192.168.250.10: bytes=32 time<1ms TTL=64 Reply from 192.168.250.10: bytes=32 time<1ms TTL=64 Reply from 192.168.250.10: bytes=32 time<1ms TTL=64 Reply from 192.168.250.10: bytes=32 time<1ms TTL=64 Reply from 192.168.250.10: bytes=32 time<1ms TTL=64 Reply from 192.168.250.10: bytes=32 time<1ms TTL=64 Reply from 192.168.250.10: bytes=32 time<1ms TTL=64 Reply from 192.168.250.10: bytes=32 time<1ms TTL=64 Reply from 192.168.250.10: bytes=32 time<1ms TTL=64 Reply from 192.168.250.10: bytes=32 time<1ms TTL=64 Reply from 192.168.250.10: bytes=32 time<1ms TTL=64 Reply from 192.168.250.10: bytes=32 time<1ms TTL=64 Reply from 192.168.250.10: bytes=32 time<1ms TTL=64 Ping statistics for 192.168.250.10: Packets: Sent = 19, Received = 19, Lost = 0 (0% loss), Approximate round trip times in milli-seconds: Minimum = 0ms, Maximum = 0ms, Average = 0ms Some activity from ESXi is interfering with network. Quote Link to comment Share on other sites More sharing options...
Portnawak Posted May 17, 2013 Share #81 Posted May 17, 2013 From the dump it looks like exactly 1 packet every 5. This looks very suspicious.Have you got the "Denial of service (DoS) protection" enabled on the DiskStation? If so, then the DiskStation is deliberately dropping the packets to prevent a flood. This would be by design. Very suspicious indeed, yes, 1 packet every 5. I've tried _many_ things, by stopping almost _all_ services, changing the IP address, making sure DOS protection was not enabled, moving the Microserver to another switch, changing the virtual NIC within ESXi as well as the physical NIC (I've got an Intel dual Gig also in the Microserver) and ... no change. I finaly gave up, deleted the Xpenology VM, created it again (with 4 VMDK for the time being and not yet 5 but that should not change anything), and now it rocks ! (I'm afraid that I'll never understand what happened...) Thanks for the tips anyway. Quote Link to comment Share on other sites More sharing options...
jukolaut Posted May 19, 2013 Share #82 Posted May 19, 2013 hi again, sorry for my absence! Noticed that my esxi install image has been downloaded over 26000 times, I thought it would interest about 10 people in the world As my system running that is stable I'm not that keen on doing new versions but that might change when synology releases a new major version. Unfortunately it seems that upgrading an exising installation will be difficult. I've packaged the kernel source code here: http://yadi.sk/d/rqFVaQhb4yFtu The package includes the source to the fake synobios module (in drivers/synobios/synobios.c) and other modifications described in this post. happy hacking! Quote Link to comment Share on other sites More sharing options...
Tuatara Posted May 19, 2013 Share #83 Posted May 19, 2013 ... I've packaged the kernel source code ... includes the source to the fake synobios module (in drivers/synobios/synobios.c) and other modifications ... You rock. 'Nuff sed. Cheers & Thank you. Quote Link to comment Share on other sites More sharing options...
bunny0007 Posted May 22, 2013 Share #84 Posted May 22, 2013 Anyone know how the N40L preform with ESXI and 4.2? Quote Link to comment Share on other sites More sharing options...
goocha Posted May 27, 2013 Share #85 Posted May 27, 2013 Hello, I would like to first thank you because i just built an HTPC on ESXi 5.1 and mounted dsm 4.2 3202 on it. It works like a charm I have seen dsm 4.2 3211 was released. Does anyone know how to upgrade the 3202 version ? Thanks. Quote Link to comment Share on other sites More sharing options...
Tuatara Posted May 27, 2013 Share #86 Posted May 27, 2013 I have seen dsm 4.2 3211 was released. Does anyone know how to upgrade the 3202 version ? First off, don't try to upgrade through DSM itself - you will overwrite all modifications which were made to 3202 (kernel, drivers, etc.) which were done in order to get it to run on ESXi. The changes for DSM 4.2 3211 sound significant enough (don't they always), that an update will probably be coming ... from someone ... sometime. Patience. ( DSM 4.2 3211 Release Notes ) Synology have also not as yet released the GPL sources to Sourceforge ( Synology GPL Source ). Since jukolaut has posted his modifications, it would only be a matter of time before the next version is created. Remember ... this DSM code isn't a "free NAS software solution" - it's hacked/modified code, "unreliable" due to that fact, coming with no support, warranty, or remedies. If your system hiccups, crashes, fails, loses all your data, goes up in flames, etc. ... you are on your own ... entirely on your own. This code was originally intended to run only on dedicated Synology hardware. Synology are proving themselves to be a fantastic Open Source company - publishing the GPL portions of their code in a very timely fashion. This dedication to F/OSS is not to be abused. For me, the ability to virtualize DSM has permitted me to spec business requirements, create a virtualized environment to suit the assumptions, check load & usage, and then deploy the needs-appropriate, fully supported, actual Synology hardware. Benefit === incalculable. Synology DiskStation DS3612xs Release Notes Version: DSM 4.2-3211 (2013/04/18) - What's New Fixed an issue where users could not connect to the DiskStation via FTP over SSL. Fixed an issue which kept the DiskStation from connecting to SNMP (Simple Network Management Protocol) UPS devices. Fixed an issue where domain users could not obtain an emergency code for 2-step verification. Improved the compatibility of file sharing via SMB/CIFS with Mac computers. Improved the stability of generating Disk Usage Reports. Improved the stability of iSCSI LUN. Improved the compatibility when backing up data between different versions of DSM. Improved stability when syncing files between two DiskStations with Shared Folder Sync. Improved the stability when backing up data to another DiskStation with Time Backup. Other issue fixes and enhancements. Quote Link to comment Share on other sites More sharing options...
Panja Posted May 30, 2013 Share #87 Posted May 30, 2013 As requested, I've created an Idiot's Guide document which details the installation steps and options required in order to install DSM 4.2 on ESXi 5.1. A complete screen-shot example of all configuration steps is provided, with a fully virtual configuration for testing purposes. Details are also included about creating RDM VMDK disk images instead of VMFS based virtual disks. Thanks again go to jukolaut and odie82544 for making DSM 4.2 on ESXi 5.1 possible. Idiot's Guide to DSM 4.2 and ESXi 5.1.docx - http://depositfiles.com/files/virzefc1a Thanks for the guide! I have been running my build on ESXi without any problems thanks to your guide! Quote Link to comment Share on other sites More sharing options...
netware00 Posted June 4, 2013 Share #88 Posted June 4, 2013 Ok, do you need sources or binaries? The binaries are here(v2): http://yadi.sk/d/fTRkFMyU3D8Yc This is based on odie82544's DS3612xs_3202-Repack images, so thanks for those! Only the kernel, modules and linuxrc.syno are modified so far. The archive contains both the .pat file for installation and .vmdk file for boot. Create the virtual machine to esxi and add hardware: IDE controller with single harddisk (vmdk file in archive). Boot from this. PVSCSI controller and raw harddisks or vmdk files for data. I've tested with 3 disks so far. VMXNET3 network. Only single interface in bridged mode is tested so far. MAC address can be anything. Follow the installation instructions elsewhere in this forum to install the pat file. I wouldn't use this for anything else than evaluation yet. Great work, Jukolaut, thank you very much for sharing it. Quote Link to comment Share on other sites More sharing options...
Panja Posted June 5, 2013 Share #89 Posted June 5, 2013 Guys is someone experiencing the same problem as I have? I installed the build as per Idiot's Guide. Running fine but I do have one problem. I setup Shared Folder Sync on my XPEnology and it syncs to my real Synology DS412+ at home. I have set the configuration of Shared Folder Sync to sync files whenever the folder changes but nothing happens. The initial sync works, so the sync itself is working, but after the initial sync it does not see anything that gets changed. I checked the logs on my XPEnology and saw this error: Jun 5 22:34:09 s2s_monitor: synotify_add_watch error: Function not implemented path:[/volume1/photo/Foto Sync] Is this a problem with this build or with XPEnology? Or is it something else? Quote Link to comment Share on other sites More sharing options...
Tuatara Posted June 5, 2013 Share #90 Posted June 5, 2013 Guys is someone experiencing the same problem as I have? I installed the build as per Idiot's Guide. Running fine but I do have one problem. I setup Shared Folder Sync on my XPEnology and it syncs to my real Synology DS412+ at home. I have set the configuration of Shared Folder Sync to sync files whenever the folder changes but nothing happens. The initial sync works, so the sync itself is working, but after the initial sync it does not see anything that gets changed. I checked the logs on my XPEnology and saw this error: Jun 5 22:34:09 s2s_monitor: synotify_add_watch error: Function not implemented path:[/volume1/photo/Foto Sync] Is this a problem with this build or with XPEnology? Or is it something else? "synotify_add_watch" with a path sounds suspiciously like the "inotify_add_watch" function ( Reference ) in Linux. It's possible that Synology has extended this function to support their special disk configurations (SHR?), and this support is not available (or not present) in the PVSCSI drivers being used for VMWare/ESXi. In which case, it will take some reverse engineering to resolve. Alternatively, there could be an error in the DSM Shared Folder Sync functionality. The first thing that comes to mind is the [space] in the directory name. It's possible that there is an error in the code, which has been corrected/updated in the newer DSM version. For reference, check out my post above: Synology DiskStation DS3612xs Release Notes Version: DSM 4.2-3211 (2013/04/18) - What's New Improved stability when syncing files between two DiskStations with Shared Folder Sync. My first attempt at a fix, would be to remove the [space] in the directory name and try the Shared Folder Sync configuration/setup again. Gut feel says that this has a high chance of working. Quote Link to comment Share on other sites More sharing options...
domwong Posted June 6, 2013 Share #91 Posted June 6, 2013 I have question if I can run DSM on VM workstation whether it implies can be run on ESXi? thank you. Quote Link to comment Share on other sites More sharing options...
Tuatara Posted June 6, 2013 Share #92 Posted June 6, 2013 I have question if I can run DSM on VM workstation whether it implies can be run on ESXi? Ummm ... I don't understand the question. This entire thread is about running DSM in ESXi. By re-reading the thread, you'll find all the information you need to run DSM on ESXi. You need a different set of installation files for ESXi than what is needed for VMWare Workstation or VirtualBox or Hyper-V or for a physical machine (HP N40, etc.) For ESXi, read (and follow exactly) the "Idiot's Guide" to install on ESXi. Idiot's Guide to DSM 4.2 and ESXi 5.1 For other VMWare or VirtualBox installations, check other forum threads. The best one would probably be "XPEnology for dummies LINK Update 02/02/2013". XPEnology for dummies LINK Update 02/02/2013 Quote Link to comment Share on other sites More sharing options...
Panja Posted June 6, 2013 Share #93 Posted June 6, 2013 Guys is someone experiencing the same problem as I have? I installed the build as per Idiot's Guide. Running fine but I do have one problem. I setup Shared Folder Sync on my XPEnology and it syncs to my real Synology DS412+ at home. I have set the configuration of Shared Folder Sync to sync files whenever the folder changes but nothing happens. The initial sync works, so the sync itself is working, but after the initial sync it does not see anything that gets changed. I checked the logs on my XPEnology and saw this error: Jun 5 22:34:09 s2s_monitor: synotify_add_watch error: Function not implemented path:[/volume1/photo/Foto Sync] Is this a problem with this build or with XPEnology? Or is it something else? "synotify_add_watch" with a path sounds suspiciously like the "inotify_add_watch" function ( Reference ) in Linux. It's possible that Synology has extended this function to support their special disk configurations (SHR?), and this support is not available (or not present) in the PVSCSI drivers being used for VMWare/ESXi. In which case, it will take some reverse engineering to resolve. Alternatively, there could be an error in the DSM Shared Folder Sync functionality. The first thing that comes to mind is the [space] in the directory name. It's possible that there is an error in the code, which has been corrected/updated in the newer DSM version. For reference, check out my post above: Synology DiskStation DS3612xs Release Notes Version: DSM 4.2-3211 (2013/04/18) - What's New Improved stability when syncing files between two DiskStations with Shared Folder Sync. My first attempt at a fix, would be to remove the [space] in the directory name and try the Shared Folder Sync configuration/setup again. Gut feel says that this has a high chance of working. Thanks mate for your reply! I'll give it a try. Although the folder I'm trying to sync is not with spaces. The name of the Sync Set is called Foto Sync (with a space). But never the less I'm going to give it a try and report back here. cheers Quote Link to comment Share on other sites More sharing options...
Panja Posted June 6, 2013 Share #94 Posted June 6, 2013 Unfortunatly it's not working. Getting the same messages in the log: Jun 6 09:49:38 s2s_monitor: synotify_add_watch error: Function not implemented path:[/volume1/Test2] Jun 6 09:49:38 s2s_monitor: synotify_add_watch error: Function not implemented path:[/volume1/Test2/synctest] [update] After testing above I did anoher test. I setup the sync the other way around. So ds412+ to sync to XPEnology. Created the sync set and set it to automatically sync when there is a change. This works without problems. Quote Link to comment Share on other sites More sharing options...
Tuatara Posted June 6, 2013 Share #95 Posted June 6, 2013 Unfortunatly it's not working. Getting the same messages in the log: Jun 6 09:49:38 s2s_monitor: synotify_add_watch error: Function not implemented path:[/volume1/Test2] Jun 6 09:49:38 s2s_monitor: synotify_add_watch error: Function not implemented path:[/volume1/Test2/synctest] [update] After testing above I did anoher test. I setup the sync the other way around. So ds412+ to sync to XPEnology. Created the sync set and set it to automatically sync when there is a change. This works without problems. Ok ... this means that the synotify_add_watch error has been removed from the ESXi patches. If you're happy having the sync operate the other way, then you have a solution. I'll see if I can find the function call/stub/omission in jukolaut's code updates, but I don't have the time right now to do so. Maybe when the 3211 version GPL code is available. Quote Link to comment Share on other sites More sharing options...
Panja Posted June 6, 2013 Share #96 Posted June 6, 2013 To be honest I only the sync the other way around for testing purposes. I understand you don't have all the time of the world. I'll just wait for it. For now manual syncing (Full Sync) works. So that's someting. Thanks! Quote Link to comment Share on other sites More sharing options...
domwong Posted June 6, 2013 Share #97 Posted June 6, 2013 I have question if I can run DSM on VM workstation whether it implies can be run on ESXi? Ummm ... I don't understand the question. This entire thread is about running DSM in ESXi. By re-reading the thread, you'll find all the information you need to run DSM on ESXi. You need a different set of installation files for ESXi than what is needed for VMWare Workstation or VirtualBox or Hyper-V or for a physical machine (HP N40, etc.) For ESXi, read (and follow exactly) the "Idiot's Guide" to install on ESXi. Idiot's Guide to DSM 4.2 and ESXi 5.1 For other VMWare or VirtualBox installations, check other forum threads. The best one would probably be "XPEnology for dummies LINK Update 02/02/2013". XPEnology for dummies LINK Update 02/02/2013 It is because I have finished set up DSM 3211 on VM Workstation 8 under Windows 2008 and wanna export this VM to ESXi 5.1. Am I just click to export in VM Workstation or fully install from the beginning under ESXi? Quote Link to comment Share on other sites More sharing options...
Panja Posted June 6, 2013 Share #98 Posted June 6, 2013 DSM 4.2 build 3211 is not yet running under ESXi. So you won't be able to export it. At the moment the most recent version for ESXi = build 3202. Tuatara made an Idiot's Guide (for dummies) to install it under ESXi 5.0/5.1 Which can be found here: viewtopic.php?f=2&t=558&start=40#p1632 Quote Link to comment Share on other sites More sharing options...
domwong Posted June 6, 2013 Share #99 Posted June 6, 2013 DSM 4.2 build 3211 is not yet running under ESXi.So you won't be able to export it. At the moment the most recent version for ESXi = build 3202. Tuatara made an Idiot's Guide (for dummies) to install it under ESXi 5.0/5.1 Which can be found here: viewtopic.php?f=2&t=558&start=40#p1632 I have first installed 3211 in VB, then copy VMware version 3202's "zImage" file to VB. Use VM workstation to run it. Success. Quote Link to comment Share on other sites More sharing options...
Panja Posted June 6, 2013 Share #100 Posted June 6, 2013 I am talking about VMware ESXi not VMware Workstation. Different product. Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.