elmuziko

@Martcostan The project is dead as far as my involvement is concerned. I no longer use Xpenology. @TheExpert has done some great work to alter my existing scripts to tailor his own requirements. It sounds like something that just works, no unnecessary complexity with a website.

You champion! Excellent. I hope this works for you. Does it allow you to run full SMART scans? That was one of my biggest frustrations. With Unraid I schedule them on a monthly basis.

@TheExpert Hey buddy, I gotta be honest with you. I've abandoned the project. There was little interest on here for it for a year or so, and I got fed up of all the frustrating bugs/quirks/limitations of Xpenology. The certificate expiry thing late last year with no meaningful update was the last straw for me so I migrated to Unraid. And doing so I found the same thing as you did. My WD Reds - 6 year old WD Reds - were failing horrifically and ESXi was giving me false information. They probably would have died without me getting any real alerts from them, thankfully I was able to replace them before losing my RAID. I would imagine the problem lies with no actual SMART tests beings ran. Whereas a native OS can run a full scan, ESXi doesn't appear to. When I was building the project I was looking to see if I could add an option to force a full SMART scan. I went down the rabbit hole and found someone had ported smartmontools. But it was all bash code, and that's an area I know so little about, I didn't want to risk borking my live system. You can find more on it here. https://www.virten.net/2016/05/determine-tbw-from-ssds-with-s-m-a-r-t-values-in-esxi-smartctl/?msclkid=da4034e4b4b611ec96b92388e10508c5 If you can get it working, fantastic. Otherwise it seems this entire project gives false hope/readings... thank you very much VMWare.

Ah that's brilliant! Love it, and a genius idea. Apologies for my tardy reply, I'm without internet at home at the moment, having to use work computers.

@humancaviar Thank you for the update. This has worked for me with regards to getting Sab to work with strict certificates. Really appreciate it. Synocommunity isn't working but that's small fry. I can live without updates until RedPill is up and ready. Thank you again. For anyone else with this, I'm on DSM 6.2.3-25426 Update 3

So I'm going to show my complete novice at life. Two things. sudo cat ${CERT_DIR}/*.crt >> /etc/ssl/certs/ca-certificates.crt Is giving me : elmuziko@DSM:~$ sudo cat ${CERT_DIR}/*.crt >> /etc/ssl/certs/ca-certificates.crt -sh: /etc/ssl/certs/ca-certificates.crt: Permission denied am I doing something wrong? And secondly, how do I run the for ; do loop from a PUTTY command loop. Will it not take each line as a single command? Thank you

Just throwing a bit more on this in the hope we find something. My Synocommunity packages list is now empty, and if I try to add a new source, I get told to sod off in not so many words. I was going to try and install nzbget as a temp work around until someone has identified and fixed the issue, though reading on the internet, nzbget is suffering the same issue. NZBGet have published a healthy page on resolutions. https://nzbget.net/certificate-verification Unfortunately a manual install of NZBGet is failing for me on my DSM so I can't put the steps into practise. But maybe some breadcrumbs for someone smarter than I? Then there's this: https://github.com/nzbget/nzbget/releases/tag/v21.2-r2333

VERSION 3 RELEASE Evening all. Version 3 is now ready on my BitBucket repo here WEBSITE CHANGES Support for NVMe drives. Colour coded drive types. Temperature graph is now a modal, it now displays 48 entries (a days worth of entries when the PowerShell script is running in its default setup). ESXi CPU, Memory and Datastore monitors. New 'Sort-By' mode added to Settings (sort by Drive Type). A ton of bugfixes and code improvement. POWERSHELL CHANGES Support for Linux!!! (Tested on Ubuntu). Removed pre-requisite 'Oracle MySQL plugin'. Script now uses SimplySQL from the PowerShell gallery. Will build a scheduled task for you if using a Windows device. Script is now truly a "run once and forget" thing. More bugfixes than I care to count. POWERSHELL FIRST USE In Windows, open up PowerShell_ISE and use : $firstRun = @{ # If you're using Synology to store the DB, change SQLUser from root, and SQLPort to 3307. If using XAMPP, root and 3306 will work. ESXiHost = "IP OR DNS NAME OF ESXI HOST" ESXiPass = ( "YOURPASSWORDHERE" | ConvertTo-SecureString -AsplainText -Force ) SQLUser = "CHANGEME" SQLPass = ( "YOURPASSWORDHERE" | ConvertTo-SecureString -AsplainText -Force ) SQLPort = 3307 SQLHost = "IP OR DNS NAME OF SYNOLOGY HERE" Email = "youremailaddresshere" EmailPass = "YOUR EMAIL APP PASSWORD HERE, OR IF YOU'RE MAD ENOUGH TO NOT HAVE TWO STEP, JUST PLAIN PASSWORD" } Import-Module ESXiSMART.psd1 -Force -Global -Verbose Start-SmartDataScript @firstRun -Verbose If you're using Linux, Install PowerShell, then use : Import-Module ESXiSMART.psd1 -Force -Global -Verbose Start-SmartDataScript -SQLUser "CHANGESQLUSER" -SQLHOST "CHANGESQLHOST" -Email "CHANGEEMAILADDRESS" -EmailPass "plainTextpasswordforemail" -Verbose This will then prompt you for the missing essential items. NEXT STEPS I would very much appreciate support for Linux. I would very much appreciate it if someone could help me build a Scheduled Task for the script to run every 30 minutes or so. I did look into this, using something called CronTab but this getting into the realm of doing stuff I have no knowledge of. Which is a dangerous road to go down. I also want to look at building a container for this. Now the PowerShell supports Linux it can actually go on ESXi. I would imagine the Scheduled Task completion thing needs doing first? Another unknown, but one I feel more comfortable tinkering with until we have it. If anyone can offer help on either a Linux scheduled task or container, I would really appreciate it. And if anyone spots any bugs or has any recommendations for improvements/new features, please do let me know.

Thank you. I've found a few bugs in the PowerShell code so have been concentrating on that. I'm also translating this into Linux PowerShell so I can create a Linux Docker image. Sadly work commitments are slowing me down but will have a lot more free time next month to complete the next version. I will absolutely message when I'm at a testing stage.

One would assume you want Docker for the scheduled task aspect yes? Windows on Docker doesn't work on Synology, but I'll look at altering the PowerShell script to run on Linux and thus solve the problem

Great idea. Consider it my next challenge!

Reserved for version 3.0.0

Good afternoon all. I've finally got round to updating both the website and PowerShell scripts. Both code improvements and new features. Website Update and Improvements. The website now hosts a 'SETTINGS' page, in which certain aspects can be controlled. Hard Drive order can now be set from small to large, large to small, Raw Device Mapping Naming convention, Controller Location, and Hard Drive Number. An option to hide Removed Drives (great for when you upgrade drives) Alter how many plot points there are on your temperature graph. Each drive can have individually set temperature threshold points. The Index page will alert you to any drive temperature alerts. PowerShell SMART Data Improvements. On the first run, if an email address and plain text password are provided an encrypted credentials file is generated in the temp directory. From there on out, if just an email address is provided, then any irregularities found in the SMART data will trigger an email alert. Email Alerts. As mentioned above, email alerts are a thing now. I've set the script up to send email from GMail, Yahoo and Microsoft based providers. If you're using your own mail server you'll need to provide details of said provider as both a parameter on the main Start-SmartDataScript and in the Send-EmailAlerts script (see image below for information). If you have MFA / 2-Step authentication switched on (and if you don't you really should!) you'll need to provide an app password rather than your standard password. More information on this is provided in the README.MD file within the PowerShell Module. If anyone uses it and finds any bugs, please do let me know. I know of two already but neither effect normal usability and will be fixed in version 3.0.0 - which will include Windows Event logging. Images from Version 2.0.0 The first is a comparison of the old index vs the new. I've replaced a drive whilst building version 2.0.0, this is no longer present on the index. Also note the graph is plotted over a longer time period. In the second image, you can see the new Settings page. Note the SSD drive has different default Temperature thresholds than spinning disks. The third image is an email I got when I removed a drive in favour of a larger one. The fourth is the PowerShell function Send-EmailAlert.psm1. You will need to edit this if your Email provider is not one of the big three. The last image is a work in progress on version 3.0.0. This is a means of building a scheduled task without having to use plain text passwords. Download Information My BitBucket Repo is here . Please note, whilst writing this forum post I noticed a small bug on ESXISmartWebsite. This has been posted to Master, but I have no idea how to update Release 2.0.0 so make sure to grab from Master. Updating Existing Files In both instances, the new repos can be placed directly over the existing ones. Both the PowerShell module and website will attempt to update the existing Database. Scheduled Task Updates Unfortunately, although the update process is easy, there is one extra thing you'll need to do, and that's to update the Scheduled Task you have running. Before that, open a PowerShell window from within the ESXiSMART folder and type: Import-Module EsxiSmart.psd1 -Force -Verbose If you get an error message about vis then just re-run. This is because your scheduled task is already using VmWare resources. Next you'll need to provide your Email address and Email password to build an encrypted file. Start-SmartDataScript -Email youremail@yourdomain.com -EmailPass yourEmailPassword You only need to do this once. From here amend your Scheduled Task script with just the extra parameter -Email youremmail@yourdomain.com Version 3.0.0 Version 3.0.0 is already in the making. Most of the work will be within PowerShell. I intend to add Event Logging so you have visibility of the what's happening in the script whenever it's ran. Also, I'm currently building a small application that encrypts your ESXi and MySQL passwords so they're not needed in plain text for the scheduled task to run. If anyone has any suggestions, notices any bugs, wants to improve the code, or whatever. I'm all ears. Please do let me know.

That was my understanding too. If fab198 has done it, maybe he could share with the community so we could build on it further

You have 6 on there? Amazing. I didn't think 6 was possible at all, thought we were stuck on 5.2. did you have to do anything special to get 6 on there?

Good afternoon all, and a happy new year to you all. So I use VMWare ESXi to host my Synology 6.2 - and though I know there's a plugin to retrieve SMART data, for me it doesn't work. Presumably because my server is a mish mash of random hardware; that or I'm too lazy to bugfix. Something I've wanted to do for as long as I can remember is to extract SMART data from ESXi and add to a database, then present it as a website that's super easy to read. What the site does Well, as you can see from the image below, I have ESXi SMART data presented in an easy to read format for each of my drives, as well as other important information such as RDM location, what drive number the hard drive is on the vm host, if it's an SSD, etc, etc. It also presents you with 5 hours worth of temperature changes, just in case like me you panic every time the sun comes out. How does it work? OK so I'm a PowerShell developer. The site was built as a learning curve using PHP - it is not my natural environment. Bash is like a totally foreign language to me and one I'm not comfortable with, so I acknowledge this is a long ass way around doing this. Nevertheless, it works and I know every single aspect of what it's doing. A PowerShell script runs as a scheduled task on a Windows machine, rips data straight out of ESXi and imports it into a MariaDB database. In my case I have a Windows 10 device on my ESXi host running said script, with my Xpenology OS hosting both the database and website. How do I install it? You want a Windows device running 24/7, since this device will be ripping SMART data every thirty minutes (or whatever schedule you give it). With that in mind you're best off using your ESXi host for your Windows machine. This website explains how to download Windows 10 without a key (this is totally legit. Downloading without a key is different to installing without a key which I do not recommend!) Once you have your windows device up and running, open PowerShell as an administrator and type: Set-ExecutionPolicy -ExecutionPolicy Unrestricted -Confirm:$false Next, you'll want to hit my BitBucket repo here and download both repos. The one specific to ripping data out is called ESXiSMART Extract the contents of the zip file and check out the README.md file for complete instructions. This will explain how to set up a scheduled task. NOTE: There are a few steps you need to take before the scheduled task will work. Create your scheduled task .ps1 file as per the README.md file, then wherever you save it, open PowerShell in the same folder and run the .ps1 file from the PowerShell console. This is important as it'll download the VMWare plugins and if MySQL dll file is missing, give you the link to download and install it. This information is contained within the README.md file but the script will prompt you for action if you can't be bothered to read it. OK I have a scheduled task running, and I can see the database on myphpadmin, now what? Well now you want the website. Which can be found at the same BitBucket url. There's instructions there on how to host. This has been tested on both XAMPP and Synology. It's working! But the temp graph is just showing the same temp every time! Yeah it shows the last 10 entries, so just be patient. It's not working! What should I do? Post here obviously, with your bugs, and I'll try resolve it for you. This is nice, but it could be better! Oh totally. Well, feel free to make your own changes. Totally open source. Or, if you have suggestions and no coding skills, just let me know what you'd like. I'd like to do a version 2 of the website which includes alerting. I can do this in PowerShell fairly easily but as I'm learning PHP would sooner take my time and do it through here. You know there's an easier way to get smart data out of VMWare without a Windows machine and PowerCLI right? Yeah I know. But bash is alien to me. With PowerShell and PHP I know what each command does and the risks involved. With bash I'm just typing out stuff totally unknown to me and hoping very bad things aren't happening under the hood. So I totally welcome any easier approach to this. If you know bash and want to help out, I'm very keen to have you on board. In fact, I encourage it. There's a guide here on how to get extended SMART data from ESXi which includes Total Bytes Written on SSD's. But it means installing stuff on ESXi and I'm not comfortable doing it. If you want to help build a tool that installs smartctl and runs it as a scheduled task, please do. Not having to use a Windows box to get SMART data out is the perfect scenario. Are there any licenses I should be aware of? Nope. Well. A windows license, but (and this hasn't been tested as mine is totally legit) I believe you could run a scheduled task without activating Windows. If someone can confirm I'll update this post. MySQL is free, PowerCLI is free, and the images used on the website are from Pixabay under open source license agreements.

@blackburn Haven't tried the above, but nonetheless, thank you for picking this up after so long. Appreciate it

Hi IG-88, Thanks for the comment. It appears to be Docker making the racket. As soon as I disable it, things calm down immensely. That's the only software that's changed after the drive change. Shame that, liked the new way of running Sab, Sick and Couch!

HP N54L JunLoader DSM 6.1.7-15284 Update 3 3x Toshiba HDWG21E 14TB drives in RAID 5. Docker install for couchpotato, sabnzbd and sickchill Hard Drive's set to 15 minutes sleep with logs switched on. I recently moved from WD red drives. In the process upgraded to Update 3 from Update 2. The hard drives just aren't spinning down. Any ideas on what's going off? At times it sounds like there's thunderstorms going off. Any ideas where I can find out what's eating away at the drives?

Sorry if this is in the wrong area; Running Synology on ESXi 6.7. As we know this means not being able to access SMART data from Synology itself. I've got the SMART data out from bash, but what I'm curious to know is, is it possible to run a full SMART test at all? In Synology you can perform long tests, in Windows WDC has an app to perform a full scan, is there anything of this nature available to us for ESXi? And if so, does anyone know how to do it? Thanks

I'd still take a permanent fix to this any day of the week but getting around it for the time being using those steps albeit from the Synology site, to chmod wherever it is I'm going to, using winSCP as a standard user, then chmod back to what it was previously. It's not clean, nor is it straight forward, but it is a work around. Again, if someone has a more permanent fix I'd take it any day of the week. Thanks

Yo, has anyone got an idiots guide to enabling root on 6.2? I use winSCP to transfer files but can't do it any more since directories are locked down. i found a guide on YouTube however on my etc/ssh/sshd_config, PermitRootLogin is set to prohibit-password. Making the changes in the questions just makes the ssh service stop and won't come back up. Thanks

Ended up having to rebuild, a misplaced letter on a mdadm switch formatted the whole lot.

Morning all, DSM 5.2 user here - for the most part - works great, but when something happens to the Hyper-V host it often fails spectacularly. Today is one of those days. I'm not great at under the hood stuff but have Google'd around and come to the conclusion the whole thing is confused! I only have one RAID5 with 7 disks in total which seems to correspond to mdadm --detail /dev/md2. Can anyone give me any guidance on how to bring the volume back to life? SuperSyno> mdadm --detail /dev/md3 /dev/md3: Version : 1.2 Creation Time : Sun Apr 1 20:22:22 2018 Raid Level : raid5 Array Size : 3906971648 (3725.98 GiB 4000.74 GB) Used Dev Size : 976742912 (931.49 GiB 1000.18 GB) Raid Devices : 5 Total Devices : 4 Persistence : Superblock is persistent Update Time : Fri Dec 21 07:52:22 2018 State : clean, degraded Active Devices : 4 Working Devices : 4 Failed Devices : 0 Spare Devices : 0 Layout : left-symmetric Chunk Size : 64K Name : SUPERSYNO:3 UUID : e3f844f6:4bd70c27:c40439c2:8d1a29e9 Events : 12216 Number Major Minor RaidDevice State 0 8 70 0 active sync /dev/sde6 1 0 0 1 removed 4 8 150 2 active sync /dev/sdj6 5 8 118 3 active sync /dev/sdh6 2 8 134 4 active sync /dev/sdi6 SuperSyno> mdadm --detail /dev/md0 /dev/md0: Version : 0.90 Creation Time : Sat Jan 1 00:04:42 2000 Raid Level : raid1 Array Size : 2490176 (2.37 GiB 2.55 GB) Used Dev Size : 2490176 (2.37 GiB 2.55 GB) Raid Devices : 12 Total Devices : 8 Preferred Minor : 0 Persistence : Superblock is persistent Update Time : Fri Dec 21 08:08:09 2018 State : clean, degraded Active Devices : 8 Working Devices : 8 Failed Devices : 0 Spare Devices : 0 UUID : 26aa5fea:f6cb55b4:3017a5a8:c86610be Events : 0.5720 Number Major Minor RaidDevice State 0 8 33 0 active sync /dev/sdc1 1 8 49 1 active sync /dev/sdd1 2 8 145 2 active sync /dev/sdj1 3 8 129 3 active sync /dev/sdi1 4 8 113 4 active sync /dev/sdh1 5 8 97 5 active sync /dev/sdg1 6 8 81 6 active sync /dev/sdf1 7 8 65 7 active sync /dev/sde1 8 0 0 8 removed 9 0 0 9 removed 10 0 0 10 removed 11 0 0 11 removed SuperSyno> mdadm --detail /dev/md1 /dev/md1: Version : 0.90 Creation Time : Thu Dec 20 22:46:39 2018 Raid Level : raid1 Array Size : 2097088 (2048.28 MiB 2147.42 MB) Used Dev Size : 2097088 (2048.28 MiB 2147.42 MB) Raid Devices : 12 Total Devices : 8 Preferred Minor : 1 Persistence : Superblock is persistent Update Time : Thu Dec 20 22:47:37 2018 State : active, degraded Active Devices : 8 Working Devices : 8 Failed Devices : 0 Spare Devices : 0 UUID : 3637e4b4:d5bb08ce:dca69c88:18a34d86 (local to host SuperSyno) Events : 0.19 Number Major Minor RaidDevice State 0 8 34 0 active sync /dev/sdc2 1 8 50 1 active sync /dev/sdd2 2 8 66 2 active sync /dev/sde2 3 8 82 3 active sync /dev/sdf2 4 8 98 4 active sync /dev/sdg2 5 8 114 5 active sync /dev/sdh2 6 8 130 6 active sync /dev/sdi2 7 8 146 7 active sync /dev/sdj2 8 0 0 8 removed 9 0 0 9 removed 10 0 0 10 removed 11 0 0 11 removed SuperSyno> mdadm --detail /dev/md2 /dev/md2: Version : 1.2 Creation Time : Sun Apr 1 20:22:22 2018 Raid Level : raid5 Array Size : 20478048192 (19529.39 GiB 20969.52 GB) Used Dev Size : 2925435456 (2789.91 GiB 2995.65 GB) Raid Devices : 8 Total Devices : 7 Persistence : Superblock is persistent Update Time : Thu Dec 20 22:34:47 2018 State : clean, degraded Active Devices : 7 Working Devices : 7 Failed Devices : 0 Spare Devices : 0 Layout : left-symmetric Chunk Size : 64K Name : SUPERSYNO:2 UUID : 7d8b04cf:aebb8d01:6034359b:c4bb62db Events : 280533 Number Major Minor RaidDevice State 9 8 53 0 active sync /dev/sdd5 1 8 69 1 active sync /dev/sde5 2 0 0 2 removed 7 8 149 3 active sync /dev/sdj5 8 8 117 4 active sync /dev/sdh5 5 8 133 5 active sync /dev/sdi5 4 8 101 6 active sync /dev/sdg5 3 8 85 7 active sync /dev/sdf5 SuperSyno> mdadm --detail /dev/md3 /dev/md3: Version : 1.2 Creation Time : Sun Apr 1 20:22:22 2018 Raid Level : raid5 Array Size : 3906971648 (3725.98 GiB 4000.74 GB) Used Dev Size : 976742912 (931.49 GiB 1000.18 GB) Raid Devices : 5 Total Devices : 4 Persistence : Superblock is persistent Update Time : Fri Dec 21 08:01:56 2018 State : clean, degraded Active Devices : 4 Working Devices : 4 Failed Devices : 0 Spare Devices : 0 Layout : left-symmetric Chunk Size : 64K Name : SUPERSYNO:3 UUID : e3f844f6:4bd70c27:c40439c2:8d1a29e9 Events : 12218 Number Major Minor RaidDevice State 0 8 70 0 active sync /dev/sde6 1 0 0 1 removed 4 8 150 2 active sync /dev/sdj6 5 8 118 3 active sync /dev/sdh6 2 8 134 4 active sync /dev/sdi6

Looks like it crashed. Turned it off and on again, it sat at BUSY Re-Scanning disks for twenty minutes, then moved on to starting services for an hour, then finally back in. Hooorah!