Djey

(ESXi) pi-control: a workaround to SMART... and more!

Recommended Posts

Hi,

 

despite all help I could get in this amazing forum, I was never able to display SMART and perform self tests in DSM, because of a lack of feature in the drivers for VMWare ParaVirtual/LSI SCSI controller.

Whereas my main concern was just to get SMART data about hard drives in my xpenology set-up as a VM (in a VMWare ESXi hypervisor), 4 hards disks in Raw Device Mapping - Hardware is a HP G7 N54L -

 

Given the fact ESXi already has what it gets to fetch a lot of information from plugged in disks, I began thinking of providing a set of services. Those relying on ESXi 5.5's integrated tools. And so pi-control project was born.

 

To be installed on an always-on device (Raspberry-pi like), pi-control can be used to monitor hypervisor state and perform basic actions. All detailed in API: https://github.com/djey47/pi-control/wiki/API-reference

  • hypervisor ON/OFF switch + schedule + status
  • virtual machine list
  • virtual machine ON/OFF switch + status
  • hard disk list
  • SMART details of particular disk(s)
  • global logging to watch all received service requests ('big brother').

 

pi-control only requires ruby interpreter and a few common tools to be installed onto Raspberry-pi located on same network as hypervisor.

Technically, it accesses ESXi system through ssh to invoke well known tools as esxcli or vim-cmd and throw results back as JSON.

JSON gives a convenient way to use those results in a higher level application (script, webapp, ...)

 

 

You can find latest release hosted on Github: https://github.com/djey47/pi-control/tr ... er/release

(select current release archive then click View Raw link to start downloading).

 

New! First docker images are available.

pi-control images do not support all features for now: e.g ESXi ON

 

Most of the information is there: https://github.com/djey47/pi-control/wiki. So, enjoy :mrgreen:

Edited by Guest

Share this post


Link to post
Share on other sites

Does this work for 5.1 ESXi?

Using 5.1 as there's more driver support than 5.5

 

 

I'm away from home at the moment so can't try anyway; but just wondered as by the time I'm back my NAS will have been on 6 months and nervous it's ready to pop so was prepared to go back to bare bones.

Share this post


Link to post
Share on other sites

what do you want to do with SMART under DSM (with RDM)?

SMART has been always work under DSM with RDM, it was just that the DSM interface wont pull the SMART data.

You can always obtain the smart detail with smartctl -a -d sat /dev/sdc.

 

If you want notification, get smartd install optware. I just received an email of warning last month that one of my drive got high ATA command failure and noticed I got a bad SATA connection. So I have to switch the bay for temporary fix. It is not difficult to schedule short or long smart test with smartd as well.

 

But now I changed my approach to passthrough the entire LSI controller and the smart status under DSM works like a charm.

 

The email alert is attached for your info.

====================

This email was generated by the smartd daemon running on:

 

host name: XXXXXXXX

DNS domain: [unknown]

NIS domain: (none)

 

The following warning/error was logged by the smartd daemon:

 

Device: /dev/sdf [sAT], ATA error count increased from 460 to 471

 

For details see host's SYSLOG (default: /var/log/messages).

 

You can also use the smartctl utility for further investigation.

No additional email messages about this problem will be sent.

Share this post


Link to post
Share on other sites

@elmuziko:

it should work with ESXi 5.1, as it does not rely on tools which are 5.5-specific.

 

@McDull:

No, SMART support has not always been available with smartctl/smartd, depending on xpenology version and installed bootloader.

 

It did work with Trantor's 4.3.

When I migrated to 5.0, gnoBoot did not support it; that's why I had to find a workaround.

Now, nanoboot seems to have it fixed. So what will be next ? I can't definely rely on the bootloader...

A major advantage is I can monitor my datastore hdd to ESxi the same way:)

 

Anyway you're right, if your hardware support VT-d, adding a 3rd part disk controller and using passthrough mode is the way to go to.

That's not an option with N54L.

Share this post


Link to post
Share on other sites

Hi, @Djey. Thanks for your great tool.

1) Is it possible to release it as Synology package to install it one click from public repo or just .spk file on the nas itself?

2) I have something strange on my ESXi + Xpenology installation. smartctl on synology console (via telnet of ssh or even the "real" con) shows relevant information about RDM drives, but DSM web interface does not.

So may be you can consider future update of your smartX app to get the info not only from remote JSON source but also from DSM local smartclt output. It would be great and very useful plugin for synology.

Share this post


Link to post
Share on other sites

Hi,

 

pi-control services do actually run on raspberryPi (or similar device). It's not so easy to install (or even create a such a spk file) on the NAS - due to lack of software support on DSM.

smartX could be however, as it only need nginx.

 

Some of the reasons why I decided to set everything on a rPi:

- the need to control the NAS even when ESXi is OFF (e.g. to start it)

- the need to check hypervisor status with all Vms plugged in, even when xpenology is not running.

 

and yes, smartctl seems to work finally on xpenology. The fact is, smartX is just a webapp and can't directly get data from the system, it must use a data source (like JSON sent by pi-control) to get values. A solution would be to enhance pi-control to provide info from smartctl as well ...

By the way, as a first step I was planning to use ESXI smartd daemon to get latest events.

https://github.com/djey47/pi-control/issues/10

 

I will take note of your idea, can't guarantee to get time to put my hands on it, but who knows ... :smile:

 

 

Thanks for your interest in this project!

Share this post


Link to post
Share on other sites

Finally!

 

I was able to create Docker images for both pi-control and smartX projects! :smile:

Those first images are made for devices running raspbian:jessie.

 

Share this post


Link to post
Share on other sites

Hi,

 

I thought I would give the Docker containers a go myself but found I was getting the following error when I tried running the docker container for Pi-control

 

"System Error: exec format error"

 

I'm new to Docker but have followed the instructions in the Wiki and put all my ESXi config into the command string correctly.

 

Any ideas? I've tried several times without success.

 

Im running ESXi 5.5 on a Microserver G8 with an XPENOLOGY 5.2 virtual machine with RDM drives.

Share this post


Link to post
Share on other sites

Hi,

 

that happens because the current image is based on Debian for ARM (raspbian:jessie) as it's running on a raspberry PI.

 

To make it run on x64 architecture, I have to create a new image. I will do eventually.

Share this post


Link to post
Share on other sites

Thanks. I thought that the containers were created to run on Docker in an Xpenology VM on ESXi. Ill keep a look out for the x64 version.

Share this post


Link to post
Share on other sites

Yeah, my fault I'm afraid. I did not provide details about image compatibility.

A x64-based image (from alpine linux I guess) should work well on syno docker.

Share this post


Link to post
Share on other sites

I keep getting an "operation failed" error message when trying to add the Pi-control image to docker using the GUI.

 

I've not tried the command line.

 

I pulled down the first two images again as a test and they pulled down without error.

 

 

UPDATE: I just tried to pull the pi-control-linux-x64 image using the command line....I get a "repository not found" error.

Share this post


Link to post
Share on other sites

That's weird, I could pull the image from PC at work, but DSM GUI couldn't for the same reasons as you.

All other images can be retrieved without a hitch.

 

Maybe I should recreate the repository and push again ...

Share this post


Link to post
Share on other sites

I tried the command line and get a "repository not found" fatal error.

 

Reading through the bug report at the link provided seems to be a different issue than having here. I can find the repository with the Docker GUI but it fails to download. Command line pull also fails.

 

Will wait until Djey has recreated the repository and will try again.

Share this post


Link to post
Share on other sites

Without version number:

dsm> docker pull djey/pi-control-linux-x64

Pulling repository djey/pi-control-linux-x64

FATA[0003] Repository not found

 

-> no latest tag!

 

with version number:

dsm1> docker pull djey/pi-control-linux-x64:0.1

0.1: Pulling from djey/pi-control-linux-x64

9b7301678506: Pull complete

06984fada629: Pull complete

cbd9326b4af3: Pull complete

b4c9f3479366: Pull complete

8e9b6908e7de: Pull complete

b3137ee5cdae: Pull complete

c66d3cde113e: Pull complete

fb77b7df8877: Pull complete

c4d65ff623c9: Pull complete

b064f42ff405: Pull complete

36c2eee769b2: Pull complete

26bf9b3819fd: Pull complete

Digest: sha256:7bb9da819495b9a7f04c878adce83ea1bf0f36f45c4575a47226f70088f5f834

Status: Downloaded newer image for djey/pi-control-linux-x64:0.1

Share this post


Link to post
Share on other sites

Thanks haydibe, looks like DSM GUI always wanna get 'latest' tag.

So latest tag is working for some images, some other won't (might be due to different docker version - ARM images are pushed from my raspberry PI directly, x64 ones from virtualized Xubuntu 14.04 )

 

I guess that manually (command line) pulled images do appear into DSM as well?

 

Sorry I haven't time to try this till tomorrow.

Share this post


Link to post
Share on other sites

I guess that manually (command line) pulled images do appear into DSM as well?

 

yes, they do :smile:

Share this post


Link to post
Share on other sites

Tested and it works as well as containerized on rPi :grin:

 

Sadly, DSM GUI did not let me provide all required arguments to run image into a container; I had to type from root shell:

 

docker run -d -p 4600:4600 -e ESXI_HOST='x.x.x.x' -e ESXI_USER='root' -e ESXI_MAC_ADDR='xx:xx:xx:xx:xx:xx' -e LAN_BROADCAST_ADDR='x.x.x.x' -v /root/.ssh:/root/.ssh --restart=always --name pi-control djey/pi-control-linux-x64:0.1

 

Running container does appear into DSM GUI afterwards, so that's alright finally.

 

- I'm with DSM 5.2-5967 U1 -

 

Edit: x64 Linux image available for smartX web interface. But need to be tested against SynoDocker, still.

Share this post


Link to post
Share on other sites

Any updates on the x64 Linux image for SmartX? I tried the published version on my Xpenology v5644.5 running under ESXi 5.5 but it bombs out when starting the container.

Share this post


Link to post
Share on other sites

Sorry for late reply, I just tested smartX x64 image against SynoDocker. Working here.

 

From root shell:

 

docker run -d -p 8080:80 -e PI_CONTROL_ADDRESS=':4600' --restart=always --name smartx djey/smartx-linux-x64:0.1

, assuming that pi-control container running under host port 4600.

 

Thus smartX web-ui will be available through http://:8080/smartx.html

Share this post


Link to post
Share on other sites

I'm close to throwing in the towel here. I have to admit I'm a complete novice at Linux and Docker and I've only made the progress I've made with information in the posts above.

 

I have both containers running and can access the SmartX web page but I don't believe I've properly set up the encryption key that allows pi-control to access the ESXi host without password authentication. The SmartX web page simply displays "updating". Looking at the SmartX container log shows repeated errors every 15 seconds stating "connection refused while connecting to upstream client...."

 

I found some information on creating a public key but not sure were to put the key to make it accessible to the pi-control container.

 

If anyone can help with step-by-step instruction for setting up the pi-control to ESXi host authentication it would be much appreciated.

Share this post


Link to post
Share on other sites

Indeed, authentication has to be correctly set-up before starting pi-control container.

 

Here is a starter link :

https://blogs.vmware.com/vsphere/2012/0 ... xi-50.html

 

 

Here comes some explanation on article steps:

To generate a private/public key pair onto dsm, you just log onto DSM shell then run:

ssh-keygen

and follow instructions. It will create keys in directory : /root/.ssh/

( or if you already have keys, copy them into /root/.ssh/ directory)

 

Then, connect to ESXl from DSM:

ssh root@

And confirm to add key to host cache

In ESXI, edit /etc/ssh/keys-/authorized_keys file and copy contents of previously generated id_rsa.pub (public key) - that's what

ssh-copy-id

command does.

 

Exit ESXI shell and try connecting again. If properly configured, next time connection won't require password.

 

To run the container, you just check volume parameter to point your DSM keys directory, check -v parameter :

-v /root/.ssh:/root/.ssh

 

Also, after running pi-control container, execute command and post results here :

docker logs pi-control

 

Should be better after that. Please let me know :wink:

Thanks

Edited by Guest

Share this post


Link to post
Share on other sites