Redpill DSM 7 - Virtual SCSI Disks on ESXi (mptsas & vmw_pvscsi) Unstable in Storage Manager


Recommended Posts

Now that Redpill loaders for DSM 7 can be reliably created (thanks to many recent contributions from @haydibe, @pocopico, others, and of course @ThorGroup's original code which was very well organized for others to understand), I am trying to focus down on some stability bugs for typical use cases, namely related to running Redpill DSM 7 on ESXi based virtual machines.

 

I'm experiencing an issue where using SCSI based virtual disks (with virtual LSI SAS or Vmware PVSCSI controllers) isn't working properly within DSM, and first off I am curious if anyone else experiences the same symptoms or if it's just me.

 

If I use mptsas or vmw_pvscsi extensions with Redpill, these virtual SCSI disks successfully show up in DSM, and can even be used to install DSM onto instead of SATA data disks. But, even though DSM install goes fine on them, and DSM runs fine, they are highly unstable when it comes to creating a storage pool in DSM on them. This seems to relate to SMART data not being reliably shimmed for these SCSI virtual disks. i.e. Fake SMART data (per TTG SMART shim in Redpill-LKM sometimes shows up, sometimes doesn't, when it does show up, it often causes disks to flip around positions in storage manager, flips the name of the disk back and forth between "Vmware Virtual Disk" and "null Virtual Disk", etc.

 

Most of the time, those issues cause the disk to not report a health status at all or causes an unhealthy status, which of course prevents the disk from being used to create a storage pool in DSM 7. To note, if I add virtual SATA disks to the VM, they show up reliably in Storage Manager, and remain stable and fully usable with no issues. Also, the SCSI virtual disks don't seem to be disconnecting or exhibiting issues at the system level, just in Storage Manager/with health status in DSM itself. This is why DSM can be installed and run off these SCSI disks, but further use of them in DSM is not possible.

 

That being said, I know others have successfully used mptsas (and mpt2sas/mpt3sas or even the built in DSM SCSI driver) extensions to used physical pass-through disks with DSM. I am guessing this works, because real SMART data gets passed through too, and DSM sees this and works with those disks correctly. That further leads me to believe the following (again, assuming others are able to test the above and get the same result):

 

I believe the issue is with the SMART Shim in Redpill-LKM and/or the SAS > SATA Shim code TTG included on their latest commit to the lkm repo. From what I have been able to tell,

these shims work through playing with "sd.c" the syno-modified SCSI driver. I'm wondering if the issue is the fact that for the above scenario, other drivers are being used and the shim isn't properly touching those of course.

 

Again, I am first curious though if this issue is widespread or if others are able to get virtual SCSI disks on ESXi to work reliably in their configuration. I don't believe comparing or testing with Proxmox helps here because I think I remember TTG saying Proxmox emulates some SMART data to begin with and you can use VirtIO on Proxmox anyway which redpill is designed to work properly with out of the box. Although, if others DO having fully reliable SCSI virtual disks in DSM on Proxmox, this does further support the issue is likely with the shims not interacting with mptsas and vmw_pvscsi properly.

Link to post
Share on other sites

I guess I should point out that "real" SCSI devices (as opposed to SAS) don't have SMART.

 

While SCSI disks were functional in 1.02b/DSM 6.1.x, the install FAQs since 6.x was released have always advised virtual SATA devices.  With 1.03b/1.04b and 6.2.x, virtual SCSI (and probably physical SCSI too) stopped working reliably. Some of my own analysis back in the early days of 6.2.x is here which may be informative and correlate to the experience you are currently having.

 

Just a conjecture, but I suspect this is not a Redpill problem at all, and the configuration is using "leftover" SCSI support code in DSM which Synology doesn't test anymore, since there is no hardware to support it.

  • Like 1
Link to post
Share on other sites
1 hour ago, flyride said:

I guess I should point out that "real" SCSI devices (as opposed to SAS) don't have SMART.

 

While SCSI disks were functional in 1.02b/DSM 6.1.x, the install FAQs since 6.x was released have always advised virtual SATA devices.  With 1.03b/1.04b and 6.2.x, virtual SCSI (and probably physical SCSI too) stopped working reliably. Some of my own analysis back in the early days of 6.2.x is here which may be informative and correlate to the experience you are currently having.

 

Just a conjecture, but I suspect this is not a Redpill problem at all, and the configuration is using "leftover" SCSI support code in DSM which Synology doesn't test anymore, since there is no hardware to support it.

 

Thanks for the insight. I agree it isn't a "problem" with Redpill itself other than simply not being supported/coded (to make the SMART data shim work on virtual SCSI/SAS drives like Redpill does with SATA.

 

That being said, on 1.04b 6.2.x I have used virtual SCSI disks in ESXi as the main DSM drives and have had 0 issues with them. The only reason I did this is because from my beginning days with Xpenology, I thought everyone would always say the virtual SCSI disks would be more performant than virtual SATA, especially with Vmware Paravirtual SCSI ones since that is made specifically for virtualization. But, if vSATA vs. vSCSI really doesn't make a a difference and both are equally as performant, then I'll just use SATA and call it a day with the SCSI stuff. Very rudimentary "benchmarks" lead me to believe they are in fact basically the same, but I didn't run anything scientific or in depth at all. Just compressed a few files and duplicated a few large files, etc... and the timing was basically the same between vSATA volume on DSM and a vSCSI volume on the same DSM VM.

Link to post
Share on other sites

AFAIK there is only one modern underlying virtual disk structure (perhaps not with vATA, haven't tested with vNVMe).  You can change a virtual disk to connect to a vSATA controller, or any dialect of the virtual SCSI controller and the data remains accessible.

Link to post
Share on other sites
On 1/10/2022 at 3:32 PM, flyride said:

AFAIK there is only one modern underlying virtual disk structure (perhaps not with vATA, haven't tested with vNVMe).  You can change a virtual disk to connect to a vSATA controller, or any dialect of the virtual SCSI controller and the data remains accessible.

 

That's a good point and I believe correct. If I take a virtual disk used under a vSATA controller and move it to vSCSI controller, it will still be accessible. The only thing I thought that made SCSI more performant when virtualized is a more performant driver, like Vmware Paravirtual SCSI.

 

But, if performance really is the same or so similar, it doesn't matter to me with DSM. It isn't like I'm running a giant database or 1,000 users trying to access and write/read at same time. There definitely are performance gains in other operating systems when using PVSCSI for example over vSATA controllers with the VM. Plenty of people have tested.

Link to post
Share on other sites
  • 2 weeks later...
On 1/10/2022 at 8:27 PM, ilovepancakes said:

That being said, on 1.04b 6.2.x I have used virtual SCSI disks in ESXi as the main DSM drives and have had 0 issues with them.

Same here. Used to have four real SATA-Disks as RDM via the PVSCSI controller... And thought I have to do so with tinycore-redpill DSM 7. 🤔

 

On 1/6/2022 at 4:32 PM, ilovepancakes said:

... and first off I am curious if anyone else experiences the same symptoms or if it's just me.

Didn't get it done here either. 🙄

 

BUT... obviously I can use my SATA RDM with vSATA AHCI. Works great so far. 🥳

Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.