Jump to content
XPEnology Community

Panic: can't recover, "something went wrong"


WanWizard

Recommended Posts

Hello community,

 

I'm in a bit of a panic at the moment.

 

I had the latest DSM 7.1.1 installed and running in an ESXi 6.5 VM. It has:

- Tiny pill vmdk on SATA 0:0
- 100GB vmdk on SATA1:0
- 100GB vmdk on SATA1:1

- 1TB vmdk on SATA1:2
- LSI 2308 using mpt2sas in passthrough, which has 7 x 6TB SATA disks

For some reason there was a gap between the virtual disks and HBA disks, the DS3622+ shows 12 bays, but my disks went to 16 (probably because it had reserved 8 for the SATA controller). This caused some sort of corruption of the DSM RAID0 (the one installed on all disks) on disks 13-16 every time the server had an issue and needed to recover, appearently DSM didn't like more than 12 disks.

So I decided in all my wisdom to deal with that. by changing the satamap. Stupid idea, it turned out to be.

I changed maxdisks from 16 to 12, and generated a satamap. It saw all controllers and all disks, and generated

 

SataPortMap=14

DiskIdxMap=0C00

 

After building and booting the image, it is found, and says I've moved disks to a new DS (other serial?). I recover, wait 10 minutes, and get "something went wrong".

 

I tried changing the maps to

 

SataPortMap=148

DiskIdxMap=0C0000

 

as I found it weird my HBA wasn't in there. I explicitly added the mpt2sas extension again for good measure.
This map I believe is the correct one, has the 3 vmdk's + a spare slot, 7 HBA disks + a spare slot, and the SATABOOT as disk 13, past maxdisks, the way it should be.

 

But again, to no avail.

 

I also tried


SataPortMap=188

DiskIdxMap=000000

which I think was the original disk order, but that doesn't work too.

 

To add to my stupidity, I didn't make a backup of the original boot disk, and I also have no idea what the original config was, in the "old" directory I only find json's with config that don't make sense to me.


From what I can see, all disks and all disk partitions are still what they should be and not damaged, so why can't I get DSM configured and booting again?

Edited by WanWizard
fixed typo's
Link to comment
Share on other sites

Connected to port 7681, according to dmesg and lsmod, the mpt2sas module isn't loaded.

 

And it doesn't seem to be installed as well:
 

SynologyNAS> ls -l /lib64/modules/mpt
mpt3sas.ko   mptbase.ko   mptctl.ko    mptfc.ko     mptsas.ko    mptscsih.ko  mptspi.ko

 

but I'm sure I have added the extension before building?

Link to comment
Share on other sites

Panic is turning into desparation.

 

I've started from scatch with a new vmdk, and followed the instructions.

 

Got mpt3sas loaded now, and all disks detected:

lrwxrwxrwx    1 root     root             0 Jun 21 17:52 sda -> ../devices/pci0000:00/0000:00:11.0/0000:02:02.0/ata2/host1/target1:0:0/1:0:0:0/block/sda
lrwxrwxrwx    1 root     root             0 Jun 21 17:52 sdb -> ../devices/pci0000:00/0000:00:11.0/0000:02:02.0/ata3/host2/target2:0:0/2:0:0:0/block/sdb
lrwxrwxrwx    1 root     root             0 Jun 21 17:52 sdc -> ../devices/pci0000:00/0000:00:11.0/0000:02:02.0/ata4/host3/target3:0:0/3:0:0:0/block/sdc
lrwxrwxrwx    1 root     root             0 Jun 21 17:52 sdf -> ../devices/pci0000:00/0000:00:15.1/0000:04:00.0/host5/port-5:0/end_device-5:0/target5:0:0/5:0:0:0/block/sdf
lrwxrwxrwx    1 root     root             0 Jun 21 17:52 sdg -> ../devices/pci0000:00/0000:00:15.1/0000:04:00.0/host5/port-5:1/end_device-5:1/target5:0:1/5:0:1:0/block/sdg
lrwxrwxrwx    1 root     root             0 Jun 21 17:52 sdh -> ../devices/pci0000:00/0000:00:15.1/0000:04:00.0/host5/port-5:2/end_device-5:2/target5:0:2/5:0:2:0/block/sdh
lrwxrwxrwx    1 root     root             0 Jun 21 17:52 sdi -> ../devices/pci0000:00/0000:00:15.1/0000:04:00.0/host5/port-5:3/end_device-5:3/target5:0:3/5:0:3:0/block/sdi
lrwxrwxrwx    1 root     root             0 Jun 21 17:52 sdj -> ../devices/pci0000:00/0000:00:15.1/0000:04:00.0/host5/port-5:4/end_device-5:4/target5:0:4/5:0:4:0/block/sdj
lrwxrwxrwx    1 root     root             0 Jun 21 17:52 sdk -> ../devices/pci0000:00/0000:00:15.1/0000:04:00.0/host5/port-5:5/end_device-5:5/target5:0:5/5:0:5:0/block/sdk
lrwxrwxrwx    1 root     root             0 Jun 21 17:52 sdl -> ../devices/pci0000:00/0000:00:15.1/0000:04:00.0/host5/port-5:6/end_device-5:6/target5:0:6/5:0:6:0/block/sdl
lrwxrwxrwx    1 root     root             0 Jun 21 18:06 synoboot -> ../devices/pci0000:00/0000:00:11.0/0000:02:01.0/ata1/host0/target0:0:0/0:0:0:0/block/synoboot

 

but it doesn't make an iota of difference. I still get the message that I've moved the disks to a new DS, and the option to press "Recover" if I want to recover settings and data. If I do so, I see 1, 2, 3, 100%, followed by a 10 minute countdown timer, followed by the exact same "something went wrong" I've been looking at for the last 24 hours.

 

 

Link to comment
Share on other sites

Ok. Bit the bullet, and started again from scratch.

 

Removed all disks from the VM definition, deleted the first two vmdk's containing volume1, created two new vmdks, downloaded the tcrp vmdk, converted and installed it, and then installed DSM 7.1.1.

 

Once up and running, added the remaining vmdk and the LSI adapter, used mdadm to manually add the additional disks to /dev/md0 and /dev/md1, and waited for the sync. Restarted, and disks are now available again in DSM.

 

Reconnected hyperbackup to my offsite backup, restored apps and config. And we're back in business.

 

Almost.

Turns out the restore doesn't restore credentials of existing users (like admin), and also NFS permissions aren't restored.

 

But all in all, I'm happy again. Thanks for having your virtual ear... :-)

Edited by WanWizard
typo
Link to comment
Share on other sites

  • 1 month later...
On 6/22/2023 at 8:28 PM, WanWizard said:

...Reconnected hyperbackup to my offsite backup, restored apps and config. And we're back in business...

 

Glad you made it at the end!


Can I ask you if you used Active Backup for Business for the hyperbackup you mention?

And if so how did you manage to activate it?

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...