• 0
geoprea1989

How to recover data from a crashed BTRFS drive? // Help request

Question

Hi guys,

 

I decided to make a call for help as right now I'm stuck on recovering data from my BTRFS drive.

 

I am using a hardware RAID 1 on the back-end [2x 4TB WD Red Drives], and on the front-end, on XPEnology I configured a Basic RAID Group with only "one drive" passed from ESXi.
Until this January I've been using EXT file system, but I read that BTRFS is better both in speed and stability terms, so I decided to give it a go :)
I run my system on UPS which can keep the system powered for more than 4 hours, in case of a blackout, so I though that my data was safe.

 

Two weeks ago I decided to power off the system after 3 months of continual usage, just for checking that everything is okay and to clean it's inside for dust, as a normal routine check.
Unfortunately, after I powered on my system, the main data drive from XPEnology was crashed, not even degraded.
I rebooted it thinking that it might run a fsck by itself, but unfortunately it remained crashed.

 

I ejected both drives from the system and ran extensive checks on them, both smart tests and surface checks, and everything looks just fine.
Given these, I decided to eject one of the drives from the system and to connect it to an external docking station, in order to perform other troubleshooting.

 

Unfortunately, seems that I'm unable to get it mounted, even as RO, with recovery parameters :(
I have the drive connected to a live Parted Magic OS running, for troubleshooting and hope so, recovery procedures :)

 

This is the output of some of the commands I issued, but with no luck to have it mounted:

 

root@PartedMagic:~# btrfs version
btrfs-progs v4.15

 

root@PartedMagic:~# cat /proc/mdstat
Personalities : [linear] [raid0] [raid1] [raid10] [raid6] [raid5] [raid4] [multipath]
md3 : active raid1 sda3[0]
      3902163776 blocks super 1.2 [1/1]

 

root@PartedMagic:~# lsblk -f
NAME    FSTYPE            LABEL                      UUID                                 MOUNTPOINT
loop253 squashfs                                                                          
sr0                                                                                       
loop254 squashfs                                                                          
loop252 squashfs                                                                          
sda                                                                                       
├─sda2  linux_raid_member                            7b661c20-ea68-b207-e8f5-bc45dd5a58c4
├─sda3  linux_raid_member DISKSTATION:3              be150a49-a5a3-f915-32bf-80e4732a20ac
│ └─md3 btrfs             2018.01.02-20:42:27 v15217 1457616f-5daf-4487-ba1c-07963a0c4723
└─sda1  linux_raid_member

 

root@PartedMagic:~# btrfs fi show -d
Label: '2018.01.02-20:42:27 v15217'  uuid: 1457616f-5daf-4487-ba1c-07963a0c4723
    Total devices 1 FS bytes used 2.15TiB
    devid    1 size 3.63TiB used 2.21TiB path /dev/md3

 

root@PartedMagic:~# mount -t btrfs -oro,degraded,recovery /dev/md3 /mnt/temp1
mount: wrong fs type, bad option, bad superblock on /dev/md3,
       missing codepage or helper program, or other error
       In some cases useful info is found in syslog - try
       dmesg | tail or so.

 

root@PartedMagic:~# dmesg | tail -n 100
[SNIP]
[39026.756985] BTRFS info (device md3): allowing degraded mounts
[39026.756988] BTRFS warning (device md3): 'recovery' is deprecated, use 'usebackuproot' instead
[39026.756989] BTRFS info (device md3): trying to use backup root at mount time
[39026.756990] BTRFS info (device md3): disk space caching is enabled
[39026.756992] BTRFS info (device md3): has skinny extents
[39027.082642] BTRFS error (device md3): bad tree block start 1917848902723122217 2473933062144
[39027.083051] BTRFS error (device md3): bad tree block start 5775457142092792234 2473933062144
[39027.083552] BTRFS error (device md3): bad tree block start 1917848902723122217 2473933062144
[39027.083936] BTRFS error (device md3): bad tree block start 5775457142092792234 2473933062144
[39027.097706] BTRFS error (device md3): bad tree block start 1917848902723122217 2473933062144
[39027.098146] BTRFS error (device md3): bad tree block start 5775457142092792234 2473933062144
[39027.114806] BTRFS error (device md3): bad tree block start 1917848902723122217 2473933062144
[39027.115410] BTRFS error (device md3): bad tree block start 5775457142092792234 2473933062144
[39027.133510] BTRFS error (device md3): bad tree block start 1917848902723122217 2473933062144
[39027.133941] BTRFS error (device md3): bad tree block start 5775457142092792234 2473933062144
[39027.136206] BTRFS error (device md3): open_ctree failed

 

root@PartedMagic:~# btrfsck /dev/md3
checksum verify failed on 2473933062144 found 908A6889 wanted 1646EB1A
checksum verify failed on 2473933062144 found 908A6889 wanted 1646EB1A
checksum verify failed on 2473933062144 found E41F2C90 wanted 90ED32B2
checksum verify failed on 2473933062144 found 908A6889 wanted 1646EB1A
bytenr mismatch, want=2473933062144, have=1917848902723122217
Couldn't setup device tree
ERROR: cannot open file system

 

root@PartedMagic:~# btrfs check --repair /dev/md3
enabling repair mode
checksum verify failed on 2473933062144 found 908A6889 wanted 1646EB1A
checksum verify failed on 2473933062144 found 908A6889 wanted 1646EB1A
checksum verify failed on 2473933062144 found E41F2C90 wanted 90ED32B2
checksum verify failed on 2473933062144 found 908A6889 wanted 1646EB1A
bytenr mismatch, want=2473933062144, have=1917848902723122217
Couldn't setup device tree
ERROR: cannot open file system

 

root@PartedMagic:~# btrfs restore -v -i /dev/md3 /mnt/temp1
checksum verify failed on 2473933062144 found 908A6889 wanted 1646EB1A
checksum verify failed on 2473933062144 found 908A6889 wanted 1646EB1A
checksum verify failed on 2473933062144 found E41F2C90 wanted 90ED32B2
checksum verify failed on 2473933062144 found 908A6889 wanted 1646EB1A
bytenr mismatch, want=2473933062144, have=1917848902723122217
Couldn't setup device tree
Could not open root, trying backup super
checksum verify failed on 2473933062144 found 908A6889 wanted 1646EB1A
checksum verify failed on 2473933062144 found 908A6889 wanted 1646EB1A
checksum verify failed on 2473933062144 found E41F2C90 wanted 90ED32B2
checksum verify failed on 2473933062144 found 908A6889 wanted 1646EB1A
bytenr mismatch, want=2473933062144, have=1917848902723122217
Couldn't setup device tree
Could not open root, trying backup super
checksum verify failed on 2473933062144 found 908A6889 wanted 1646EB1A
checksum verify failed on 2473933062144 found 908A6889 wanted 1646EB1A
checksum verify failed on 2473933062144 found E41F2C90 wanted 90ED32B2
checksum verify failed on 2473933062144 found 908A6889 wanted 1646EB1A
bytenr mismatch, want=2473933062144, have=1917848902723122217
Couldn't setup device tree
Could not open root, trying backup super

 

root@PartedMagic:~# btrfs rescue super-recover -v /dev/md3
All Devices:
    Device: id = 1, name = /dev/md3
 
Before Recovering:
    [All good supers]:
        device name = /dev/md3
        superblock bytenr = 65536
 
        device name = /dev/md3
        superblock bytenr = 67108864
 
        device name = /dev/md3
        superblock bytenr = 274877906944
 
    [All bad supers]:
 
All supers are valid, no need to recover

 

root@PartedMagic:~# smartctl -a /dev/sda | grep PASSED
SMART overall-health self-assessment test result: PASSED

 

root@PartedMagic:~# btrfs inspect-internal dump-super /dev/md3
superblock: bytenr=65536, device=/dev/md3
---------------------------------------------------------
csum_type        0 (crc32c)
csum_size        4
csum            0x91d56f51 [match]
bytenr            65536
flags            0x1
            ( WRITTEN )
magic            _BHRfS_M [match]
fsid            1457616f-5daf-4487-ba1c-07963a0c4723
label            2018.01.02-20:42:27 v15217
generation        31802
root            2473938698240
sys_array_size        129
chunk_root_generation    30230
root_level        1
chunk_root        20987904
chunk_root_level    1
log_root        0
log_root_transid    0
log_root_level        0
total_bytes        3995815706624
bytes_used        2366072147968
sectorsize        4096
nodesize        16384
leafsize (deprecated)        16384
stripesize        4096
root_dir        6
num_devices        1
compat_flags        0x0
compat_ro_flags        0x0
incompat_flags        0x16b
            ( MIXED_BACKREF |
              DEFAULT_SUBVOL |
              COMPRESS_LZO |
              BIG_METADATA |
              EXTENDED_IREF |
              SKINNY_METADATA )
cache_generation    22402
uuid_tree_generation    31802
dev_item.uuid        8c0579fe-b94c-465a-8005-c55991b8727e
dev_item.fsid        1457616f-5daf-4487-ba1c-07963a0c4723 [match]
dev_item.type        0
dev_item.total_bytes    3995815706624
dev_item.bytes_used    2428820783104
dev_item.io_align    4096
dev_item.io_width    4096
dev_item.sector_size    4096
dev_item.devid        1
dev_item.dev_group    0
dev_item.seek_speed    0
dev_item.bandwidth    0
dev_item.generation    0

 

Given these, do you have any other idea that I should apply to get my data recovered?
In case you need the output of other troubleshooting commands, please, just let me know, and I will post them here.

 

Thank you in advance for your tips and help! :)

Edited by Polanskiman
Added code tags and formatted OP

Share this post


Link to post
Share on other sites

3 answers to this question

Recommended Posts

  • 0

I would personally just try and salvage the data using one of the Synology guides out there.

The last time I tried to restore data from a BTRFS volume it failed for me even though the data looked like it was indeed recoverable. The rebuild would take hours and in the end was a waste of time. Here is other examples of such an experience.

 

I don't care what Synology says, but IMHO Ext4 is more stable for your MEDIA/DATA pool even with ECC memory. and BTRFS is best used for VMM and multi-user quota access because of snapshots and possible prevention of bitrot. I would not use it as a daily driver because it is still a beta, and not for mission critical operations.

 

Edited by mr9v9

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Answer this question...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.