I decided to make a call for help as right now I'm stuck on recovering data from my BTRFS drive.
I am using a hardware RAID 1 on the back-end [2x 4TB WD Red Drives], and on the front-end, on XPEnology I configured a Basic RAID Group with only "one drive" passed from ESXi.
Until this January I've been using EXT file system, but I read that BTRFS is better both in speed and stability terms, so I decided to give it a go :)
I run my system on UPS which can keep the system powered for more than 4 hours, in case of a blackout, so I though that my data was safe.
Two weeks ago I decided to power off the system after 3 months of continual usage, just for checking that everything is okay and to clean it's inside for dust, as a normal routine check.
Unfortunately, after I powered on my system, the main data drive from XPEnology was crashed, not even degraded.
I rebooted it thinking that it might run a fsck by itself, but unfortunately it remained crashed.
I ejected both drives from the system and ran extensive checks on them, both smart tests and surface checks, and everything looks just fine.
Given these, I decided to eject one of the drives from the system and to connect it to an external docking station, in order to perform other troubleshooting.
Unfortunately, seems that I'm unable to get it mounted, even as RO, with recovery parameters :(
I have the drive connected to a live Parted Magic OS running, for troubleshooting and hope so, recovery procedures :)
This is the output of some of the commands I issued, but with no luck to have it mounted:
root@PartedMagic:~# btrfs version
btrfs-progs v4.15
root@PartedMagic:~# btrfs fi show -d
Label: '2018.01.02-20:42:27 v15217' uuid: 1457616f-5daf-4487-ba1c-07963a0c4723
Total devices 1 FS bytes used 2.15TiB
devid 1 size 3.63TiB used 2.21TiB path /dev/md3
root@PartedMagic:~# mount -t btrfs -oro,degraded,recovery /dev/md3 /mnt/temp1
mount: wrong fs type, bad option, bad superblock on /dev/md3,
missing codepage or helper program, or other error
In some cases useful info is found in syslog - try
dmesg | tail or so.
root@PartedMagic:~# dmesg | tail -n 100
[SNIP]
[39026.756985] BTRFS info (device md3): allowing degraded mounts
[39026.756988] BTRFS warning (device md3): 'recovery' is deprecated, use 'usebackuproot' instead
[39026.756989] BTRFS info (device md3): trying to use backup root at mount time
[39026.756990] BTRFS info (device md3): disk space caching is enabled
[39026.756992] BTRFS info (device md3): has skinny extents
[39027.082642] BTRFS error (device md3): bad tree block start 1917848902723122217 2473933062144
[39027.083051] BTRFS error (device md3): bad tree block start 5775457142092792234 2473933062144
[39027.083552] BTRFS error (device md3): bad tree block start 1917848902723122217 2473933062144
[39027.083936] BTRFS error (device md3): bad tree block start 5775457142092792234 2473933062144
[39027.097706] BTRFS error (device md3): bad tree block start 1917848902723122217 2473933062144
[39027.098146] BTRFS error (device md3): bad tree block start 5775457142092792234 2473933062144
[39027.114806] BTRFS error (device md3): bad tree block start 1917848902723122217 2473933062144
[39027.115410] BTRFS error (device md3): bad tree block start 5775457142092792234 2473933062144
[39027.133510] BTRFS error (device md3): bad tree block start 1917848902723122217 2473933062144
[39027.133941] BTRFS error (device md3): bad tree block start 5775457142092792234 2473933062144
[39027.136206] BTRFS error (device md3): open_ctree failed
root@PartedMagic:~# btrfsck /dev/md3
checksum verify failed on 2473933062144 found 908A6889 wanted 1646EB1A
checksum verify failed on 2473933062144 found 908A6889 wanted 1646EB1A
checksum verify failed on 2473933062144 found E41F2C90 wanted 90ED32B2
checksum verify failed on 2473933062144 found 908A6889 wanted 1646EB1A
bytenr mismatch, want=2473933062144, have=1917848902723122217
Couldn't setup device tree
ERROR: cannot open file system
root@PartedMagic:~# btrfs check --repair /dev/md3
enabling repair mode
checksum verify failed on 2473933062144 found 908A6889 wanted 1646EB1A
checksum verify failed on 2473933062144 found 908A6889 wanted 1646EB1A
checksum verify failed on 2473933062144 found E41F2C90 wanted 90ED32B2
checksum verify failed on 2473933062144 found 908A6889 wanted 1646EB1A
bytenr mismatch, want=2473933062144, have=1917848902723122217
Couldn't setup device tree
ERROR: cannot open file system
root@PartedMagic:~# btrfs restore -v -i /dev/md3 /mnt/temp1
checksum verify failed on 2473933062144 found 908A6889 wanted 1646EB1A
checksum verify failed on 2473933062144 found 908A6889 wanted 1646EB1A
checksum verify failed on 2473933062144 found E41F2C90 wanted 90ED32B2
checksum verify failed on 2473933062144 found 908A6889 wanted 1646EB1A
bytenr mismatch, want=2473933062144, have=1917848902723122217
Couldn't setup device tree
Could not open root, trying backup super
checksum verify failed on 2473933062144 found 908A6889 wanted 1646EB1A
checksum verify failed on 2473933062144 found 908A6889 wanted 1646EB1A
checksum verify failed on 2473933062144 found E41F2C90 wanted 90ED32B2
checksum verify failed on 2473933062144 found 908A6889 wanted 1646EB1A
bytenr mismatch, want=2473933062144, have=1917848902723122217
Couldn't setup device tree
Could not open root, trying backup super
checksum verify failed on 2473933062144 found 908A6889 wanted 1646EB1A
checksum verify failed on 2473933062144 found 908A6889 wanted 1646EB1A
checksum verify failed on 2473933062144 found E41F2C90 wanted 90ED32B2
checksum verify failed on 2473933062144 found 908A6889 wanted 1646EB1A
bytenr mismatch, want=2473933062144, have=1917848902723122217
Couldn't setup device tree
Could not open root, trying backup super
root@PartedMagic:~# btrfs rescue super-recover -v /dev/md3
All Devices:
Device: id = 1, name = /dev/md3
Before Recovering:
[All good supers]:
device name = /dev/md3
superblock bytenr = 65536
device name = /dev/md3
superblock bytenr = 67108864
device name = /dev/md3
superblock bytenr = 274877906944
[All bad supers]:
All supers are valid, no need to recover
root@PartedMagic:~# smartctl -a /dev/sda | grep PASSED
SMART overall-health self-assessment test result: PASSED
Given these, do you have any other idea that I should apply to get my data recovered? In case you need the output of other troubleshooting commands, please, just let me know, and I will post them here.
Thank you in advance for your tips and help! :)
Edited by Polanskiman Added code tags and formatted OP
Question
geoprea1989
Hi guys,
I decided to make a call for help as right now I'm stuck on recovering data from my BTRFS drive.
I am using a hardware RAID 1 on the back-end [2x 4TB WD Red Drives], and on the front-end, on XPEnology I configured a Basic RAID Group with only "one drive" passed from ESXi.
Until this January I've been using EXT file system, but I read that BTRFS is better both in speed and stability terms, so I decided to give it a go :)
I run my system on UPS which can keep the system powered for more than 4 hours, in case of a blackout, so I though that my data was safe.
Two weeks ago I decided to power off the system after 3 months of continual usage, just for checking that everything is okay and to clean it's inside for dust, as a normal routine check.
Unfortunately, after I powered on my system, the main data drive from XPEnology was crashed, not even degraded.
I rebooted it thinking that it might run a fsck by itself, but unfortunately it remained crashed.
I ejected both drives from the system and ran extensive checks on them, both smart tests and surface checks, and everything looks just fine.
Given these, I decided to eject one of the drives from the system and to connect it to an external docking station, in order to perform other troubleshooting.
Unfortunately, seems that I'm unable to get it mounted, even as RO, with recovery parameters :(
I have the drive connected to a live Parted Magic OS running, for troubleshooting and hope so, recovery procedures :)
This is the output of some of the commands I issued, but with no luck to have it mounted:
root@PartedMagic:~# btrfs version btrfs-progs v4.15root@PartedMagic:~# cat /proc/mdstat Personalities : [linear] [raid0] [raid1] [raid10] [raid6] [raid5] [raid4] [multipath] md3 : active raid1 sda3[0] 3902163776 blocks super 1.2 [1/1]root@PartedMagic:~# lsblk -f NAME FSTYPE LABEL UUID MOUNTPOINT loop253 squashfs sr0 loop254 squashfs loop252 squashfs sda ├─sda2 linux_raid_member 7b661c20-ea68-b207-e8f5-bc45dd5a58c4 ├─sda3 linux_raid_member DISKSTATION:3 be150a49-a5a3-f915-32bf-80e4732a20ac │ └─md3 btrfs 2018.01.02-20:42:27 v15217 1457616f-5daf-4487-ba1c-07963a0c4723 └─sda1 linux_raid_memberroot@PartedMagic:~# btrfs fi show -d Label: '2018.01.02-20:42:27 v15217' uuid: 1457616f-5daf-4487-ba1c-07963a0c4723 Total devices 1 FS bytes used 2.15TiB devid 1 size 3.63TiB used 2.21TiB path /dev/md3root@PartedMagic:~# mount -t btrfs -oro,degraded,recovery /dev/md3 /mnt/temp1 mount: wrong fs type, bad option, bad superblock on /dev/md3, missing codepage or helper program, or other error In some cases useful info is found in syslog - try dmesg | tail or so.root@PartedMagic:~# dmesg | tail -n 100 [SNIP] [39026.756985] BTRFS info (device md3): allowing degraded mounts [39026.756988] BTRFS warning (device md3): 'recovery' is deprecated, use 'usebackuproot' instead [39026.756989] BTRFS info (device md3): trying to use backup root at mount time [39026.756990] BTRFS info (device md3): disk space caching is enabled [39026.756992] BTRFS info (device md3): has skinny extents [39027.082642] BTRFS error (device md3): bad tree block start 1917848902723122217 2473933062144 [39027.083051] BTRFS error (device md3): bad tree block start 5775457142092792234 2473933062144 [39027.083552] BTRFS error (device md3): bad tree block start 1917848902723122217 2473933062144 [39027.083936] BTRFS error (device md3): bad tree block start 5775457142092792234 2473933062144 [39027.097706] BTRFS error (device md3): bad tree block start 1917848902723122217 2473933062144 [39027.098146] BTRFS error (device md3): bad tree block start 5775457142092792234 2473933062144 [39027.114806] BTRFS error (device md3): bad tree block start 1917848902723122217 2473933062144 [39027.115410] BTRFS error (device md3): bad tree block start 5775457142092792234 2473933062144 [39027.133510] BTRFS error (device md3): bad tree block start 1917848902723122217 2473933062144 [39027.133941] BTRFS error (device md3): bad tree block start 5775457142092792234 2473933062144 [39027.136206] BTRFS error (device md3): open_ctree failedroot@PartedMagic:~# btrfsck /dev/md3 checksum verify failed on 2473933062144 found 908A6889 wanted 1646EB1A checksum verify failed on 2473933062144 found 908A6889 wanted 1646EB1A checksum verify failed on 2473933062144 found E41F2C90 wanted 90ED32B2 checksum verify failed on 2473933062144 found 908A6889 wanted 1646EB1A bytenr mismatch, want=2473933062144, have=1917848902723122217 Couldn't setup device tree ERROR: cannot open file systemroot@PartedMagic:~# btrfs check --repair /dev/md3 enabling repair mode checksum verify failed on 2473933062144 found 908A6889 wanted 1646EB1A checksum verify failed on 2473933062144 found 908A6889 wanted 1646EB1A checksum verify failed on 2473933062144 found E41F2C90 wanted 90ED32B2 checksum verify failed on 2473933062144 found 908A6889 wanted 1646EB1A bytenr mismatch, want=2473933062144, have=1917848902723122217 Couldn't setup device tree ERROR: cannot open file systemroot@PartedMagic:~# btrfs restore -v -i /dev/md3 /mnt/temp1 checksum verify failed on 2473933062144 found 908A6889 wanted 1646EB1A checksum verify failed on 2473933062144 found 908A6889 wanted 1646EB1A checksum verify failed on 2473933062144 found E41F2C90 wanted 90ED32B2 checksum verify failed on 2473933062144 found 908A6889 wanted 1646EB1A bytenr mismatch, want=2473933062144, have=1917848902723122217 Couldn't setup device tree Could not open root, trying backup super checksum verify failed on 2473933062144 found 908A6889 wanted 1646EB1A checksum verify failed on 2473933062144 found 908A6889 wanted 1646EB1A checksum verify failed on 2473933062144 found E41F2C90 wanted 90ED32B2 checksum verify failed on 2473933062144 found 908A6889 wanted 1646EB1A bytenr mismatch, want=2473933062144, have=1917848902723122217 Couldn't setup device tree Could not open root, trying backup super checksum verify failed on 2473933062144 found 908A6889 wanted 1646EB1A checksum verify failed on 2473933062144 found 908A6889 wanted 1646EB1A checksum verify failed on 2473933062144 found E41F2C90 wanted 90ED32B2 checksum verify failed on 2473933062144 found 908A6889 wanted 1646EB1A bytenr mismatch, want=2473933062144, have=1917848902723122217 Couldn't setup device tree Could not open root, trying backup superroot@PartedMagic:~# btrfs rescue super-recover -v /dev/md3 All Devices: Device: id = 1, name = /dev/md3 Before Recovering: [All good supers]: device name = /dev/md3 superblock bytenr = 65536 device name = /dev/md3 superblock bytenr = 67108864 device name = /dev/md3 superblock bytenr = 274877906944 [All bad supers]: All supers are valid, no need to recoverroot@PartedMagic:~# smartctl -a /dev/sda | grep PASSED SMART overall-health self-assessment test result: PASSEDroot@PartedMagic:~# btrfs inspect-internal dump-super /dev/md3 superblock: bytenr=65536, device=/dev/md3 --------------------------------------------------------- csum_type 0 (crc32c) csum_size 4 csum 0x91d56f51 [match] bytenr 65536 flags 0x1 ( WRITTEN ) magic _BHRfS_M [match] fsid 1457616f-5daf-4487-ba1c-07963a0c4723 label 2018.01.02-20:42:27 v15217 generation 31802 root 2473938698240 sys_array_size 129 chunk_root_generation 30230 root_level 1 chunk_root 20987904 chunk_root_level 1 log_root 0 log_root_transid 0 log_root_level 0 total_bytes 3995815706624 bytes_used 2366072147968 sectorsize 4096 nodesize 16384 leafsize (deprecated) 16384 stripesize 4096 root_dir 6 num_devices 1 compat_flags 0x0 compat_ro_flags 0x0 incompat_flags 0x16b ( MIXED_BACKREF | DEFAULT_SUBVOL | COMPRESS_LZO | BIG_METADATA | EXTENDED_IREF | SKINNY_METADATA ) cache_generation 22402 uuid_tree_generation 31802 dev_item.uuid 8c0579fe-b94c-465a-8005-c55991b8727e dev_item.fsid 1457616f-5daf-4487-ba1c-07963a0c4723 [match] dev_item.type 0 dev_item.total_bytes 3995815706624 dev_item.bytes_used 2428820783104 dev_item.io_align 4096 dev_item.io_width 4096 dev_item.sector_size 4096 dev_item.devid 1 dev_item.dev_group 0 dev_item.seek_speed 0 dev_item.bandwidth 0 dev_item.generation 0Given these, do you have any other idea that I should apply to get my data recovered?
In case you need the output of other troubleshooting commands, please, just let me know, and I will post them here.
Thank you in advance for your tips and help! :)
Edited by PolanskimanAdded code tags and formatted OP
Link to comment
Share on other sites
3 answers to this question
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.