HOW TO: Boot Time / Offline Full Volume Integrity Check Procedure - e2fsck/fsck.ext4

[gericb]

I have DSM 6.2.3-25426 Update 3, DS3615xs running in RAID1, EXT4, 2 disk setup, super basic.... about a week ago, the box ended up losing power and re-starting 2x due to power glitches in my building.  As a result, I later noticed the dashboard was reporting it was running in degraded mode, seems the system partition on disk2 was corrupted.  So I just added disk2 back into the RAID, and let it rebuild.  Since then, I have noticed if I am playing most any of my Plex content, it often times jerky/jittery, I assume to other corruptions.

 

Because of this, I have looked around to try to find the step-by-step to umount and take down the Volume 1 and either to a boot-time or online e2fsck/fsck.ext4 check of the whole thing.  There use to be a Wiki Tutorial on Synology's site, but that is gone.  I known to do this kind of check on an active filesystem can/will cause additional corruptions/damage, so I am obviously not wanting to make things worse.  Volume 1 and the respective disks are showing "Healthy" in the Storage Manager, but still don't have warm fuzzies because of the sudden new content playback jitters.

 

Would appreciate any help you could provide on how to properly take down the system and run this full system/volume/disk check and repair process.

 

Thank You In Advance!

 

 

[gericb]

On 4/16/2022 at 11:02 AM, gericb said:

I have DSM 6.2.3-25426 Update 3, DS3615xs running in RAID1, EXT4, 2 disk setup, super basic.... about a week ago, the box ended up losing power and re-starting 2x due to power glitches in my building.  As a result, I later noticed the dashboard was reporting it was running in degraded mode, seems the system partition on disk2 was corrupted.  So I just added disk2 back into the RAID, and let it rebuild.  Since then, I have noticed if I am playing most any of my Plex content, it often times jerky/jittery, I assume to other corruptions.

 

Because of this, I have looked around to try to find the step-by-step to umount and take down the Volume 1 and either to a boot-time or online e2fsck/fsck.ext4 check of the whole thing.  There use to be a Wiki Tutorial on Synology's site, but that is gone.  I known to do this kind of check on an active filesystem can/will cause additional corruptions/damage, so I am obviously not wanting to make things worse.  Volume 1 and the respective disks are showing "Healthy" in the Storage Manager, but still don't have warm fuzzies because of the sudden new content playback jitters.

 

Would appreciate any help you could provide on how to properly take down the system and run this full system/volume/disk check and repair process.

 

Thank You In Advance!

 

 

Hey there @Peter Suh @flyride @haldi @IG-88 @gadreel good day!  This seems to be a pretty illusive procedure these days.  There are number posts from days gone by, but they all seem so be lacking in one way or another, which in the end don't actually execute successfully.  I remember doing this successfully on DSM 5, but now on DSM 6, not so straight forward.  Do you guru's known off the top of your head, the right, working steps needed to either force a full fsck.ext4/e2fsck, either at startup or via SSH/Telnet to reliably kick this off?

 

Initially I'm doing:

 

"ssh admin@192.168.1.244"

<entering correct password when prompted>

"sudo su -"

<entering correct password when prompted>

"mount" <produces the below listing below>  My RAID1, ext4 name is Volume1

 

/dev/md0 on / type ext4 (rw,relatime,journal_checksum,barrier,data=ordered)

none on /dev type devtmpfs (rw,nosuid,noexec,relatime,size=4018724k,nr_inodes=1004681,mode=755)

none on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)

none on /proc type proc (rw,nosuid,nodev,noexec,relatime)

none on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)

/tmp on /tmp type tmpfs (rw,relatime)

/run on /run type tmpfs (rw,nosuid,nodev,relatime,mode=755)

/dev/shm on /dev/shm type tmpfs (rw,nosuid,nodev,relatime)

none on /sys/fs/cgroup type tmpfs (rw,relatime,size=4k,mode=755)

cgmfs on /run/cgmanager/fs type tmpfs (rw,relatime,size=100k,mode=755)

cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,relatime,cpuset,release_agent=/run/cgmanager/agents/cgm-release-agent.cpuset,clone_children)

cgroup on /sys/fs/cgroup/cpu type cgroup (rw,relatime,cpu,release_agent=/run/cgmanager/agents/cgm-release-agent.cpu)

cgroup on /sys/fs/cgroup/cpuacct type cgroup (rw,relatime,cpuacct,release_agent=/run/cgmanager/agents/cgm-release-agent.cpuacct)

cgroup on /sys/fs/cgroup/memory type cgroup (rw,relatime,memory,release_agent=/run/cgmanager/agents/cgm-release-agent.memory)

cgroup on /sys/fs/cgroup/devices type cgroup (rw,relatime,devices,release_agent=/run/cgmanager/agents/cgm-release-agent.devices)

cgroup on /sys/fs/cgroup/freezer type cgroup (rw,relatime,freezer,release_agent=/run/cgmanager/agents/cgm-release-agent.freezer)

cgroup on /sys/fs/cgroup/blkio type cgroup (rw,relatime,blkio,release_agent=/run/cgmanager/agents/cgm-release-agent.blkio)

none on /proc/bus/usb type devtmpfs (rw,nosuid,noexec,relatime,size=4018724k,nr_inodes=1004681,mode=755)

none on /sys/kernel/debug type debugfs (rw,relatime)

securityfs on /sys/kernel/security type securityfs (rw,relatime)

/dev/mapper/cachedev_0 on /volume1 type ext4 (rw,relatime,journal_checksum,synoacl,data=writeback,jqfmt=vfsv0,usrjquota=aquota.user,grpjquota=aquota.group)

none on /config type configfs (rw,relatime)

/dev/mapper/cachedev_0 on /volume1/@docker type ext4 (rw,relatime,journal_checksum,synoacl,data=writeback,jqfmt=vfsv0,usrjquota=aquota.user,grpjquota=aquota.group)

none on /volume1/@docker/aufs/mnt/29f38ffeba90da8a04e2c8aafbc2980db6ca5344c3efacde17fd251da3001ec1 type aufs (rw,relatime,si=18a301f96028ed6,dio,dirperm1)

none on /volume1/@docker/aufs/mnt/13b1ccb4878b0ab5a078eb4ba4a4a61d06754bc04ae8614389def1ec22a28843 type aufs (rw,relatime,si=18a301c70c84ed6,dio,dirperm1)

shm on /volume1/@docker/containers/b0e293907199fc286db4a664f4f4797b78ce22d03725ce09e1178b96d7dec1bc/mounts/shm type tmpfs (rw,nosuid,nodev,noexec,relatime,size=65536k)

none on /run/docker/netns/fc0cc1851c5e type proc (rw,nosuid,nodev,noexec,relatime)

none on /run/docker/netns/374a308122d5 type proc (rw,nosuid,nodev,noexec,relatime)

 

OR this listing:

 

"df"

 

Filesystem              1K-blocks       Used Available Use% Mounted on

/dev/md0                  2385528    1226936   1039808  55% /

none                      4018724          0   4018724   0% /dev

/tmp                      4065428       1164   4064264   1% /tmp

/run                      4065428       8812   4056616   1% /run

/dev/shm                  4065428        296   4065132   1% /dev/shm

none                            4          0         4   0% /sys/fs/cgroup

cgmfs                         100          0       100   0% /run/cgmanager/fs

/dev/mapper/cachedev_0 7747501168 7329826976 417555408  95% /volume1

none                   7747501168 7329826976 417555408  95% /volume1/@docker/aufs/mnt/29f38ffeba90da8a04e2c8aafbc2980db6ca5344c3efacde17fd251da3001ec1

none                   7747501168 7329826976 417555408  95% /volume1/@docker/aufs/mnt/13b1ccb4878b0ab5a078eb4ba4a4a61d06754bc04ae8614389def1ec22a28843

shm                         65536          0     65536   0% /volume1/@docker/containers/b0e293907199fc286db4a664f4f4797b78ce22d03725ce09e1178b96d7dec1bc/mounts/shm

 

"syno_poweroff_task -d"  <Appears to shutdown processes, and disconnects the SSH. I then just SSH back in again>

"umount /volume1" - umount: /volume1: not mounted

"umount /dev/md0" - umount: /: target is busy (In some cases useful info about processes that use the device is found by lsof(8) or fuser(1).) 

 

Indeed when I "fsck.ext4 -pvf /dev/md0"

I get: 

/dev/md0 is mounted.

WARNING!!!  The filesystem is mounted.   If you continue you ***WILL***

cause ***SEVERE*** filesystem damage.

 

So of course, I didn't proceed...  

 

Would really appreciate your insight and guru knowledge on how to get past this, so I can actually execute the system file check.  Possible something in /etc/fstab?  I'd like to run a full and complete start to finish check and automatic repair, even if it takes days to do it, and even if the filesystem is reported clean.

 

Thank You In Advance!

 

 

 

[flyride]

FYI /dev/md0 is your operating system not the volume.

 

The target mount device for your /volume1 is /dev/mapper/cachedev_0 since you have a SSD cache.  This was reported by your df command.

 

According to this unofficial guide here https://sageadviceltd.com/how-to-run-fsck-on-a-synology-nas/ the command would be

# e2fsck -Dfttvy -C 0 /dev/mapper/cachedev_0

 

Have a backup.  Good luck.