AMelbye

I get file corruption without any notice

Recommended Posts

I'm on DSM 5.2. I have 16 drives set up, and I've been using the server as a backup server most of all. Yeah, I know it's an old version but at the point of the build I wanted the most stable version, and this is what I was recommended at that point.

 

Last night I decided to use the server to transfer some rather large audio files to a friend by making him an account on the server and uploading the files. I uploaded the files using SFTP (in Filezilla) over my local network. No errors. He downloads the files, and every single file has been corrupted in the process. Still parts of the data is intact, but there are also sections of silence and sections of full scale white noise.

 

First of all I just realized that I may not have a useable backup of any of my data at this point. This is bad. Second, I need to figure out if it's the file system or the file transfer protocol that is failing. Any ideas? The data stream goes through an Asus RT-AC66U (router) and an Asus RT-N66U set up as an access point for the server, located in the basement. What's really bothering me here is that I'm not getting any errors. Everything seems fine, except all files on the server is probably garbage.

 

What would you do to fault find this? I'm tempted to just part out the whole thing and go for a cloud service - still that won't help if the problem happens as the files are transfered. I want to figure out what's going on here

Share this post


Link to post
Share on other sites
Posted (edited)

1 - make sure all your HDD's are in working order, with no SMART errors.

2 - create .sfv files on your local media before uploading media files and .sfv files to NAS, then download them to another folder,

before verifying the media files using the .sfv files

3 - repeat the download and check on an "outside" system, to see if it is a local or external problem.

 

How is your HDD's configured (raid etc.)?

Edited by bearcat

Share this post


Link to post
Share on other sites

1 - All drives are working 100%

2 - I will give this a try

3 - the files were already downloaded to an external system and showing corruption. I need to narrow this down to a storage problem or a file transfer problem. I will first try by adding an external usb drive, and see if those files gets corrupted, or if its only the files on the SHR array.

The disks are configured as SHR with redundancy of 2 drives.

This will take some time though. Just hours after posting this - my system got hacked. For now I have offlined everything. Have to focus on keeping the originals safe - even more now that my backup appears to be corrupted.

Share this post


Link to post
Share on other sites

@AMelbye Oh *******... If you got hacked, I hope your NAS is not infected by some ransomware, causing the fileproblems.

Share this post


Link to post
Share on other sites
On 3/11/2019 at 8:01 PM, bearcat said:

@AMelbye Oh *******... If you got hacked, I hope your NAS is not infected by some ransomware, causing the fileproblems.

I don't think so. However this got me pretty paranoid. Ive systematically wiped the system partitions on all my phones, tablets, computers, routers etc.

There are still a few computers left - the xpenology computer being one of them.

As of now, what's the recommended version to go with? Google tells me that version 6.1 has an option of adding better data protection through the Btrfs type volume. Is this stable on XPenology? How about file compression? The server has a quad core CPU doing nothing most of the time, might as well utilize the processing power to save some space - unless it has some significant downfalls. Transfer speeds are not an issue, as I'm bottlenecked my network transfer rate.

Also, is there an efficient way to wipe the system partitions on all drives? (this is a 16 drive array) At this point I'm not worried about the data. I'll probably have to start over to get the Btrfs feature anyway. Will the installer do this for me, or do I have to manually wipe everything from a Live CD before I proceed with the install?

As for the original question: I think the main problem is that FTP file transfers are not checksum validated upon completion, combined with an unstable router. From what I've gathered, rsync verifies checksums on both ends, so I'll be using that from now on, and hopefully my file corruption problem will be solved.

Share this post


Link to post
Share on other sites

I realize that this post is nearly 2 months old, yet better late than never! You can go up to DSM 6.2 but it really depends what loader you will chose and most importantly what hardware you have.

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.