Rhubarb

Synology's Drive, Moments, Photo Station and Deduplication?

Recommended Posts

Synology's Drive, Moments, Photo Station and Deduplication?

Background:
Originally built and operating flawlessly for over 2 years since April, 2016, my Xpenology DS3615xs compatible (Supermicro X10SL7-F-0; Intel Xeon E3-1241-V3 (3.5 GHz 4 core/8 thread Haswell); 16 GB ECC RAM; began exhibitting regular hard disk disconnections/RAID 6 failures between July 2018 and late February, 2019. It was only after I replaced the Corsair VS450 450W power supply with a brand new 650W Coolermaster Gold 80+ unit on 1 March, 2019, that all of the disk error problems disappeared. The unit has now been operating flawlessly again since then, with performance far superior to consumer grade Synology products.

I am a long time user of Photo Station and I like it very much. During this time I have been concerned about disk wastage due to data duplication caused by Photo Station in particular, as it places photo files in the volume's photo folder in those cases where original photo files reside in my "/home" directory on the NAS. When these photos are uploaded from other storage areas on the NAS, such as in homes/user directories (as a result of PC backups to the NAS) the /photo storage folder is holding duplicate copies of those files.

I have for some time been running Synology's Drive and Moments applications on my DS3615xs based platform which is running DSM 6.2.2-24922. The system is also running btrfs (for it's Copy on Write features; bitrot protection, etc) which I have found to be very reliable. Other applications installed include Active Backup for Business and Synology's Hyper Backup suite (both applications not currently being used).

Drive is enabled (with Intelliversion=32) against my /home (My Drive) folder, and; the /photo, /music folders on the volume.

 

Observations:

When I copied approx. 1 TB (actually 971.95 GB in 29320 files/70 folders) of RAW image files from my /homes folder to a /homes/Drive sub-folder (virtually duplicating the files in the homes directory), the homes folder storage increased from 9.71 TB to 10.66 TB; this was accompanied by a rise in CPU resources dedicated to the Drive service (initially in order of 30+%, then reducing to 8-20%, and finally when CPU resources allocated to “Drive” were 0 (when checked in the morning after rising), total storage of the /homes folder was then shown as 10.66 TB (443,891 files, 44885 folders), but total storage on the volume remained at 27% (15.31 TB of 55.86 TB). (Note: this system also functions as a Plex server on my home network. Plex continued to serve my Samsung smart TV Plex client and a remote Plex client, without playback issues during the copy process).

Finally, when I deleted all of the 971.95 GB of files copied (duplicated) to the “Drives” sub-folder (next morning), volume usage still remained at 27% and storage in the /homes folder again reduced back to 9.71 TB (414571 files, 44814 folders).

I am wondering if anyone else has observed similar circumstances. Is this some indication of deduplication of redundant data files by the Synology Drive app?

System data:
10 x 8 TB WD-Red hard disks in RAID 6 configuration (not SHR2).

Storage Manager: Volume1 (Normal) 27% Storage Pool 1; 15.3 TB/55.9 TB
Volume 1 - Normal: RAID 6 (with 2 disk data redundancy); Total capacity: 55.86 TB; Used Capacity: 15.32 TB

Shared Folder storage:
Shared folder1: 292.5 GB (314,067,717,955 Bytes), 87013 file(s), 838 folder(s)

homes folder: 9.71 TB (10,676,301,073,915 Bytes), 414571 file(s), 44814 folder(s)

music folder: 7.67 GB (8,234,597,498 Bytes) 2416 file(s), 52 folder(s)

photo folder: 476.23 GB (511,352,024,764 Bytes), 72262 file(s), 698 folder(s)
Rod 220.73 GB (37502 files, 459 folders)
Rail 6.43 GB (402 files, 3 folders)
Moments 1014.7 KB (1 file, 2 folders)
Holidays 249.07 GB (34,357 files, 230 folders)

video folder: 4.54 TB (4,996,187,380,629 Bytes), 2964 file(s), 479 folder(s)

web folder: 452.6 KB (463,478 Bytes), 62 file(s), 7 folder(s)

 

Rhubarb
 

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.