Memory usage are increasing over weeks and the NAS finally crashed

[snakefox666]

Hello,

 

I've a strange issue with my brand new DS 916+, I had previously a HP N40L with xpenology and after more than 5 years of good services I decided to buy a real synology

 

The RAM usage of my DS916+ (8Gb) is increasing over weeks and finally the nas is crashing. After couples of investigations it seem to be related to slab memory usage that increasing dramatically. Bellow you will find the result of some commands related to slab functionnality. We can see that "cred_jar" is using most of the memory and it's "Unreclaimable"

After a reboot of the nas the memory start at more or less 20% of usage and is incresing over 2 or 3 weeks to more than 90%.

 

ash-4.4# cat /proc/meminfo
MemTotal: 8093200 kB
MemFree: 1224388 kB
Buffers: 9892 kB
Cached: 948788 kB
SwapCached: 42332 kB
Active: 652760 kB
Inactive: 1078964 kB
Active(anon): 384036 kB
Inactive(anon): 885104 kB
Active(file): 268724 kB
Inactive(file): 193860 kB
Unevictable: 1184 kB
Mlocked: 1184 kB
SwapTotal: 6954924 kB
SwapFree: 6654860 kB
Dirty: 136 kB
Writeback: 0 kB
AnonPages: 742560 kB
Mapped: 312764 kB
Shmem: 494924 kB
Slab: 4547088 kB
SReclaimable: 77560 kB
SUnreclaim: 4469528 kB
KernelStack: 13104 kB
PageTables: 60516 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 11001524 kB
Committed_AS: 7307152 kB
VmallocTotal: 34359738367 kB
VmallocUsed: 525616 kB
VmallocChunk: 34359113488 kB
DirectMap4k: 19200 kB
DirectMap2M: 8282112 kB


ash-4.4# slabtop --sort c
Active / Total Objects (% used) : 37272947 / 37437940 (99.6%)
Active / Total Slabs (% used) : 1131412 / 1131432 (100.0%)
Active / Total Caches (% used) : 147 / 269 (54.6%)
Active / Total Size (% used) : 4192989.21K / 4223942.65K (99.3%)
Minimum / Average / Maximum Object : 0.02K / 0.11K / 4096.00K

OBJS ACTIVE USE OBJ SIZE SLABS OBJ/SLAB CACHE SIZE NAME
18127720 18127720 100% 0.19K 906386 20 3625544K cred_jar
18155200 18155200 100% 0.03K 162100 112 648400K kmalloc-32
494020 493871 99% 0.19K 24701 20 98804K kmalloc-192
30919 11710 37% 0.55K 4417 7 17668K radix_tree_node
25398 24524 96% 0.63K 4233 6 16932K inode_cache
64008 61953 96% 0.18K 3048 21 12192K vm_area_struct
57920 40462 69% 0.19K 2896 20 11584K dentry
670 655 97% 16.00K 670 1 10720K kmalloc-16384
272 272 100% 32.00K 272 1 8704K kmalloc-32768
2133 2133 100% 4.00K 2133 1 8532K kmalloc-4096
107484 103436 96% 0.07K 2028 53 8112K btrfs_free_space
864 862 99% 8.00K 864 1 6912K kmalloc-8192
4098 4096 99% 1.20K 1366 3 5464K raid5-md2
3768 2229 59% 1.19K 1256 3 5024K btrfs_inode
53900 10859 20% 0.09K 1225 44 4900K btrfs_extent_state
70092 60033 85% 0.06K 1188 59 4752K anon_vma_chain
5705 5676 99% 0.70K 1141 5 4564K proc_inode_cache
14745 10194 69% 0.25K 983 15 3932K filp
3368 3311 98% 1.00K 842 4 3368K kmalloc-1024
4055 4020 99% 0.70K 811 5 3244K shmem_inode_cache
46551 35428 76% 0.06K 789 59 3156K kmalloc-64
2244 1915 85% 1.02K 748 3 2992K ext4_inode_cache
43282 32811 75% 0.05K 646 67 2584K anon_vma
5096 5096 100% 0.50K 637 8 2548K kmalloc-512
8151 1422 17% 0.30K 627 13 2508K btrfs_delayed_node
18060 6824 37% 0.12K 602 30 2408K kmalloc-128
20094 20072 99% 0.11K 591 34 2364K sysfs_dir_cache
962 962 100% 2.00K 481 2 1924K kmalloc-2048
880 877 99% 1.97K 440 2 1760K task_struct
5605 1205 21% 0.20K 295 19 1180K btrfs_extent_buffer
9 9 100% 128.00K 9 1 1152K kmalloc-131072
405 385 95% 2.06K 135 3 1080K sighand_cache
7440 3999 53% 0.12K 248 30 992K kmalloc-96
273 256 93% 2.06K 91 3 728K idr_layer_cache
860 811 94% 0.69K 172 5 688K sock_inode_cache
5550 2592 46% 0.10K 150 37 600K buffer_head


ash-4.4# cat /proc/slabinfo | grep -i cred_jar
cred_jar 18127976 18128000 192 20 1 : tunables 120 60 8 : slabdata 906400 906400 0

 

I tried to contact the synology support but the only answer is "Grant us remote control" and I don't want to let them access my private NAS

Any help is welcome, thanks guys 😎