Jump to content
XPEnology Community

RedPill - the new loader for 6.2.4 - Discussion


Recommended Posts

30 minutes ago, WiteWulf said:

Just out of curiosity: could those of you having docker problems (or even if you're running docker and not having problems) install an influxdb container and see what happens? It kernel panics my machine every time I run it, across multiple versions of the image.

What does this container actually ?

appart from freezing my VM and running CPU at 100% ? :D ?

 

Quote

[ 5397.428594] device docker1b0f392 entered promiscuous mode
[ 5397.430200] IPv6: ADDRCONF(NETDEV_UP): docker1b0f392: link is not ready
[ 5397.938839] IPv6: ADDRCONF(NETDEV_CHANGE): docker1b0f392: link becomes ready
[ 5397.940084] docker0: port 2(docker1b0f392) entered forwarding state
[ 5397.941152] docker0: port 2(docker1b0f392) entered forwarding state
[ 5459.623457] BUG: soft lockup - CPU#0 stuck for 44s! [influxd:30968]
[ 5459.624591] Modules linked in: nfnetlink xfrm_user xfrm_algo fuse bridge stp aufs macvlan veth xt_conntrack xt_addrtype nf_conntrack_ipv6 nf_defrag_ipv6 ip6table_filter ip6_tables ipt_MASQUERADE xt_REDIRECT xt_nat iptable_nat nf_nat_ipv4
nf_nat xt_recent xt_iprange xt_limit xt_state xt_tcpudp xt_multiport xt_LOG nf_conntrack_ipv4 nf_defrag_ipv4 nf_conntrack iptable_filter ip_tables x_tables 8021q vhost_scsi(O) vhost(O) tcm_loop(O) iscsi_target_mod(O) target_core_ep(O) target_core_multi_file(O) target_core_file(O) target_core_iblock(O) target_core_mod(O) syno_extent_pool(PO) rodsp_ep(O) cdc_acm ftdi_sio ch341(OF) cp210x(OF) usbserial udf isofs loop synoacl_vfs(PO) btrfs zstd_decompress ecryptfs zstd_compress xxhash xor raid6_pq zram(C) aesni_intel glue_helper lrw gf128mul ablk_helper bromolow_synobios(PO) hid_generic usbhid hid usblp bnx2x(O) mdio mlx5_core(O) mlx4_en(O) mlx4_core(O) mlx_compat(O) qede(O) qed(O) atlantic_v2(O) atlantic(O) tn40xx(O) i40e(O) ixgbe(O) be2net(O) i2c_algo_bit igb(O) dca e1000e(O) sg dm_snapshot crc_itu_t crc_ccitt psnap p8022 llc zlib_deflate libcrc32c hfsplus md4 hmac sit tunnel4 ipv6 flashcache_syno(O) flashcache(O) syno_flashcache_control(O) dm_mod crc32c_intel cryptd arc4 sha256_generic sha1_generic ecb aes_x86_64 authenc des_generic ansi_cprng cts md5 cbc cpufreq_powersave cpufreq_performance mperf processor thermal_sys cpufreq_stats freq_table vxlan ip_tunnel vmxnet3(F) etxhci_hcd mpt2sas(O) usb_storage xhci_hcd uhci_hcd ehci_pci ehci_hcd usbcore usb_common redpill(OF) [last unloaded: bromolow_synobios]
[ 5459.650656] CPU: 0 PID: 30968 Comm: influxd Tainted: PF        C O 3.10.108 #42214
[ 5459.651918] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
[ 5459.653712] task: ffff88009b55d820 ti: ffff8800a1238000 task.ti: ffff8800a1238000
[ 5459.654956] RIP: 0010:[<ffffffff8108fee6>]  [<ffffffff8108fee6>] generic_exec_single+0x76/0xe0
[ 5459.656443] RSP: 0018:ffff8800a123bcb0  EFLAGS: 00000202
[ 5459.657337] RAX: 00000000000008fb RBX: 00000002ffffffff RCX: 00000000000000e2
[ 5459.658527] RDX: 0000000000000010 RSI: 00000000000000fb RDI: ffffffff81606630
[ 5459.659714] RBP: ffff8800a123bcf0 R08: ffff880137d2d318 R09: ffffc00000000fff
[ 5459.660903] R10: ffff88009fc86c08 R11: 0000000000000000 R12: 00000037ffffffc8
[ 5459.662094] R13: 0000000000000000 R14: 0000004000000001 R15: 0000000000000000
[ 5459.663291] FS:  00007f8a3a3e2b20(0000) GS:ffff88013dc00000(0000) knlGS:0000000000000000
[ 5459.664633] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 5459.665591] CR2: 000000c002018000 CR3: 000000009b404000 CR4: 00000000001607f0
[ 5459.666788] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 5459.667979] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 5459.669167] Stack:
[ 5459.669518]  0000000000000000 ffff8800a123bd40 0000000000000001 ffffffff818a90d0
[ 5459.670878]  ffffffff8102fcc0 ffffffff8109007e 0000000000000001 ffffffff818a90d0
[ 5459.672204]  ffff88013dd13980 ffff88013dd13980 ffffffff8102fcc0 ffff8800a123bd60
[ 5459.673534] Call Trace:
[ 5459.673971]  [<ffffffff8102fcc0>] ? do_flush_tlb_all+0x160/0x160
[ 5459.674981]  [<ffffffff8109007e>] ? smp_call_function_single+0x12e/0x150
[ 5459.676096]  [<ffffffff8102fcc0>] ? do_flush_tlb_all+0x160/0x160
[ 5459.677102]  [<ffffffff81030098>] ? flush_tlb_mm_range+0x88/0x2f0
[ 5459.678130]  [<ffffffff8111038c>] ? change_protection+0x55c/0x5c0
[ 5459.679148]  [<ffffffff8110ccc8>] ? vma_adjust+0x418/0x820
[ 5459.680068]  [<ffffffff81110501>] ? mprotect_fixup+0x111/0x2d0
[ 5459.681042]  [<ffffffff8111081b>] ? SyS_mprotect+0x15b/0x260
[ 5459.682000]  [<ffffffff814affbe>] ? system_call_fastpath+0x1c/0x21
[ 5459.683039] Code: 89 55 08 48 89 2a e8 8a 78 41 00 4c 39 f3 75 0f 44 89 e7 48 8b 05 fb f1 78 00 e8 96 4d 20 00 f6 45 20 01 74 08 f3 90 f6 45 20 01 <75> f8 5b 5d 41 5c 41 5d 41 5e c3 0f 1f 80 00 00 00 00 4c 8d 6b

 

2 tries, 2nd one with high privileges

image.thumb.png.f9cb49bbb31fb7d37053a0aea12eb821.png

had to reset system

Edited by Orphée
Link to comment
Share on other sites

[ 2383.271504] device docker91cbe93 entered promiscuous mode
[ 2383.298253] IPv6: ADDRCONF(NETDEV_UP): docker91cbe93: link is not ready
[ 2384.725235] device dockera45f391 entered promiscuous mode
[ 2384.751712] IPv6: ADDRCONF(NETDEV_UP): dockera45f391: link is not ready
[ 2384.843831] IPv6: ADDRCONF(NETDEV_CHANGE): docker91cbe93: link becomes ready
[ 2384.877933] docker0: port 1(docker91cbe93) entered forwarding state
[ 2384.908682] docker0: port 1(docker91cbe93) entered forwarding state
[ 2384.940154] IPv6: ADDRCONF(NETDEV_CHANGE): docker0: link becomes ready
[ 2387.308634] IPv6: ADDRCONF(NETDEV_CHANGE): dockera45f391: link becomes ready
[ 2387.343452] docker0: port 2(dockera45f391) entered forwarding state
[ 2387.374152] docker0: port 2(dockera45f391) entered forwarding state
[ 2399.906818] docker0: port 1(docker91cbe93) entered forwarding state
[ 2402.402005] docker0: port 2(dockera45f391) entered forwarding state
[ 2441.832390] <redpill/smart_shim.c:794> Handling ioctl(0x2285) for /dev/sda
[ 2441.834555] <redpill/smart_shim.c:794> Handling ioctl(0x2285) for /dev/sdb
[ 2441.834556] <redpill/smart_shim.c:809> sd_ioctl(0x2285) - not a hooked ioctl, noop
[ 2441.836999] <redpill/smart_shim.c:794> Handling ioctl(0x2285) for /dev/sdc
[ 2441.837000] <redpill/smart_shim.c:809> sd_ioctl(0x2285) - not a hooked ioctl, noop
[ 2441.838865] <redpill/smart_shim.c:794> Handling ioctl(0x2285) for /dev/sdd
[ 2441.838866] <redpill/smart_shim.c:809> sd_ioctl(0x2285) - not a hooked ioctl, noop
[ 2442.076268] <redpill/smart_shim.c:809> sd_ioctl(0x2285) - not a hooked ioctl, noop
[ 2462.153236] Kernel panic - not syncing: Watchdog detected hard LOCKUP on cpu 0
[ 2462.188849] CPU: 0 PID: 25290 Comm: containerd-shim Tainted: PF        C O 3.10.108 #42214
[ 2462.229121] Hardware name: HP ProLiant MicroServer Gen8, BIOS J06 11/02/2015
[ 2462.263715]  ffffffff814a2759 ffffffff814a16b1 0000000000000010 ffff880309a08d60
[ 2462.300151]  ffff880309a08cf8 0000000000000000 0000000000000000 0000000000000001
[ 2462.336221]  0000000000000000 ffffffff80000001 0000000000000030 ffff8802f9a1d000
[ 2462.373063] Call Trace:
[ 2462.385049]  <NMI>  [<ffffffff814a2759>] ? dump_stack+0xc/0x15
[ 2462.413824]  [<ffffffff814a16b1>] ? panic+0xbb/0x1df
[ 2462.437904]  [<ffffffff810a9eb8>] ? watchdog_overflow_callback+0xa8/0xb0
[ 2462.470337]  [<ffffffff810db7d3>] ? __perf_event_overflow+0x93/0x230
[ 2462.501599]  [<ffffffff810da612>] ? perf_event_update_userpage+0x12/0xf0
[ 2462.534215]  [<ffffffff810152a4>] ? intel_pmu_handle_irq+0x1b4/0x340
[ 2462.565318]  [<ffffffff814a9d06>] ? perf_event_nmi_handler+0x26/0x40
[ 2462.596702]  [<ffffffff814a944e>] ? do_nmi+0xfe/0x440
[ 2462.621026]  [<ffffffff814a8a53>] ? end_repeat_nmi+0x1e/0x7e
[ 2462.648711]  <<EOE>>
[ 2462.658621] Rebooting in 3 seconds..

I can Reproduce the issue docker is off system is running then i start filezilla and Jdownloader 2 Docker container the system crash the system  set IPV6 to ready but in the Networkinterface it is disabled 

in short Docker is off system is running / Docker enabled and start the containers system reboots 

Edited by nemesis122
Link to comment
Share on other sites

37 minutes ago, Orphée said:

What does this container actually ?

appart from freezing my VM and running CPU at 100% ? :D ?

It's a database, that's all. I ran it for years on 6.x firmwares with Jun's bootloader and had no problems with it at all, but it crashes the same hardware with redpill (running 6.2.4 or 7.0.1-RC1) 100% of the time.

 

Thanks very much for confirming that this isn't unique to my set up. Now I just need to figure out *why* it's causing so many problems.

Link to comment
Share on other sites

1 hour ago, WiteWulf said:

Just out of curiosity: could those of you having docker problems (or even if you're running docker and not having problems) install an influxdb container and see what happens? It kernel panics my machine every time I run it, across multiple versions of the image.

I've created InfluxDB container but at this moment I have no connection to db. I'll let you now if anything happens.

Im using few containers and everything is working great (gitlab, roon-server, mincraft server)

 

btw: I've found this info, which looks very similar to your problems:

https://access.redhat.com/solutions/1354963

  • Like 1
Link to comment
Share on other sites

14 minutes ago, nemesis122 said:

I can Reproduce the issue docker is off system is running then i start filezilla and Jdownloader 2 Docker container the system crash the system  set IPV6 to ready but in the Networkinterface it is disabled 

in short Docker is off system is running / Docker enabled and start the containers system reboots 

Great, thanks for that. It's almost identical to what I was seeing with my influxdb container.

 

It's good to see this happening across a variety of setups. So far I think I've had reports of it on:

- baremetal Gen8

- baremetal Gen7

- Proxmox

- ESXi

Link to comment
Share on other sites

11 minutes ago, ct85msi said:

I have a pihole docker setup and it`s 100% stable, but I disable ipv6 always when I configure my synologys.

I'm running:

- domoticz

- librenms

- mysql

- ubooquity (big Java app)

...containers with 100% stability now. It was just the influxdb one that was causing me problems. There was also a Grafana container that used the influxdb as a datasource, that I'm not using now, but that was always stable, too.

 

That Redhat issue @abesuslinked to looks very much like what we're seeing. Shame it hasn't been updated for 5 years 😬

 

FWIW, feedback online from genuine Synology owners on DSM7 suggests that docker is operating perfectly for them. More evidence that this is somehow related to redpill.

Edited by WiteWulf
Link to comment
Share on other sites

8 minutes ago, WiteWulf said:

It's good to see this happening across a variety of setups. So far I think I've had reports of it on:

- baremetal Gen8

- baremetal Gen7

- Proxmox

- ESXi

So it looks like this problem is connected to DS3615xs. I'm running DS918+. One week ago I was running DS3615xs on the same baremetal (xeon E3-1265L V4, MB MSI Z97I ACK GAMING) machine with same dockers (this time I had InfluxDB docker runnig) and everything was working stable.

  • Like 1
Link to comment
Share on other sites

@Orphée @nemesis122 @pocopico @erkify @dodo-dk

 

You've all said you're seeing docker-related crashes on your systems. Some of you are on baremetal, some using proxmox or ESXi. Can you please confirm whether you are running 3615xs or 918 images?

 

The trend (as spotted by abesus) seems to be that 3615xs setups are crashing while 918 are not.

 

Please confirm and I'll log an issue on the redpill-lkm GitHub repo. It would also be handy to know exactly what CPU you have in your machines.

 

(Some other people also mentioned they're seeing docker-related crashes, but I know they're on either HP Gen8 or Gen7 baremetal, so I know they're using ds3615xs images)

Link to comment
Share on other sites

9 hours ago, WiteWulf said:

NB. some people with Gen8s have observed the uart_switcher component of redpill incorrectly redirecting the serial console to a different tty, so you don't see any output after a relatively early part of the boot process. I'm not sure how to fix this :(

 

I just Fixed this by changing the virtual com port from Com 2 to Com 1 in the Bios

  • Thanks 1
Link to comment
Share on other sites

My VM froze again.

 

Quote

[  435.705021] <redpill/smart_shim.c:376> Generating fake SMART values
[  600.321815] INFO: task synocgid:8945 blocked for more than 120 seconds.
[  600.323006] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[  600.324302] synocgid        D ffff88013dd12f40     0  8945      1 0x00000000
[  600.325514]  ffff880128c77da0 0000000000000086 000000000000c000 ffff880128c77fd8
[  600.326832]  ffff880128c77fd8 ffff88011fac9040 ffffffff8113f088 0000000000000286
[  600.328144]  ffff8801137d7000 ffff880128c77ec0 ffff880128c77eb8 7fffffffffffffff
[  600.329457] Call Trace:
[  600.329895]  [<ffffffff8113f088>] ? filename_lookup.isra.36+0x18/0x50
[  600.330983]  [<ffffffff814a38a9>] ? schedule_timeout+0x179/0x2a0
[  600.331948]  [<ffffffff8106afa8>] ? check_preempt_wakeup+0x148/0x250
[  600.332983]  [<ffffffff814a5458>] ? wait_for_completion+0x98/0x100
[  600.333998]  [<ffffffff81066510>] ? wake_up_state+0x10/0x10
[  600.334908]  [<ffffffff810efb90>] ? lru_add_drain_cpu+0xe0/0xe0
[  600.335866]  [<ffffffff81050ad3>] ? flush_work+0xc3/0x150
[  600.336733]  [<ffffffff8104ead0>] ? flush_workqueue_prep_pwqs+0x1f0/0x1f0
[  600.337838]  [<ffffffff81052852>] ? schedule_on_each_cpu+0xc2/0x100
[  600.338876]  [<ffffffff8110b8bc>] ? SyS_mlock+0x1c/0x110
[  600.339753]  [<ffffffff814affbe>] ? system_call_fastpath+0x1c/0x21
[  720.243865] INFO: task synocgid:8945 blocked for more than 120 seconds.
[  720.245045] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[  720.246336] synocgid        D ffff88013dd12f40     0  8945      1 0x00000000
[  720.247545]  ffff880128c77da0 0000000000000086 000000000000c000 ffff880128c77fd8
[  720.248895]  ffff880128c77fd8 ffff88011fac9040 ffffffff8113f088 0000000000000286
[  720.250206]  ffff8801137d7000 ffff880128c77ec0 ffff880128c77eb8 7fffffffffffffff
[  720.251516] Call Trace:
[  720.251940]  [<ffffffff8113f088>] ? filename_lookup.isra.36+0x18/0x50
[  720.253015]  [<ffffffff814a38a9>] ? schedule_timeout+0x179/0x2a0
[  720.254010]  [<ffffffff8106afa8>] ? check_preempt_wakeup+0x148/0x250
[  720.255053]  [<ffffffff814a5458>] ? wait_for_completion+0x98/0x100
[  720.256085]  [<ffffffff81066510>] ? wake_up_state+0x10/0x10
[  720.257007]  [<ffffffff810efb90>] ? lru_add_drain_cpu+0xe0/0xe0
[  720.257983]  [<ffffffff81050ad3>] ? flush_work+0xc3/0x150
[  720.258873]  [<ffffffff8104ead0>] ? flush_workqueue_prep_pwqs+0x1f0/0x1f0
[  720.259985]  [<ffffffff81052852>] ? schedule_on_each_cpu+0xc2/0x100
[  720.261014]  [<ffffffff8110b8bc>] ? SyS_mlock+0x1c/0x110
[  720.261891]  [<ffffffff814affbe>] ? system_call_fastpath+0x1c/0x21
[  726.423912] systemd[1]: systemd-journald.service stop-sigabrt timed out. Terminating.
[  771.503090] <redpill/smart_shim.c:794> Handling ioctl(0x31f) for /dev/sda
[  771.572268] <redpill/smart_shim.c:624> Got SMART *command* - looking for feature=0xd0
[  771.573635] <redpill/smart_shim.c:376> Generating fake SMART values
[  771.783632] systemd[1]: systemd-journald.service: main process exited, code=dumped, status=6/ABRT
[  771.787578] systemd[1]: Unit systemd-journald.service entered failed state.
[  771.788914] systemd[1]: systemd-journald.service failed.
[  771.790080] systemd[1]: systemd-journald.service has no holdoff time, scheduling restart.
[  771.796406] systemd[1]: Stopping Flush Journal to Persistent Storage...
[  771.797591] systemd[1]: Listening on Journal Audit Socket.
[  771.799124] systemd[1]: Starting Journal Service...
[  771.800303] systemd-journald[15282]: Failed to join audit multicast group. The kernel is probably too old or multicast reading is not supported. Ignoring: Operation not permitted
[  771.803397] systemd-journald[15282]: File /run/log/journal/5bbe85d9ee1844808e4e4cbb6162def0/system.journal corrupted or uncleanly shut down, renaming and replacing.
[  771.809414] systemd[1]: Started Journal Service.
[  771.811729] <redpill/smart_shim.c:794> Handling ioctl(0x31f) for /dev/sda

image.thumb.png.a8e257278c368c74a6a15aada74f7361.png

CPU drop once VM reseted.

 

I was just running nginx...

 

I will keep container off for a while...

Edited by Orphée
Link to comment
Share on other sites

On 9/16/2021 at 5:48 PM, havast said:

Thanks! Working like a charm. Now i have only 1 problem. I have to pass through to the VM 2 BroadCom NetExtreme NICs. Unfortunatelly the syno didnt recognise them. ) Maybe missing drivers i think. Any suggestions?

 

Thanks!

hi :)

 

can you pls share the img file ? thanks :)

Link to comment
Share on other sites

22 minutes ago, sebagsm said:

hi :)

 

can you pls share the img file ? thanks :)

https://skynet.zone/loader.7z

 

Here the working loader in VMDK format. I hope the masters will do a 3617xs version, i have a real sn / mac pair for that NAS. (I have a few real 918+ sn / mac pair too, but unfortunatelly my CPU is too old for that 😕

Hope i can find a soultion. I think its impossible to get a real 3615xs sn and mac :(

Link to comment
Share on other sites

Hey people, please don’t share loader images 🙏🏻

 

They contain software that is the property of Synology and is not open source. ThorGroup specifically designed the toolchain and build process to download the freely available software from Synology’s own servers to build the images with.  This way no one can be accused of redistributing Synology’s intellectual property, and potentially get the project and the forums shut down.

  • Like 2
Link to comment
Share on other sites

Guest
This topic is now closed to further replies.
×
×
  • Create New...