RedPill - the new loader for 6.2.4 - Discussion


Recommended Posts

On 7/28/2021 at 11:05 PM, jumkey said:

echo "args: -device 'qemu-xhci,addr=0x18' -drive 'id=synoboot,file=/var/lib/vz/images/{VM_ID}/synoboot.img,if=none,format=raw' -device 'usb-storage,id=synoboot,drive=synoboot,bootindex=5'" >> /etc/pve/qemu-server/{VM_ID}.conf

 

Create on the first Partition in the Bootloader following directory structure: EFI/boot

 

Then touch EFI/boot/SynoBootLoader.conf

@haydibe

you can try debian:latest (10)?

Edited by jumkey
  • Thanks 1
Link to post
Share on other sites
3 hours ago, haydibe said:

The redpill tool chain docker image builder is updated to v0.4:

- proper DSM7 support for apollolake (thnx @jumkey)

- switched from kernel sources based build to toolkit dev based builds for DSM6.2.4 and DSM7.0 (thnx @jumkey

- make targets for bromolow and apollolake merged: platform and version must be configure in the Makefile now

- image is ~720-780MB instead of ~1.200-1.500MB now

 

Note: since toolkit dev lacks the required sources for fs/proc, they are taken from the extracted DSM6.2.4 kernel sources.

          The build requires the sources for this single folder, but does not use the kernel source to build the redpill.ko module. 

 

If you see something is wrong in how the toolchain is build or have ideas how to make it better: please let me know.

For every other problem: please address it to the community - I don't know more than others do. 

 

Note#2: before someone asks: I haven't managed a successfull installation/migration with the created bootloader so far. I am testing exclusivly on ESXi 6.7. The migration always stops at 56% and bails out with error 13.

 

Usage:

- (on host) configure the Makefile and configure TARGET_PLATFORM (default: bromolow) and TARGET_VERSION (default: 6.2 - will build 6.2.4)

- (on host) create your own user_config.json or edit the USERCONFIG_* variables in the Makefile

- (on host) build image: make build_image

- (on host) run container: make  run_container

- (in container) build redpill.ko module and redpill bootloader image: make build_all

 

After running `make build_all` the created redpill bootloader image will be present in the ./image folder on the host.

 

Tested with hosts: Ubuntu 18.04 VM, Ubuntu 20.04 WSL2 and XPE  (the make binary to build on Synology/XPE can be found here)

 

Dependencies: make and docker

redpill-tool-chain_x86_64_v0.4.zip 5.01 kB · 3 downloads

Looks like Bromolow for DSM 7.0 still fails to compile since there is no config in the redpill-load repo.

 

Everything else seems to work fine on MacOS 10.15.7.  However v0.3 was faster to `make build_image`, which is odd since the image is smaller.

Edited by cwiggs
Link to post
Share on other sites
4 hours ago, jumkey said:

you can try debian:latest (10)?

Sure, after work.

 

I just need to change two digits of the value of the DOCKER_BASE_IMAGE variable in the outer Makefile for that. 

 

3 hours ago, cwiggs said:

Looks like Bromolow for DSM 7.0 still fails to compile since there is no config in the redpill-load repo.

 

Technicaly it's possible to create an image that sets up the toolchain for Bromolow DSM7.0, but indeed so far there is no loader profile for it. 

Link to post
Share on other sites
3 hours ago, cwiggs said:

Everything else seems to work fine on MacOS 10.15.7.  However v0.3 was faster to `make build_image`, which is odd since the image is smaller.

I switch to a mulit-stage Dockerfile. The first "extract" image is volatile and simply extracts the kernel source and the toolkit dev, the second image copies only a copy of folders from bother extracted archvies into the main image. Each PLATFORM/VERSION combination will create the extract image, every follow up build will leverage docker's build cache to speed up things :)

Link to post
Share on other sites
11 minutes ago, kelvin701 said:

hi guys, anyone try it on UNRAID?

@kelvin701 I did, and as I posted a few posts back to Haydibe that using his docker image on Unraid it does not boot up. I do not know what is going on. Maybe is something that has to do with GRUB... I have no idea really. Feel free to try though :)

 

Keep in mind that is still pre-alpha version and is mainly for developers. If it does not work we can wait patiently for ThorGroup to finish their amazing work on the bootloader.

Edited by gadreel
Link to post
Share on other sites
5 minutes ago, gadreel said:

@kelvin701 I did, and as I posted a few posts back to Haydibe that using his docker image on Unraid it does not boot up. I do not know what is going on. Maybe is something that has to do with GRUB... I have no idea really. Feel free to try though :)

me 2, my image always stopped at grub before loading kernel.  Since I am not good at it, probably my mistake in following the procedure.

(proxmox 7.0)

Link to post
Share on other sites

Status Update! ;)

Recently we were working hard in the kernel space to bring even more missing things and bring it into a more stable state. That meant we had little time to respond but we're blown away by the number of posts here!


mfgBIOS shimming stability
As we discussed here before there were moments where some BIOS functions weren't shimmed or there were flaky. This caused the OS to sometimes work correctly but sometimes fail the 24h test. Guess how easy was that to debug and fix :D
However, we did it and now the mfgBIOS shimming should be almost bulletproof!


RTC emulation
We've added a full RTC proxy layer. This wasn't strictly needed for 3515xs series. However, 918+ and other models use RTC chip which is not natively/fully supported for the Linux kernel. This means that various subsystems rely on mfgBIOS to update the time from RTC on boot. Since that chip does not exist outside of the official hardware we needed to go around that. In short the LKM code plugs into mfgBIOS and serves as a proxy layer between what's a real RTC (or a one from hypervisor) and what mfgBIOS can understand.

Currently, the time set/get is implemented. Alarms are mocked to the point that mfgBIOS doesn't complain. In fact the BIOS code can even set these alarms but they will be erased upon reboot. We didn't bother actually writing them to the RTC. The reason being a) we've never seen anybody using it and b) most RTCs [or actually the hw connected to them] is broken to the point that they don't work anyway (or only work from certain sleep states).
If the community desires we can add that.


Inverting serial ports on 3615xs
We hate it. We did it. It's sort of working and it's a big hack. We will most likely disable it by default as this isn't even fully stable. Also this method cannot work on kernel v4 which... well, v7 and 3615xs. Whoever in syno had this brilliant idea should seriously think again before pulling such a stunt again. This breaks a lot of stuff.


Fix ttyS0 on 918+
On some platforms first serial port is intentionally disabled. We added a small hack to re-enable it.


Virtual Serial Port / UART
This is big. In many ways actually. This single commit with over 1300 new lines and the patch taking 70KB was by far the biggest addition. The complexity of this part is really something we grind our teeth on. To make sure the solution works on all kernels without hacks we had to emulate a physical UART chip. So we did. We wrote a whole software-based emulator of National Semiconductors 16550A UART controller, along with support for FIFOs. Additionally, as we wanted the code to perform great we raised the bar and wrote an additional layer of virtual IRQ (as the real ones cannot be used for many reasons here) which actually made the module multithreaded in the kernel.

This whole thing took, across all of us, probably close to 150 hours to develop. There were many ideas and failed endeavors. But the full chip emulation should serve us a long way you truly cannot distinguish it from a real thing (as it is not hacking the kernel in any way).

 

If anyone wants to borrow that code for something non-DSM related feel free - it's actually modular and not coupled with the RedPill LKM in any substantial way.

 

Currently, this vUART stuff is not running in the actual LKM. It is meant to be a component to implement PMU emulation. That part requires a two (hopefully) simple functions which react to the data from the DSM and respond like the PMU would. This will also allow us to give a simple hook to anybody who's willing to connect a hardware PMU to get LEDs and what not working ;)


*********************************************************************************

 

v7 testing
We're so happy to see the community effort to add v7. As it is now available for 3615xs we will definitelly jump into it and merge PRs if they pass ;)

AFTER merging them we will probably reorganize template files a bit as that symlink-hell was a bad idea.

 

Now let us respond to everything:

On 8/2/2021 at 10:14 PM, flyride said:

Just FYI for those testing, probably unwise to display your serial number (even if it is generated), as it could theoretically give Synology a way to identify you/your IP/etc.

RedPill actually blocks telemetry as well ;) If you want to check that just go the Control Panel and click "What's new" on an update - you will not see a syno page.

 

 

On 8/2/2021 at 10:54 PM, pehun said:

Thanks for all your work...

Does it work on baremetal?

Would it be possible to have a Plug and play loader, as jun's one?

Currently it SHOULD work on some baremetal configs but it's definitely not tested. It will be plug-and-play when it's stable.

 

 

On 8/3/2021 at 7:31 PM, supajason said:

@RedwinX I'm getting the same issue; When I download the boot image from ESXi it's been formatted.

When I click to install the pat file it shows (me anyway) 3 disks that be formatted. 2 yes but the 3rd must be the boot image so I'm guess the VID & PID are wrong(?)

We confirmed that ESXi doesn't allow adding virtual USB drives, so we need SATA DOM boot support. Stay tuned.

Setting vid/pid or any other options will NOT make the loader work without LKM changes (ok, technically MAYBE if you recreate PCI tree in Proxmox but .... no, that's not the solution and definitelly not for ESXi or baremetal ;)).

 

 

On 8/3/2021 at 11:27 PM, progressives said:

What should I do to fix this error?

캡처.JPG

You didn't provide RedPill LKMs. You need to compile them if you want to play with the current build.

 

 

On 8/4/2021 at 7:44 PM, scoobdriver said:

could anybody provide any help with compiling the model with toolkit , 

@Vortex said this is possible. .

 

When I try to do this I get the following error

You shouldn't try to compile the kernel but the module. Executing make in the kernel dir will try to build the whole kernel (which doesn't make any sense as it will not be usable really). If you're having problems compiling the module the release is not meant for you (yet).

 

 

On 8/4/2021 at 9:28 PM, Aigor said:

Have you followed telegram chat? 
RedPill-TTG

We weren't able to access the telegram as they force us to give a valid, physical phone number. For obvious reasons we didn't give them one ;) When we have a moment to arrange that we will arrive for sure. However, as the current stage is still intended for developers (and we're BEGGING normal users to not use it due to missing stuff which causes various trips in the DSM itself).

Unless there's some way to use telegram without a phone number which we don't know about?

 

 

On 8/4/2021 at 11:42 PM, progressives said:

I'm an amateur.
That's why we're stuck. I need your help.

how to use redpill-lkm

DO NOT USE IT. Seriously, we don't want to sound rude but if we say this is a development release we mean it. The kernel module lacks some functionality required for the DSM to stay stable. In the current state data loss is a possibility.

 

 

On 8/5/2021 at 10:21 PM, loomes said:

I have compiled evdev.ko (kernel 4.4.180+ DSM7) and arranged a tar file for working Power Button in Proxmox: https://www.file-upload.net/download-14654672/powerbutton-7.0.zip.html

Copy to Machine an run install.sh.

Thanks! We need some smart way for adding kernel extensions outside of the ramdisk (as it's limited in size). Something like Jun's extra.lzma. However, as it uses a different method of loading we cannot simply merge ramdisks automatically hmm...

 

 

On 8/6/2021 at 3:11 AM, haydibe said:

For all of you that want a "redpill tool chain".

Well, I created a make based project that creates a docker image with the toolchain for brolow or apollolake

Great job! We love the approach of going docker. We honestly didn't look into the docker toolchain (yet!) you've built but we're excited!

What do you think about incorporating it into RedPill-Load repository? We think it will be a great addition so that the whole thing can be run on any dockerized system.
Drop us a PM to not pollute the thread. In general it's great - the only thing which we will keep is making sure the user can actually use JSON config and not makefile - we pulled teeth for hours to make it happen in bash as JSON is user-friendly while Makefile... well :D

 

 

On 8/7/2021 at 8:31 AM, djvas335 said:

Thanks guys, I used this docker image builder to build a loader for a baremetal Microserver Gen7, its working so far, however with an intel network card, no onboard network, will test some more and report the results

Onboard network will require a separate kernel extension. This is the same issue as with power button kernel extension - we don't have a good way for not including all extensions in the main ramdisk (which cannot grow too much, especially for 918+).

 

 

On 8/7/2021 at 5:00 PM, Aigor said:

I would like boot redpill image via pxe, i have already setup pxe boot env, and i'm trying to add Redpill boot, so i setup a entry like this 

This looks like a broken patch for ramdisk hmm. Is the file even loaded via network?

 

 

On 8/7/2021 at 11:05 PM, cwiggs said:

 

Used the tool chain here to create a 6.2.4 image.  The image seems to boot correctly in proxmox 6.4-13 but the VM never gets an IP, and I can't find it with find.synology.com.  I updated the s/n, nic, usb vid/pid but still no luck.  I am booting the img as sata0 after using a 'qm import'.

(...)

tail of serial1:


Running /usr/syno/etc/rc.d/J03ssdpd.sh...
/usr/bin/minissdpd
Usage: /usr/bin/minissdpd [-d] [-6] [-s socket] [-p pidfile] -i <interface> [-i <interface2>] ...

  <interface> is either an IPv4 address such as 192.168.1.42, or an
interface name such as eth0.

  By default, socket will be open as /var/run/minissdpd.sock
and pid written to file /var/run/minissdpd.pid
start SSDPD failed
Running /usr/syno/etc/rc.d/J04synoagentregisterd.sh...
Starting synoagentregisterd...
Running /usr/syno/etc/rc.d/J30DisableNCQ.sh...
Running /usr/syno/etc/rc.d/J80ADTFanControl.sh...
Running /usr/syno/etc/rc.d/J98nbnsd.sh...
Starting nbnsd...
Running /usr/syno/etc/rc.d/J99avahi.sh...
Starting Avahi mDNS/DNS-SD Daemon
cname_load_conf failed:/var/tmp/nginx/avahi-aliases.conf
:: Loading module hid ... [  OK  ]
:: Loading module usbhid ... [  OK  ]
============ Date ============
Sat Aug  7 15:06:55 UTC 2021
==============================
starting pid 5050, tty '': '/sbin/getty 115200 console'

 

Nothing on serial2

 

Any ideas?

Did you use virtio driver for the ethernet?

 

 

On 8/7/2021 at 11:28 PM, p3t3 said:

I make loader on 6.2.4 - apollolake, test on vmware workstation (nic: e1000e)

While testing, find.synology.com is successful, but dsm install step gets error 13.

Is it works on vmware workstation? Also, loader cannot boot efi. It boots bios only?

For any error 13 a full log from /var/log/messages and install_log is needed. However, in general VMWare will not work until we implement SATA DOM as VMWare Workstation/Fusion/ESXi doesn't allow creating virtual USB drives.

 

To add to that EFI boot is problematic in many way as it tries to initialize devices before Linux can touch them. This can end poorly on some hardware when running DSM. That's why we disabled EFI in the GRUB deliberately.

 

 

On 8/8/2021 at 3:47 AM, haydibe said:

I get the same console output. Though, Syno assistent finds the box and opens the browser page to migrate to the new OS version.

I assume due to missing parameters in the user_config.json the installer tries to write on the boot device and fails.  

 

I took these parameters from the old grub.cfg and added them to the user_config.json.

    "bootdev": "sata", 
    "sata_args": "sata_uid=1 sata_pcislot=5 synoboot_satadom=1 DiskIdxMap=0C SataPortMap=1 SasIdxMap=0"


    "bootdev": "sata", 
    "sata_args": "sata_uid=1 sata_pcislot=5 synoboot_satadom=1 DiskIdxMap=0C SataPortMap=1 SasIdxMap=0"

 

Still no luck. 

You cannot do that. There's no support for SATA DOM in the LKM.

 

On 8/8/2021 at 4:50 AM, cwiggs said:

I've never been able to get DS918 to install in proxmox, have other people?  It boots, but then just seems to stop booting the kernel all the way.  The last thing in serial0 is: (...)

This is a known issue: https://github.com/RedPill-TTG/redpill-lkm/issues/3

 

 

On 8/8/2021 at 4:55 AM, Orphée said:

There is no telnet access with serial port like Jun's loader I guess

Telnet access has nothing to do with the loader. It's up to the settings you set in your hypervisor ;) On Proxmox there's a local access through "qm console", on VirtualBox you can set COM1-4 in settings to be a telnet or a raw socket or whatever else.

 

 

On 8/8/2021 at 5:00 AM, haydibe said:

@cwiggs The variable CROSS_COMPILE is responsible that the Synology tool chain is used. Can you try if the result changes if you execute `unset CROSS_COMPILE`  before `make build_all`

 

 What if using the official Synology tool chain wasn't such a good idea at all.

 

At least for the Apollolake build, these tree warning are there regardless wether gcc from the Synology Toolchain, or gcc4.9.2 from the debian package is used for the build:


...
  CC [M]  /opt/redpill-lkm/config/runtime_config.o
/opt/redpill-lkm/config/runtime_config.c: In function 'validate_runtime_config':
/opt/redpill-lkm/config/runtime_config.c:137:47: warning: passing argument 2 of 'validate_nets' discards 'const' qualifier from pointer target type
     valid &= validate_nets(config->netif_num, config->macs);
                                               ^
/opt/redpill-lkm/config/runtime_config.c:49:20: note: expected 'char (**)[13]' but argument is of type 'char (* const*)[13]'
 static inline bool validate_nets(const unsigned short if_num, mac_address *macs[MAX_NET_IFACES])
                    ^
...
  CC [M]  /opt/redpill-lkm/shim/bios/bios_shims_collection.o
/opt/redpill-lkm/shim/bios/bios_shims_collection.c:11:13: warning: 'shim_null_void' defined but not used [-Wunused-function]
 static void shim_null_void(void) { }
             ^
...
  CC [M]  /opt/redpill-lkm/shim/uart_fixer.o
/opt/redpill-lkm/shim/uart_fixer.c: In function 'swap_uarts':
/opt/redpill-lkm/shim/uart_fixer.c:89:9: warning: unused variable 'out' [-Wunused-variable]
     int out = 0;
         ^
...

 

I am not sure if the ubuntu:18.04 based image had the same warning..   it does, just with some colored parts in the warning (same as with gcc 5.4)

 

Update: I just tried ubuntu:14.04 as base image -> The lkm build shows the same warnings as obove, just the packaging of the laoder fails, because losetup in ubuntu 14.04 doesn't know the `--partscan` option.

 

Update2: tried ubuntu:16.04 as base image -> same warnings as above, compiles with Synology tool chain or without (gcc 5.4, colored output)

 

Update3: tried ubuntu:18.04 as base image -> same warnings as above, compiles with Synology tool chain (colored output, for the kernel module build, non colored for the lkm module build) or without (gcc 7.5.0, colored output everywhere and many warnings) 

additional warning:


  HOSTCC  arch/x86/tools/relocs_64.o
In file included from arch/x86/tools/relocs_64.c:17:0:
arch/x86/tools/relocs.c: In function 'process_64':
arch/x86/tools/relocs.c:949:2: warning: argument 1 null where non-null expected [-Wnonnull]
  qsort(r->offset, r->count, sizeof(r->offset[0]), cmp_relocs);
  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from arch/x86/tools/relocs.h:6:0,
                 from arch/x86/tools/relocs_64.c:1:
/usr/include/stdlib.h:827:13: note: in a call to function 'qsort' declared here
 extern void qsort (void *__base, size_t __nmemb, size_t __size,
             ^~~~~

 

These warning are just really warnings and GCC isn't correct here actually. As 4.8 is old we need some creative way to make it happy. But since this code works as expected and is a weak warning we aren't worried about that.
The one regarding unused function is there because we left some skeleton for future shims - that one used to be used before we implemented proper RTC.

As for rebuilding kernel: you MUST use the proper GCC for that. It will NOT build with GCC 5.4 properly as v3 only supports up to v4.9 (or something along the lines).

 

 

On 8/8/2021 at 5:17 AM, Orphée said:

I confirm you have minimal CPU requirement for DS918+... I'm running DS3615xs because I can't run DS918+ on HP Gen8

image.thumb.png.655127f9a366075ca22f7ea8a6da9db0.png

Unfortunately Proxmox (or rather QEMU) cannot do a split-brain situation where some instructions are virtualized with hardware KVM acceleration and some are done in software. This means it's not feasible to run emulated haswell when your host CPU doesn't support all the instructions. You CAN do that for testing (disable HW acceleration in VM options) but it's PAINFULLY slow. We use it from time-to-time for testing LKM but it's really like working on a Pentium MMX.

 

 

11 hours ago, haydibe said:

Feel free to point out if something in the process of setting up the toolchain is incorrect or can be done better.

Appart from that: I have no idea if something in redpill-lkm, redpill-load or the generated bootloader image does not work - haven't managed a successful migration so far myself .. it always bails out at 56% with error 13 - I use ESXi 6.7

That's a good decision in our opinion to move to debian8. It is old... but so is the kernel. We also use Debian 8 VM internally. From what we've heard syno does not use a cross-toolchain for building kernels either. That says something.

 

As for booting on ESXi see above - it will not boot as of now.

 

 

11 minutes ago, kelvin701 said:

hi guys, anyone try it on UNRAID?

Isn't unraid using qemu? It should boot no problem (at least to grub) as long as it's a TRUE usb boot and not a sata boot. Can you run the VM and check the output from "ps -a"? Go through it manually and try to find the qemu/kvm process. It will have a VERY long cmdline (like several lines long). This is the ultimate source of truth regarding what's being run.

 

16 minutes ago, mcdull said:

me 2, my image always stopped at grub before loading kernel.  Since I am not good at it, probably my mistake in following the procedure.

(proxmox 7.0)

Where precisely does it stop? Can you post a screenshot?

  • Like 10
  • Thanks 5
Link to post
Share on other sites
23 minutes ago, gadreel said:

@kelvin701 I did, and as I posted a few posts back to Haydibe that using his docker image on Unraid it does not boot up. I do not know what is going on. Maybe is something that has to do with GRUB... I have no idea really. Feel free to try though :)

 

Keep in mind that is still pre-alpha version and is mainly for developers. If it does not work we can wait patiently for ThorGroup to finish their amazing work on the bootloader.

You means you try it on Docker or VMS? 

Link to post
Share on other sites
16 minutes ago, ThorGroup said:

Status Update! ;)

Recently we were working hard in the kernel space to bring even more missing things and bring it into a more stable state. That meant we had little time to respond but we're blown away by the number of posts here!


mfgBIOS shimming stability
As we discussed here before there were moments where some BIOS functions weren't shimmed or there were flaky. This caused the OS to sometimes work correctly but sometimes fail the 24h test. Guess how easy was that to debug and fix :D
However, we did it and now the mfgBIOS shimming should be almost bulletproof!


RTC emulation
We've added a full RTC proxy layer. This wasn't strictly needed for 3515xs series. However, 918+ and other models use RTC chip which is not natively/fully supported for the Linux kernel. This means that various subsystems rely on mfgBIOS to update the time from RTC on boot. Since that chip does not exist outside of the official hardware we needed to go around that. In short the LKM code plugs into mfgBIOS and serves as a proxy layer between what's a real RTC (or a one from hypervisor) and what mfgBIOS can understand.

Currently, the time set/get is implemented. Alarms are mocked to the point that mfgBIOS doesn't complain. In fact the BIOS code can even set these alarms but they will be erased upon reboot. We didn't bother actually writing them to the RTC. The reason being a) we've never seen anybody using it and b) most RTCs [or actually the hw connected to them] is broken to the point that they don't work anyway (or only work from certain sleep states).
If the community desires we can add that.


Inverting serial ports on 3615xs
We hate it. We did it. It's sort of working and it's a big hack. We will most likely disable it by default as this isn't even fully stable. Also this method cannot work on kernel v4 which... well, v7 and 3615xs. Whoever in syno had this brilliant idea should seriously think again before pulling such a stunt again. This breaks a lot of stuff.


Fix ttyS0 on 918+
On some platforms first serial port is intentionally disabled. We added a small hack to re-enable it.


Virtual Serial Port / UART
This is big. In many ways actually. This single commit with over 1300 new lines and the patch taking 70KB was by far the biggest addition. The complexity of this part is really something we grind our teeth on. To make sure the solution works on all kernels without hacks we had to emulate a physical UART chip. So we did. We wrote a whole software-based emulator of National Semiconductors 16550A UART controller, along with support for FIFOs. Additionally, as we wanted the code to perform great we raised the bar and wrote an additional layer of virtual IRQ (as the real ones cannot be used for many reasons here) which actually made the module multithreaded in the kernel.

This whole thing took, across all of us, probably close to 150 hours to develop. There were many ideas and failed endeavors. But the full chip emulation should serve us a long way you truly cannot distinguish it from a real thing (as it is not hacking the kernel in any way).

 

If anyone wants to borrow that code for something non-DSM related feel free - it's actually modular and not coupled with the RedPill LKM in any substantial way.

 

Currently, this vUART stuff is not running in the actual LKM. It is meant to be a component to implement PMU emulation. That part requires a two (hopefully) simple functions which react to the data from the DSM and respond like the PMU would. This will also allow us to give a simple hook to anybody who's willing to connect a hardware PMU to get LEDs and what not working ;)


*********************************************************************************

 

v7 testing
We're so happy to see the community effort to add v7. As it is now available for 3615xs we will definitelly jump into it and merge PRs if they pass ;)

AFTER merging them we will probably reorganize template files a bit as that symlink-hell was a bad idea.

 

Now let us respond to everything:

RedPill actually blocks telemetry as well ;) If you want to check that just go the Control Panel and click "What's new" on an update - you will not see a syno page.

 

 

Currently it SHOULD work on some baremetal configs but it's definitely not tested. It will be plug-and-play when it's stable.

 

 

We confirmed that ESXi doesn't allow adding virtual USB drives, so we need SATA DOM boot support. Stay tuned.

Setting vid/pid or any other options will NOT make the loader work without LKM changes (ok, technically MAYBE if you recreate PCI tree in Proxmox but .... no, that's not the solution and definitelly not for ESXi or baremetal ;)).

 

 

You didn't provide RedPill LKMs. You need to compile them if you want to play with the current build.

 

 

You shouldn't try to compile the kernel but the module. Executing make in the kernel dir will try to build the whole kernel (which doesn't make any sense as it will not be usable really). If you're having problems compiling the module the release is not meant for you (yet).

 

 

We weren't able to access the telegram as they force us to give a valid, physical phone number. For obvious reasons we didn't give them one ;) When we have a moment to arrange that we will arrive for sure. However, as the current stage is still intended for developers (and we're BEGGING normal users to not use it due to missing stuff which causes various trips in the DSM itself).

Unless there's some way to use telegram without a phone number which we don't know about?

 

 

DO NOT USE IT. Seriously, we don't want to sound rude but if we say this is a development release we mean it. The kernel module lacks some functionality required for the DSM to stay stable. In the current state data loss is a possibility.

 

 

Thanks! We need some smart way for adding kernel extensions outside of the ramdisk (as it's limited in size). Something like Jun's extra.lzma. However, as it uses a different method of loading we cannot simply merge ramdisks automatically hmm...

 

 

Great job! We love the approach of going docker. We honestly didn't look into the docker toolchain (yet!) you've built but we're excited!

What do you think about incorporating it into RedPill-Load repository? We think it will be a great addition so that the whole thing can be run on any dockerized system.
Drop us a PM to not pollute the thread. In general it's great - the only thing which we will keep is making sure the user can actually use JSON config and not makefile - we pulled teeth for hours to make it happen in bash as JSON is user-friendly while Makefile... well :D

 

 

Onboard network will require a separate kernel extension. This is the same issue as with power button kernel extension - we don't have a good way for not including all extensions in the main ramdisk (which cannot grow too much, especially for 918+).

 

 

This looks like a broken patch for ramdisk hmm. Is the file even loaded via network?

 

 

Did you use virtio driver for the ethernet?

 

 

For any error 13 a full log from /var/log/messages and install_log is needed. However, in general VMWare will not work until we implement SATA DOM as VMWare Workstation/Fusion/ESXi doesn't allow creating virtual USB drives.

 

To add to that EFI boot is problematic in many way as it tries to initialize devices before Linux can touch them. This can end poorly on some hardware when running DSM. That's why we disabled EFI in the GRUB deliberately.

 

 

You cannot do that. There's no support for SATA DOM in the LKM.

 

This is a known issue: https://github.com/RedPill-TTG/redpill-lkm/issues/3

 

 

Telnet access has nothing to do with the loader. It's up to the settings you set in your hypervisor ;) On Proxmox there's a local access through "qm console", on VirtualBox you can set COM1-4 in settings to be a telnet or a raw socket or whatever else.

 

 

These warning are just really warnings and GCC isn't correct here actually. As 4.8 is old we need some creative way to make it happy. But since this code works as expected and is a weak warning we aren't worried about that.
The one regarding unused function is there because we left some skeleton for future shims - that one used to be used before we implemented proper RTC.

As for rebuilding kernel: you MUST use the proper GCC for that. It will NOT build with GCC 5.4 properly as v3 only supports up to v4.9 (or something along the lines).

 

 

Unfortunately Proxmox (or rather QEMU) cannot do a split-brain situation where some instructions are virtualized with hardware KVM acceleration and some are done in software. This means it's not feasible to run emulated haswell when your host CPU doesn't support all the instructions. You CAN do that for testing (disable HW acceleration in VM options) but it's PAINFULLY slow. We use it from time-to-time for testing LKM but it's really like working on a Pentium MMX.

 

 

That's a good decision in our opinion to move to debian8. It is old... but so is the kernel. We also use Debian 8 VM internally. From what we've heard syno does not use a cross-toolchain for building kernels either. That says something.

 

As for booting on ESXi see above - it will not boot as of now.

 

 

Isn't unraid using qemu? It should boot no problem (at least to grub) as long as it's a TRUE usb boot and not a sata boot. Can you run the VM and check the output from "ps -a"? Go through it manually and try to find the qemu/kvm process. It will have a VERY long cmdline (like several lines long). This is the ultimate source of truth regarding what's being run.

 

Where precisely does it stop? Can you post a screenshot?

 

Thanks for replying.  It must be some stupid mistakes that I've made.

Screenshot here:-

 

grub.png

Link to post
Share on other sites
24 minutes ago, kelvin701 said:

You means you try it on Docker or VMS? 

As mention by the ThorGroup the procedure to build the bootloader image is kind of difficult (for us the amateurs), Haydibe provided us a zip file that after running some commands will create a docker image and build the bootloader.img for you automatically. I did that, but when I added the image to Unraid's VM (like the synoboot.img we have on unraid to boot DS918+) it does not boot at all, it goes back to BIOS. Maybe something is wrong with the GRUB or the usb is not bootable...

 

ThorGroup said that since Unraid which indeed uses QEMU also it should work without any issue...but again these builds a for developers and not stable...

Edited by gadreel
Link to post
Share on other sites
57 minutes ago, ThorGroup said:

We weren't able to access the telegram as they force us to give a valid, physical phone number

Already seen that, we should move to discord or some plain old technology, they are already there ;-)

Edited by smoojay
  • Like 2
Link to post
Share on other sites
9 minutes ago, gadreel said:

As mention by the ThorGroup the procedure to build the bootloader image is kind of difficult (for us the amateurs), Haydibe provided us a zip file that after running some commands will create a docker image and build the bootloader.img for you automatically. I did that, but when I added the image to Unraid's VM (like the synoboot.img we have on unraid to boot DS918+) it does not boot at all, it goes back to BIOS. Maybe something is wrong with the GRUB or the usb is not bootable...

 

ThorGroup said that since Unraid which indeed uses QEMU also it should work without any issue...but again these builds a for developers and not stable...

I'm using synoboot.img on unraid to boot DS3617xs+, DS918+ not work for me cause my machine is too old (i7-3770 Ivy Bridge)

Link to post
Share on other sites
19 minutes ago, kelvin701 said:

I'm using synoboot.img on unraid to boot DS3617xs+, DS918+ not work for me cause my machine is too old (i7-3770 Ivy Bridge)

you modified it to support 3617?

I read redpill-lkm only supports 3615 and 918.

Link to post
Share on other sites

 

13 hours ago, haydibe said:

The redpill tool chain docker image builder is updated to v0.4:

- proper DSM7 support for apollolake (thnx @jumkey)

- switched from kernel sources based build to toolkit dev based builds for DSM6.2.4 and DSM7.0 (thnx @jumkey

- make targets for bromolow and apollolake merged: platform and version must be configure in the Makefile now

- image is ~720-780MB instead of ~1.200-1.500MB now

 

Note: since toolkit dev lacks the required sources for fs/proc, they are taken from the extracted DSM6.2.4 kernel sources.

          The build requires the sources for this single folder, but does not use the kernel source to build the redpill.ko module. 

 

If you see something is wrong in how the toolchain is build or have ideas how to make it better: please let me know.

For every other problem: please address it to the community - I don't know more than others do. 

 

Note#2: before someone asks: I haven't managed a successfull installation/migration with the created bootloader so far. I am testing exclusivly on ESXi 6.7. The migration always stops at 56% and bails out with error 13.

 

Usage:

- (on host) configure the Makefile and configure TARGET_PLATFORM (default: bromolow) and TARGET_VERSION (default: 6.2 - will build 6.2.4)

- (on host) create your own user_config.json or edit the USERCONFIG_* variables in the Makefile

- (on host) build image: make build_image

- (on host) run container: make  run_container

- (in container) build redpill.ko module and redpill bootloader image: make build_all

 

After running `make build_all` the created redpill bootloader image will be present in the ./image folder on the host.

 

Tested with hosts: Ubuntu 18.04 VM, Ubuntu 20.04 WSL2 and XPE  (the make binary to build on Synology/XPE can be found here)

 

Dependencies: make and docker

redpill-tool-chain_x86_64_v0.4.zip 5.01 kB · 29 downloads

 

 

@haydibe Great Idea, works as expected. Maybe its a good idea to add the instructions on a Readme file inside the zip file

Edited by pocopico
Link to post
Share on other sites

Finally got DSM to boot with haydibe's v0.4 script.

under proxmox v7 and AMD Ryzen 3900X, I have to use qemu64 as cpu type or it will not boot.

 

Update: 

[  141.005313] EXT4-fs (md0): couldn't mount as ext3 due to feature incompatibilities
[  141.008687] EXT4-fs (md0): mounted filesystem with ordered data mode. Opts: (null)
[  141.032935] Failed to set alarm data.
[  141.037685] EXT4-fs (md0): couldn't mount as ext3 due to feature incompatibilities
[  141.041520] EXT4-fs (md0): mounted filesystem with ordered data mode. Opts: (null)
[  141.056471] EXT4-fs (md0): couldn't mount as ext3 due to feature incompatibilities
[  141.059888] EXT4-fs (md0): mounted filesystem with ordered data mode. Opts: (null)

 

Failed with installation.  dunno if it is cpu incompatibility. (AMD Ryzen 3900x)

 

Update2 :

Seems the above error is not fatal, I stopped at 55%. And in the next trial, DSM7 detected my hdd has old configuration, which was a brand new one created under pve 7.0

 

Update3 :

Added 2 SATA hdd, stopped at 55% with error 21

Aug  9 14:12:53 updater: updater.c:7029 ==== Start flash update ====
Aug  9 14:12:53 updater: updater.c:7033 This is X86 platform
Aug  9 14:12:53 updater: boot/boot_lock.c(228): failed to mount boot device /dev/synoboot2 /tmp/bootmnt (errno:2)
Aug  9 14:12:53 updater: updater.c:6494 Failed to mount boot partition
Aug  9 14:12:53 updater: updater.c:3118 No need to reset reason for v.41890 
Aug  9 14:12:53 updater: updater.c:7636 Failed to accomplish the update! (errno = 21)
Aug  9 14:12:53 install.cgi: ninstaller.c:1546 Executing [/tmpData/upd@te/updater -v /tmpData > /dev/null 2>&1] error[21]
Aug  9 14:13:07 install.cgi: ninstaller.c:123 Mount partion /dev/md0 /tmpRoot
Aug  9 14:13:07 install.cgi: ninstaller.c:1515 Moving updater for configuration upgrade...cmd=[/bin/mv -f /tmpData/upd@te/updater /tmpRoot/.updater > /dev/null 2>&1]
Aug  9 14:13:07 install.cgi: ninstaller.c:152 umount partition /tmpRoot
Aug  9 14:13:07 install.cgi: ErrFHOSTCleanPatchDirFile: After updating /tmpData/upd@te...cmd=[/bin/rm -rf /tmpData/upd@te > /dev/null 2>&1]
Aug  9 14:13:07 install.cgi: ErrFHOSTCleanPatchDirFile: Remove /tmpData/upd@te.pat...
Aug  9 14:13:07 install.cgi: ErrFHOSTDoUpgrade(1794): child process failed, retv=-21
Aug  9 14:13:07 install.cgi: ninstaller.c:1811(ErrFHOSTDoUpgrade) err=[-1]
Aug  9 14:13:07 install.cgi: ninstaller.c:1815(ErrFHOSTDoUpgrade) retv=[-21]
Aug  9 14:13:07 install.cgi: install.c:409 Upgrade by the manual patch fail.
Aug  9 14:13:07 install.cgi: install.c:678 Upgrade by the uploaded patch /tmpData/@autoupdate/upload.pat fail.
Jan  1 00:00:00 install.cgi: ninstaller.c:152 umount partition /tmpData

This error seems familiar: failed to mount boot device /dev/synoboot2 /tmp/bootmnt (errno:2)

Edited by mcdull
Link to post
Share on other sites
4 hours ago, ThorGroup said:

RedPill actually blocks telemetry as well ;)

 

This is awesome, although I am wondering, does it block all synology domains? For example, I created a patch that allows push notifications to work, so does RedPill-LKM block anything going out to *.synology.com, etc. or just the domains known to do telemetry, like Active Insight? (https://kb.synology.com/en-my/DSM/tutorial/What_websites_does_Synology_NAS_connect_to_when_running_services_or_updating_software).

 

Also, how is this blocking done? Host file or something?

Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.