Leaderboard

Update time! Support for v4.4 kernel TL;DR: it now works with v4.4 After some of you took a stab at compiling the LKM for Linux v4.4 (presumably for DS918+ support) we did too. We didn't see any PRs besides a small patch from @jumkey (thanks!). However, the reality came to be much MUCH more complex, especially if we want to keep the consistent codebase for all kernels. So, it took 400 lines over 12 commits by 3 people: https://github.com/RedPill-TTG/redpill-lkm/compare/7f85cb2ceff44d6914135bf3841eb3cb2148af11...1abcbe13df39055dc8b925f74a00a75163157d60 However, the story is more complex here. The kernel development velocity is huge, to add to that DSM's kernel has its own peculiarities (more on that below ;)) too. The following things took us the time: Adjusting to build with a new source tree (very easy) Running the actual kernel in an isolated VM (quite challenging). This posed a problem because the first round of patches for v4.4 one of us made weren't stable. Currently, the ramdisk patch works (oh irony, the one generated by the automatic tool we hacked together) but the "boot_params" causes random KPs. Additionally, some of our HVs don't have CPUs with hardware MOVBE support, or they crash/freeze the machine after snapshot restore. This made debugging challenging as you don't know if it's your CPU, hypervisor, configuration, kernel patch, LKM, or a combination of multiple ones causing the problem. Another challenge here is a bug in Synology's v4.4 for apollolake (possibly others too). In essence, they modified ramdisk unpacking routine and broke it in the process. The effect of it is no matter what tools we used the ramdisk wouldn't unpack making the system unbootable. We have no idea how they pack their ramdisks but we failed to recreate that. This took us finally to a flat CPIO archive (more in docs: https://github.com/RedPill-TTG/dsm-research/blob/master/quirks/ramdisk-checksum.md#recreating-ramdisks) Getting cmd sanitization working on v4 (easy): they've changed the format of /proc linklists, so we had to adjust. However, we saw that @jumkey posted a possibly better (muuuch shorter) solution to the problem. We didn't try it yet, but by all means, please make a PR! It should actually work for both v3 & v4 lines. Chasing an annoying bug with preempts: it seems like v3 silently fixes these but v4 traps them and crashes. Refactoring how the re-export of functions work to make it much easier for others to contribute to (and make our life more pleasant). This was part of the fix for set_memory_rw/ro not being officially available anymore. Now you can just use 2 macros and call it a day in 30 seconds. Making vPCI stuff working on v4 (difficulty: 11/10) : this was probably the hardest thing to debug and the easiest things to fix (it took a bloody SINGLE line!). There are many levels of hell in PCI. In around v4.1 they've added a couple more. Now all PCI stuff assumes ACPI compliance and static DSDTs on x86. In the same time they broke dynamic bus scanning for new busses which aren't PC-BIOS based. We're assuming it was accidental as what they did was a refactor only. Here comes the syno kernel too: in v3 they have dynamic debugging enabled (but not the full kdebug). This gives at least a possibility of seeing some more messages On v4 the debugging cycle is: stub a method with some debug prints, override the method in kernel using override_symbol.c, compile the module, reboot/restore the test VM, download the module to a test VM, load the module and see what you wanted. Repeat 500x times Fixing execve() shimming wasn't as hard as vPCI stuff but quite challenging as well. Since v3.18 the call structures of ASM shims changed. In essence they use different function call convention than GCC generates (there's a good article about that:L https://my.oschina.net/macwe/blog/603583). This cannot be achieved (correct us if we're wrong) using C but requires custom ASM. Once we realized what's the problem we decided to make route the call differently by skipping the shim altogether. This is less clean and may be a tad bit slower (nothing noticeable thou). What is an elevator? For long time we were wondering why Jun's loader sets "elevator=elevator" parameter to the kernel. Normally it sets the I/O scheduling, but obviously "elevator" is not a valid value. We think we accidentally found what this is for. Linux automatically loads a custom elevator module with the name specified (with some pattern, we didn't dig into that deeply yet) before ANY other modules are loaded. And viola, that's how the LKM can load as early as possible It will be included in RedPill as well as this is a cool trick and ensures the module loads earlier than with init script. Anybody want the Synobios source code? It's public. It has been for years in fact. The struct posted by Vortex lead us to explore older dumps. While lurking through them we found that Syno (accidentally?) used to publish full Synobios source code. They fixed their mistake but didn't remove these old files. You can find it as a normal kernel driver in all GPL archives till 5004 (e.g. synogpl-3776-bromolow/source/linux-2.6.32/drivers/synobios). After talking with @Vortex we learned that the full headers (but not the code this time) for synobios are present in DSMv7 dev env (more details: https://github.com/RedPill-TTG/dsm-research/blob/master/quirks/mfgbios.md#appendix-official-synobios_ops-structure) Synobios ioctl emulation We found out what the previous author meant in the PMU document. It seems like there are two part of the synobios: shimming the synobios_ops (done) responding to proxy communication It turns out that synobios not only performs its own calls but other system components (e.g. secmd) can use ioctl to ask it to send some arbitrary commands to the PMU. These commands are actually what causes these logged messages which @Vortex explained as beeps etc. We need to respond to them in software (i.e. in LKM). Making the GPL sources more accessible Let us preface this by saying this isn't anything we're prioritizing. However, we're putting this onto the radar. While browsing through the code we realized two things: it's annoying hard to find what you're looking for and/or compare, as they publish a sea of large archives they remove things Because of these we made, currently by hand, a rundown of architectures and formats over the versions. We are also planning to replicate the whole SF archive in a form of GH repos. Full details: https://github.com/RedPill-TTG/dsm-research/blob/master/quirks/gpl.md DSM v7 sources Synology did already publish v7 sources and.... they pulled them quickly: https://sourceforge.net/p/dsgpl/discussion/862835/thread/a519b80124/. They used to be in the "41890branch" as you can see from the project's history: https://sourceforge.net/p/dsgpl/activity/?page=0&limit=100 We weren't lucky enough to get them. Did anyone maybe was able to grab them? ----------------------------------------- Quite doable. Code-wise we need a similar interface for ioctl emulation (see above). Hardware-wise... well, *I* personally don't see the point as I virtualize what I can (and some of my colleagues disagree with me) but it's an easy addition. Currently, we're trying to get the software side of all the things to a non-developer-usable state. However once we do the ioctl interface patches for adding support for submitting events to some external source are welcomed. Then we can start another topic discussing making a small module for that. The cheapest AVR/arduino/PIC will do here. I know you meant to control the LEDs but in case you're not aware: the actual power off via ACPI just lacks a kernel module and doesn't need any custom stuff. Correct us if we're wrong but the architecture here is such that the kernel uses ASCII-based interface over UART to a bridge. Then the bridge/PMU has I2C for controlling other chips for like LEDs, fans etc. So strictly speaking we don't need I2C for anything. As I see the forum doesn't have any Telegram channel nor IRC. We believe the latter is actually better in many ways (we're old I think ;)) but Telegram is more modern. We will join for sure. The DSM consists of a heavily modified kernel, modified userland applications, and a completely custom web UI to manage all that. Since they have to should abide by the GPL they agreed to when modifying the kernel & libs they do should publish the full source with modifications for GPL-licensed part. In practice this means we get kernel, some libs, and some userland apps code. The whole DSM web management is their own custom stuff. So, here's actually the issue. The source code for the kernel is readable-ish but it cannot be compiled. It used to be but it's not anymore. The biggest problem is they replaced a lot of constants with "MY_ABC_HERE". In our view this is a violation of the GPL but good luck pressuring them to do anything about that. It's probably possible to compile the kernel if someone goes through and manually decides which "MY_ABC_HERE" to turn on and off but even we aren't crazy enough to do so. What the kernel sources are good for, besides reading, is compiling modules against them. You cannot use vanilla v4.4 or v3.10 sources for this. DSM kernels are heavily modified (so e.g. structs have new fields added to them, sometimes in the middle). Additionally, their v3.10 is not a v3.10 mainline + their patches but v3.10+DSM patches+a ton of backported changes from newer kernels. Supporting many devices is difficult because of the standard matrix of permutations problem, as in any software project, AND on top of this because Synology releases the same but different "v3.10" for different platform. For example apollolake v4.4 has broken ramdisk support We don't compile the kernel. We use DSMs kernel + binary patches from the dsm-research repo patches. You can load the patched image using kexec or repack it to boot from grub. You CANNOT use Jun's loader for this. Currently, we only published sources for the LKM. This is not because we have some secret one but because the kernel modification process is MUCH easier to do. To give you a glimpse of how we do the development we will just say that it's a manual process. We deliberately did not write a step-by-step instruction because we don't want regular folks making YT videos how to hack this around. It is NOT production safe yet. If you want to play with it as a developer simply compile GRUB (or even just copy one from any Linux distro from /boot). Then get the kernel + rd.gz from 25556 PAT file. Next apply binary patches from dsm-research repo and put redpill ko in the rd.gz. Next modify the init script to include redpill loading at the very top. We are looking here and discussing what you guys bring but we publish updates in batches as we don't want to rush with conclusions or publish FUD. It takes some times as we have full time jobs, families, kids, and socialize sometimes too (despite the rumors developers do sometimes leave their caves ;)). PCI addresses are very easy with RedPill. We can most likely prepare a patch in 5 minutes if someone posts "lspci -tvv" from a real DS/RS (ok, not just right now because we're currently trying to come up with a since runtime-level model-dependent configuration switching. The other things like testing and potentially different hardware requirements are the issue. Once we get a stable-stable version we can play with other ones. Another issue are S/Ns. They need to validate. We saw the validation somewhere in the kernel but the community developed serial generator only supports a handful of models. If someone wants to take a peek / knows how the previous serial generator was developed we're open. However, we think @flyride nailed it. The number of disks is not limited and other features are already present in DS3615/7xs & DS918+. DS1621+ may be worth looking into due to the AMD support.