Drivers extension for RedPill - request for comments & drivers packages

[ThorGroup]

Hello!

TL;DR: We've developed a new loader for v6.2.4/v7+ which contains a way to install custom extensions/drivers/mods. We're looking for feedback of the extension manager & kindly asking for drivers to be made compatible with it.

 

First of all, let us thank you guys for all the driver packages you were always preparing - we used them ourselves. As some of you may be aware we are developing a new loader called RedPill. There's a long thread in the developer's section of the forum:

 

Today we've added a long-awaited functionality of custom drivers. We've chosen to go with a more modular approach rather than offering a clone of Jun's loader functionality of extra.lzma archive. Our solution relies on a simple extension manager which is responsible for downloading and packing drivers automatically. This lets users install only the drivers they need while also mixing-and-matching multiple ones. With "extra.lzma" the limitation was that users either had to build their own packages or choose between having the HBA driver or VirtIO driver... that, in our opinion, was a poor experience. Additionally, the new format allows you to keep a single git repository containing both the metadata for RedPill extension manager and the build environment you use to compile drivers.

 

We would like to collect your opinion on the new extension manager we created and ask you to port your drivers to the new format. We prepared two documents:

• https://github.com/RedPill-TTG/redpill-load/blob/master/docs/extensions-overview.md (summary of what the extension manager is capable of and how to use it manually)
• https://github.com/RedPill-TTG/redpill-load/blob/master/docs/extensions-for-devs.md (document intended for people creating extensions/compiling drivers detailing the architecture)

 

In addition, we published two packages, which can serve as an example:

• https://github.com/RedPill-TTG/redpill-virtio (VirtIO driver)
• https://github.com/RedPill-TTG/redpill-boot-wait (shell-only package; this will be similar to how e.g. a CPU governor mod can be implemented)

There's currently no central way to discover packages but is it planned. For now we're simply collecting links to indexes in a separate git repo - https://github.com/RedPill-TTG/redpill-extensions - you can just toss a link there from the GitHub web UI without cloning the repo.

 

 

What do you all think?

special cc @IG-88

 

p.s. We didn't post this in the driver's extensions subforum as this topic isn't intended for users. When some drivers are built and the official beta version of RedPill is published we will create a separate thread there.

[pocopico]

Are we looking for a whole batch of modules or a by module/driver lets say compilation and packaging ?

[ThorGroup]

We think it should be 1 driver = 1 extension. This way you install driver for your ethernet, driver for your GPU, and a driver for your USB 3 chipset. This will also let us do another cool thing down the road: provide a small image which will boot on your hardware, scan it (essentially grab vids & pids), and suggest a platform and drivers needed to make it work. VID & PID info are embedded in the drivers on linux anyway so it's easy to extract. If you pack many drivers in one package you can easily run into conflicts and load way too much. Jun's loader loaded a ton... literally all modules. In a system with just extra.lzma to get virtio we see 178 kernel modules - mptsas, mpt2sas, mp3sas, vmxnet, megaraid... and many many more which have nothing to do with the platform.

Of course, naturally some drivers will have multiple kernel extensions as it makes no sense to separate them. For example VirtIO is technically like 5 drivers but we never saw anyone using just VirtIO-SCSI but not load VirtIO-NET. With some HBAs it may also make sense to make a bundle containing like 3 drivers for LSI cards. But definitely bundling e.g. VirtIO and LSI woudln't make sense. It's a delicate balance

 

Same for CPU governor: for example we don't need it at all as we either want performance on a bare metal server-server or run it under a VM. This is why we didn't want to bundle it as many folks will not use it. To be honest the concept of power management on Linux is slightly foreign for all of us as we either lazily use Dells supported by Ubuntu or run Linux on servers which don't really care about PM.

Edited September 28, 2021 by ThorGroup

[Piteball]

Just for the curiosity of it, I tried making a driver extension for LSI SAS cards adding pocopico compiled mpt2sas and mpt3sas drivers as the seem to work with the correct dependencies. However I haven't got time to test it, so I just toss it in here for anyone to either copy or do with whatever pleases you.

 

Repo

https://gitea.wickenberg.nu/Piteball/redpill-mptxsas

 

Extension JSON index file

https://gitea.wickenberg.nu/Piteball/redpill-mptxsas/raw/branch/master/rpext-index.json

[pocopico]

Nice ! You are the first one !!! I think they need some modifications. 

 

I'm planning to release some more drivers but I wanted to have some time to compile them for 4.4.59 (918+ 6.2.4) and 3.10.105 (918+ 6.2.4).

 

 

[Amoureux]

15 минут назад, pocopico сказал:

Nice ! You are the first one !!! I think they need some modifications. 

 

I'm planning to release some more drivers but I wanted to have some time to compile them for 4.4.59 (918+ 6.2.4) and 3.10.105 (918+ 6.2.4).

 

 

Extract drivers from jun's extra.lzma and use them?

Edited September 30, 2021 by Amoureux

[pocopico]

1 minute ago, Amoureux said:

Export drivers from jun's extra.lzma it's possible?

 

No these are for previous versions <= 6.2.3

Edited September 30, 2021 by pocopico

[ThorGroup]

15 hours ago, Amoureux said:

Extract drivers from jun's extra.lzma and use them?

Even if you forcefully insert them that's a bad idea - you can yolo things in user space but not in the kernel as userspace is guarded by the kernel and kernel is only guarded by a sparse memory managemnt unit checks The kernel space has literally no memory protection per se (ok, there are small bits here and there but this is mostly bug-catching and not protection between drivers due to performance reasons). It's a PITA that kernel modules in Linux are specific to that exact version up to the t but that's a small price to pay for a great kernel. That decision was famously made in 1991 (if someone's interested google for Linux vs Minix and now-legendary exchange between Linus T and Minix author - it could be a hollywood movie).

 

This is why RedPill extensions system is VERY specific to the versions of modules so nobody accidentally even try loading wrong driver's version as this is much more dangerous on Linux than on Windows or macOS. Drivers on Linux are literally running on the same privilege level as the kernel itself. On macOS more and more are in userspace or are at least partially isolated. This is why nobody should experiment with e.g. SAS drivers on a drive with data which they care about [unless tested by many and confirmed working properly].

Edited October 1, 2021 by ThorGroup

[pocopico]

@ThorGroup Some text files have been created on the extensions repository. I only have a question, how are the modules loaded ? How do you take care of the dependencies ? Should i also create a script for loading the modules ?

[haydibe]

My current understanding is that an extension bundles one or more main driver and their dependencies. The "kmods" item in a recepie decleare the load order of the modules for this extension during boot. 

 

If the dependencies are not supposed to be included in an extension, is there a mechanism in place to defince dependencies to other extensions? I assume this would require a dependency graph/tree detection, which I assume is highly messy or even imposible to archive with plain bash. 

Edited October 2, 2021 by haydibe

[ThorGroup]

On 10/1/2021 at 9:15 PM, pocopico said:

@ThorGroup Some text files have been created on the extensions repository. I only have a question, how are the modules loaded ? How do you take care of the dependencies ? Should i also create a script for loading the modules ?

There's a script which takes care of that. Its template is present in redpill-load repo in include/loader-ext/target_exec.sh_ - it is filled-in with by the extension manager.

From the perspective of adding new drivers you should just add entries under kmods key like described in the documentation for devs and the target_exec script will take care of the rest (i.e. insmodding them etc).

 

There are no interextension dependencies (i.e. between multiple extensions). However, when you specify multiple .ko to load in kmods key they're loaded in that order. There's a mechanism to force extensions load order (extensions key in user_conf as described in dev docs for extensions) as a stop-gap. This should be rare however as drivers loading is for the most part asynchronous in linux anyway.

 

 

On 10/2/2021 at 6:48 PM, haydibe said:

My current understanding is that an extension bundles one or more main driver and their dependencies. The "kmods" item in a recepie decleare the load order of the modules for this extension during boot. 

 

If the dependencies are not supposed to be included in an extension, is there a mechanism in place to defince dependencies to other extensions? I assume this would require a dependency graph/tree detection, which I assume is highly messy or even imposible to archive with plain bash. 

Spot on: dependency graph is needed for that. It would TECHNICALLY be possible in bash but oh boy... that's insane in a very wrong way Being serious we're planning and preparing for a rewrite of extension manager so that it can support dependencies properly. The current solution is a good-enough PoC to see if the idea will work. We designed it and brainstormed through multiple people to give the community a PoC which is functional (this is probably close to a 10th iteration of the design of extensions itself) but maybe not with all bells and whistles right away.

 

From the perspective of time choosing bash on principle of "it doesn't require installation" was a mistake.

[WiteWulf]

Question: at what point during the boot process are extension scripts for “enhancements” run, and is there any control over it from user perspective?

 

While trying to diagnose the kernel panic problems I’ve been having with bromolow on baremetal I’ve found that booting with nmi watchdog disabled, but manually enabling it after boot, seems to fix it.

 

I figured I would keep nmi_watchdog=0 in grub.cfg, then use an extension/enhancement script to turn it back on again. Does that sound like an appropriate use of this functionality?

[haydibe]

@WiteWulf: If this can be done like this -> charming idea 😀

[pocopico]

@ThorGroup 

 

I have noticed some issues with the extensions.

 

1. I create for instance an extension for IXGBE driver and the module has some dependencies, e.g. vxlan. vxlan module has other dependencies for instance udp_tunnel.ko and ip6_udp_tunnel.ko. 

 

A. These modules are included and could be satisfied if the loading process started later.
B. Even If a single extension fails to load it exits with exit code 99 and fails into an install loop.

 

2. If i create extensions whose dependencies collide with other extensions dependencies and are loaded in the same system the load will fail.

 

A. The extension loading script should have some kind of mechanism to identify deps and if they are already loaded it should skip loading. 
B. It should not exit with an error if an extension fails to load as it ends in an install loop.

 

[pocopico]

@ThorGroup Sorry cannot edit my precious post ,

 

3. Some devices need to have firmwares under /usr/lib/firmware but this is not taken into consideration. So some modules might not function properly. e.g a dual port ethernet might operate only as single port

[volitation]

@pocopico 

Because the LGA1155 motherboard uses RealTek RTL-8111E chip, I would like to ask how I need to compile the driver.

[pocopico]

@volitationYou can find the modules and extensions for 8168 extension here :

 

https://github.com/pocopico/rp-ext/blob/main/r8168/rpext-index.json

[volitation]

@pocopico

Thank you for your reply.

I have integrated the rtl8168 driver you compiled, but I still can’t find the IP. Is it something I haven’t noticed?

[volitation]

Have tried to identify the actual model of the network card in proxmox ve is rtl8139

[pocopico]

18 hours ago, volitation said:

Have tried to identify the actual model of the network card in proxmox ve is rtl8139

You can try : 

https://github.com/pocopico/rp-ext/blob/main/8139cp/rpext-index.json

or 

https://github.com/pocopico/rp-ext/blob/main/8139too/rpext-index.json

[serrgey255]

Hello! Could you put together a package for realtek 8111?

Edited November 21, 2021 by serrgey255

[volitation]

@pocopico Thank you for your help. Compile the RTL8139 series driver

 

I would like to ask whether the RTL8169 and RTL8169SC drivers are shared

Edited November 22, 2021 by volitation

[pocopico]

6 hours ago, volitation said:

@pocopico Thank you for your help. Compile the RTL8139 series driver

 

I would like to ask whether the RTL8169 and RTL8169SC drivers are shared

Hi there,

 

you can try adding :

 

https://github.com/pocopico/rp-ext/blob/main/r8169/rpext-index.json

 

 

[serrgey255]

Hello! Can you build the driver for the pci-e sata controller Marvell 88SE9230?

[D.S]

@pocopico , just tested the atlantic driver for 918 Plus 7.0.1, it will fail to load due to file crc-itu-t.ko issue, could you update the extension? I attached the files which is works.

atlantic.zip