RedPill - the new loader for 6.2.4 - Discussion


Recommended Posts

Il y a 21 heures, pocopico a dit :


what is the last message you see in serial ? Try adding the latest tg3.ko 

 

https://raw.githubusercontent.com/pocopico/rp-ext/master/tg3/rpext-index.json

 

Just tried with it, fresh build completely with jumkey develop branch, but same result.

I see the server while 30 seconds, and it restart.

I think I have to forget DSM 7 on my machine 😢

Link to post
Share on other sites
Il y a 7 heures, WiteWulf a dit :

 

Can you confirm the above? Your system no longer kernel panics under load when using the tg3 drivers? If so this is very important.

 

Could you please try running the docker influx-test on your system as described by TTG here? This test is guaranteed to crash my Gen8 100% of the time with NMI watchdog enabled.  If it turns out to be an interaction with the PCIe bus or the NC360T card and drivers I'm using this could help solve the problem.

I'm running the influx-test container wait and see.

 

msedge_f8eOKhKFja.png

msedge_M7sqj4tf10.png

 

20 minutes after :

msedge_rrUZmL2t0c.thumb.png.c0a26a4c558c16c50e3dc040b24d473d.png

Edited by Kouill
  • Thanks 1
Link to post
Share on other sites
il y a 7 minutes, WiteWulf a dit :

 

Amazing! I'm going to build a 7.0.1 bromolow image with tg3 now and see if I can replicate.

 

Nice I can stop the container now ;)

msedge_oFFOxyKBau.thumb.png.e0d76ab79014858ccfbafec0bb3de9bc.png

 

My (ugly) edits :

 

Makefile :

 

    @if [ "$(TARGET_PLATFORM)" == "bromolow" ]; then \

        pushd $(REDPILL_LOAD_SRC) && \

        ./ext-manager.sh add 'https://raw.githubusercontent.com/pocopico/rp-ext/master/tg3/rpext-index.json'; \

        ./build-loader.sh 'DS3615xs' '$(TARGET_VERSION)-$(TARGET_REVISION)'; \

 

global_config.json :

 

        {

            "id": "bromolow-7.0.1-42218",

            "platform_version": "bromolow-7.0.1-42218",

            "user_config_json": "bromolow_user_config.json",

            "docker_base_image": "debian:8-slim",

            "compile_with": "toolkit_dev",

            "redpill_lkm_make_target": "dev-v7",

            "downloads": {

                "kernel": {

                    "url": "https://sourceforge.net/projects/dsgpl/files/Synology%20NAS%20GPL%20Source/25426branch/bromolow-source/linux-3.10.x.txz/download",

                    "sha256": "18aecead760526d652a731121d5b8eae5d6e45087efede0da057413af0b489ed"

                },

                "toolkit_dev": {

                    "url": "https://sourceforge.net/projects/dsgpl/files/toolkit/DSM7.0/ds.bromolow-7.0.dev.txz/download",

                    "sha256": "a5fbc3019ae8787988c2e64191549bfc665a5a9a4cdddb5ee44c10a48ff96cdd"

                }

            },

            "redpill_lkm": {

                "source_url": "https://github.com/RedPill-TTG/redpill-lkm.git",

                "branch": "master"

            },

            "redpill_load": {

                "source_url": "https://github.com/420Xnu/redpill-load.git",

                "branch": "7.0.1"

            }

 

Dockerfile :

 

ARG REDPILL_LKM_REPO=https://github.com/RedPill-TTG/redpill-lkm.git

ARG REDPILL_LKM_BRANCH=master

ARG REDPILL_LKM_SRC=/opt/redpill-lkm

 

ARG REDPILL_LOAD_REPO=https://github.com/420Xnu/redpill-load.git

ARG REDPILL_LOAD_BRANCH=7.0.1

ARG REDPILL_LOAD_SRC=/opt/redpill-load

Edited by Kouill
  • Like 5
Link to post
Share on other sites
40 minutes ago, Kouill said:

ARG REDPILL_LKM_REPO=https://github.com/RedPill-TTG/redpill-lkm.git

ARG REDPILL_LKM_BRANCH=master

ARG REDPILL_LKM_SRC=/opt/redpill-lkm

 

ARG REDPILL_LOAD_REPO=https://github.com/420Xnu/redpill-load.git

ARG REDPILL_LOAD_BRANCH=7.0.1

ARG REDPILL_LOAD_SRC=/opt/redpill-load

No need to touch those. The values are just "fallback values" if they are not provided by build-args, which they always are by the values from global_config.json.

Link to post
Share on other sites
2 hours ago, WiteWulf said:

 

Amazing! I'm going to build a 7.0.1 bromolow image with tg3 now and see if I can replicate.

 

Okay, mixed results for me, I'm sad to say. @ThorGroup may find the results useful, however.

 

I built a new bromolow 7.0.1-42218 image using redpill-helper-v.0.12 and jumkey's "develop" branch of redpoll-load, with pocopico's tg3 extension.

 

I disabled the PCIe NC360T NIC and re-enabled the internal NIC from the BIOS and booted from the new USB stick.

 

Initially the server looked good: all docker containers started, as did Plex Media Server, with no kernel panic.

 

I ran a metadata update on a large library in Plex with no crashes and updated my influxdb docker container, also without a crash.

 

Next I ran the test TTG had previously asked me to do over on GitHub:

docker run --name influx-test -d -p 8086:8086 -v $PWD:/var/lib/influxdb influxdb:1.8
docker exec -it influx-test sh

# inside of the container:
wget https://golang.org/dl/go1.17.1.linux-amd64.tar.gz &&
    tar -C /usr/local -xzf go1.17.1.linux-amd64.tar.gz &&
    rm go1* &&
    export PATH=$PATH:/usr/local/go/bin &&
    go get -v 'github.com/influxdata/influx-stress/cmd/...'
/root/go/bin/influx-stress insert -f --stats

 

This time I got a kernel panic:

[  518.193477] Kernel panic - not syncing: Watchdog detected hard LOCKUP on cpu 6
[  518.228126] CPU: 6 PID: 28191 Comm: influx-stress Tainted: PF          O 3.10.108 #42218
[  518.267191] Hardware name: HP ProLiant MicroServer Gen8, BIOS J06 04/04/2019
[  518.301214]  ffffffff814a2759 ffffffff814a16b1 0000000000000010 ffff880409b88d60
[  518.337704]  ffff880409b88cf8 0000000000000000 0000000000000006 0000000000000001
[  518.374045]  0000000000000006 ffffffff80000001 0000000000000030 ffff8803f4c4bc00
[  518.410480] Call Trace:
[  518.422504]  <NMI>  [<ffffffff814a2759>] ? dump_stack+0xc/0x15
[  518.451063]  [<ffffffff814a16b1>] ? panic+0xbb/0x1df
[  518.475140]  [<ffffffff810a9eb8>] ? watchdog_overflow_callback+0xa8/0xb0
[  518.508166]  [<ffffffff810db7d3>] ? __perf_event_overflow+0x93/0x230
[  518.539141]  [<ffffffff810da612>] ? perf_event_update_userpage+0x12/0xf0
[  518.571601]  [<ffffffff810152a4>] ? intel_pmu_handle_irq+0x1b4/0x340
[  518.603124]  [<ffffffff814a9d06>] ? perf_event_nmi_handler+0x26/0x40
[  518.634926]  [<ffffffff814a944e>] ? do_nmi+0xfe/0x440
[  518.659908]  [<ffffffff814a8a53>] ? end_repeat_nmi+0x1e/0x7e
[  518.688056]  <<EOE>>
[  518.698239] Rebooting in 3 seconds..

 

 

So this is a *huge* improvement on where I was before, but it still crashes if I really push it, and I'm not sure why @Kouill's server *didn't* crash running the same test 🤔

 

One thing to note is that the NC360T card is still physically installed in my machine, but disabled in the BIOS.

  • Like 1
Link to post
Share on other sites
2 hours ago, WiteWulf said:

 

Okay, mixed results for me, I'm sad to say. @ThorGroup may find the results useful, however.

 

I built a new bromolow 7.0.1-42218 image using redpill-helper-v.0.12 and jumkey's "develop" branch of redpoll-load, with pocopico's tg3 extension.

 

I disabled the PCIe NC360T NIC and re-enabled the internal NIC from the BIOS and booted from the new USB stick.

 

Initially the server looked good: all docker containers started, as did Plex Media Server, with no kernel panic.

 

I ran a metadata update on a large library in Plex with no crashes and updated my influxdb docker container, also without a crash.

 

Next I ran the test TTG had previously asked me to do over on GitHub:


docker run --name influx-test -d -p 8086:8086 -v $PWD:/var/lib/influxdb influxdb:1.8
docker exec -it influx-test sh

# inside of the container:
wget https://golang.org/dl/go1.17.1.linux-amd64.tar.gz &&
    tar -C /usr/local -xzf go1.17.1.linux-amd64.tar.gz &&
    rm go1* &&
    export PATH=$PATH:/usr/local/go/bin &&
    go get -v 'github.com/influxdata/influx-stress/cmd/...'
/root/go/bin/influx-stress insert -f --stats

 

This time I got a kernel panic:


[  518.193477] Kernel panic - not syncing: Watchdog detected hard LOCKUP on cpu 6
[  518.228126] CPU: 6 PID: 28191 Comm: influx-stress Tainted: PF          O 3.10.108 #42218
[  518.267191] Hardware name: HP ProLiant MicroServer Gen8, BIOS J06 04/04/2019
[  518.301214]  ffffffff814a2759 ffffffff814a16b1 0000000000000010 ffff880409b88d60
[  518.337704]  ffff880409b88cf8 0000000000000000 0000000000000006 0000000000000001
[  518.374045]  0000000000000006 ffffffff80000001 0000000000000030 ffff8803f4c4bc00
[  518.410480] Call Trace:
[  518.422504]  <NMI>  [<ffffffff814a2759>] ? dump_stack+0xc/0x15
[  518.451063]  [<ffffffff814a16b1>] ? panic+0xbb/0x1df
[  518.475140]  [<ffffffff810a9eb8>] ? watchdog_overflow_callback+0xa8/0xb0
[  518.508166]  [<ffffffff810db7d3>] ? __perf_event_overflow+0x93/0x230
[  518.539141]  [<ffffffff810da612>] ? perf_event_update_userpage+0x12/0xf0
[  518.571601]  [<ffffffff810152a4>] ? intel_pmu_handle_irq+0x1b4/0x340
[  518.603124]  [<ffffffff814a9d06>] ? perf_event_nmi_handler+0x26/0x40
[  518.634926]  [<ffffffff814a944e>] ? do_nmi+0xfe/0x440
[  518.659908]  [<ffffffff814a8a53>] ? end_repeat_nmi+0x1e/0x7e
[  518.688056]  <<EOE>>
[  518.698239] Rebooting in 3 seconds..

 

 

So this is a *huge* improvement on where I was before, but it still crashes if I really push it, and I'm not sure why @Kouill's server *didn't* crash running the same test 🤔

 

One thing to note is that the NC360T card is still physically installed in my machine, but disabled in the BIOS.

He used 420Xnu repo, are there fixes in there which are not included in jumkey's repo?

Link to post
Share on other sites

I add mp2sas driver use LSI9211-8I on ESXi,I just Identify 6 HDDs,but I have 8 HDDSs,I tried this config:

 

1.

"DiskIdxMap" : "0002",
 "SataPortMap" : "28",

 

"DiskIdxMap" : "00020C",
 "SataPortMap" : "288",

 

"DiskIdxMap" : "000C02",
 "SataPortMap" : "268",

vmware disk is 2

SAS is 3~8 lose latest 2 HDD,just Identify 6 HDDs

 

 

2."DiskIdxMap" : "0006",
 "SataPortMap" : "58",

vmware disk is 2

SAS is 6 7 8 12 13 lose latest 2 HDD too,just Identify 6 HDDs

 

3.

"DiskIdxMap" : "0009",
 "SataPortMap" : "88",

vmware disk is 2

SAS is 12 13 14 15 lose latest 4 HDD too,just Identify 4 HDDs

 

 

I tried more ,bug only can Identify 6 HDDs to more

 

is this loader can‘t Identify all LSI-SAS Disks?

 

 

Link to post
Share on other sites

Seeing all this… @ThorGroup I can’t help but hope that an easy to use GUI interface for the build process might be in the cards for the beta… the driver thing while good seems like it can get complicated.  Looking at it right now I get a bit overwhelmed haha

Edited by cferra
Link to post
Share on other sites
7 hours ago, Mixpower said:

He used 420Xnu repo, are there fixes in there which are not included in jumkey's repo?

Yes, I realise that. It’s the redpill-load repo, though, not redpill-lkm, so this only affects how the image is built and not the code that’s actually run on the system.

 

I’m using haydibe’s docker scripts to build my images (kouill appears to be building by hand) and couldn’t get them to run successfully using the 420Xnu redpill-load repo (which I’m unfamiliar with), so I used jumkey’s repo, which is a known-good alternative to the official TTG repos for unsupported DSM versions, and is what I’ve been using for all the other builds I’ve been testing. 

Edited by WiteWulf
Link to post
Share on other sites
58 minutes ago, cferra said:

Seeing all this… @ThorGroup I can’t help but hope that an easy to use GUI interface for the build process might be in the cards for the beta… the driver thing while good seems like it can get complicated.  Looking at it right now I get a bit overwhelmed haha

Go back and read through ThorGroup's posts in this thread, there aren't many of them and you'll learn a lot. A GUI of some sort is definitely planned, just not sure if it'll be around in time for the beta. Choosing and adding drivers is always going to be necessary as storage space restrictions mean that everything can't be loaded by default, but again, there was talk of a custom image that could be loaded before building a redpill boot stick that will figure out what drivers might be necessary for a given system.

Link to post
Share on other sites
il y a 34 minutes, WiteWulf a dit :

Yes, I realise that. It’s the redpill-load repo, though, not redpill-lkm, so this only affects how the image is built and not the code that’s actually run on the system.

 

I’m using haydibe’s docker scripts to build my images (kouill appears to be building by hand) and couldn’t get them to run successfully using the 420Xnu redpill-load repo (which I’m unfamiliar with), so I used jumkey’s repo, which is a known-good alternative to the official TTG repos for unsupported DSM versions, and is what I’ve been using for all the other builds I’ve been testing. 

 

I'm using the haydibe's docker for building images :)

I've just modify the repo for the loader and use of tg3 driver

  • Like 1
Link to post
Share on other sites
2 hours ago, WiteWulf said:

Go back and read through ThorGroup's posts in this thread, there aren't many of them and you'll learn a lot. A GUI of some sort is definitely planned, just not sure if it'll be around in time for the beta. Choosing and adding drivers is always going to be necessary as storage space restrictions mean that everything can't be loaded by default, but again, there was talk of a custom image that could be loaded before building a redpill boot stick that will figure out what drivers might be necessary for a given system.

No you’re right, the forum is definitely informative, I was following along up until build docker  .10 then life happened haha 

 

Yeah I think that if there were a gui or some kind of menu to select drivers and add them to the config file automatically it would be great, manually adding the extensions url to the config xml can get dicey, but you’re probably right about not hitting prime time until after the beta. 
 

im also sure that once it gets closer to that stage documentation will be more consolidated in a more digestible way too. 
 

:-)

Edited by cferra
  • Like 1
Link to post
Share on other sites

420Xnu is me , it is used and tune for my specific use case, and i was trying latest point release and pull from orphiee repo, and tune for my use case, when i tried it worked but i can’t take guarantees and responsibility if that doesn’t work for you. If you don’t know what you are doing always stick to thor group official repo.


Sent from my iPhone using Tapatalk

  • Like 1
Link to post
Share on other sites
4 minutes ago, cferra said:

manually adding the extensions url to the config xml can get dicey, but you’re probably right about not hitting prime time until after the beta.

 

There's hopefully a better way of managing extensions coming to haydibe's docker scripts soon....watch this space :D

  • Like 1
Link to post
Share on other sites
16 hours ago, haydibe said:

The first red line of the error message indicates that the "no space left on device" error is raised by the target, which is the first patition of the bootloader image.

 

Just create a new disk : 

dd if=/dev/zero of=redpill-template.img bs=1024k seek=256 count=0

+100M for the 1st partition

+75M for the 2nd

all for the rest 

 

losetup -P /dev/loop18 redpill-template.img

mkdosfs -F32 /dev/loop18p1
mkfs.ext2 /dev/loop18p2

 

mv redpill-template.img boot-image-template.img

gzip redpill-template.img

cp redpill-template.img.gz to ext folder (replace the existing)

 

Works fine ;)

  • Like 1
  • Thanks 1
Link to post
Share on other sites
3 hours ago, Kouill said:

 

I'm using the haydibe's docker for building images :)

I've just modify the repo for the loader and use of tg3 driver

I've tried doing it exactly the same as you, but it fails whenever I try to create the image as Orphee removed some extensions from their repo that weren't working properly (they were never intended for public use, but psychoboi32 used them in their forked repo. I'm guessing that as you have the extensions cached locally it carries on building okay for you.

Link to post
Share on other sites
2 minutes ago, WiteWulf said:

I've tried doing it exactly the same as you, but it fails whenever I try to create the image as Orphee removed some extensions from their repo that weren't working properly (they were never intended for public use, but psychoboi32 used them in their forked repo. I'm guessing that as you have the extensions cached locally it carries on building okay for you.

Do you need orphee extensions ? if not then ```docker run``` command and ext-manager remove that extension

Edited by psychoboi32
Link to post
Share on other sites
39 minutes ago, psychoboi32 said:

Do you need orphee extensions ? if not then ```docker run``` command and ext-manager remove that extension

 

Yeah, just trying to figure that out. You added them to the bundled_exts.json file, though, so I don't believe they can be removed with ext-manager

 

*edit*

 

Never mind. Kouill gave me a copy of their boot loader image and it still kernel panics my system when running the influx-test

Edited by WiteWulf
Link to post
Share on other sites
3 hours ago, RedwinX said:

Just create a new disk : 

dd if=/dev/zero of=redpill-template.img bs=1024k seek=256 count=0

+100M for the 1st partition

+75M for the 2nd

all for the rest 

 

losetup -P /dev/loop18 redpill-template.img

mkdosfs -F32 /dev/loop18p1
mkfs.ext2 /dev/loop18p2

 

mv redpill-template.img boot-image-template.img

gzip redpill-template.img

cp redpill-template.img.gz to ext folder (replace the existing)

 

Works fine ;)

 

 

I think the broader point being that with the size of the current drivers being put out, that volume needs to be larger by default unless there's some reason I'm unaware of that it's stuck at 40MB.  Probably wouldn't be a bad idea to make it a flag or part of the config for the docker script @haydibe @ThorGroup.   For instance, I can tell you with the mptsas drivers alone that pretty much blows up all the space for drivers by themselves.  If you need mptsas and mpt2sas or mpt3sas and the 42218 build you're dead in the water without a larger partition.

Link to post
Share on other sites

I am not going to implement functionality to the docker toochain that modifies the behavior of rp-lkm and rp-load.

It is already hard enough for people to distinguished if problems are located in the docker toochain or rp-load.

 

The only thing that might sense from my perspective is to support hooks for custom before/after scripts.

Though, even that will introduce a new vector for problems nobody wants to deal with.

 

On the other side, the toolchain builder script/files are all in "clear text" and not realy hard to understand..

Everyone is free to taylor them to their needs :)

 

Edited by haydibe
Link to post
Share on other sites
1 hour ago, RedwinX said:

Hey,

 

Have a little pb with a baremetal. It boot correctly (918+ 7.0 redpill repo), status : not installed 

Install ok, reboot and then : status not installed...

didn't have serial outpout. Any idea ? 

 

It occured once or twice, i dont remember the solution .. maybe replace the hdd ?

Link to post
Share on other sites
2 minutes ago, haydibe said:

I am not going to implement functionality to the docker toochain that modifies the behavior of rp-lkm and rp-load.

It is already hard enough for people to distinguished if problems are located in the docker toochain or rp-load.

 

The only thing that might sense from my perspective is to support hooks for custom before/after scripts.

Though, even that will introduce a new vector for problems nobody wants to deal with.

 

On the other side, the toolchain builder script/files are all in "clear text" and not realy hard to understand..

Everyone is free to taylor them to their needs :)

 

 

Understand, that's why I @'d you both.  I foresee endless requests for help by people adding drivers if those sizes remain the way they are.  If the end goal is: usable by everyone, the manual steps listed above probably won't cut it.  If the intended audience is the same as the beta: you understand enough coding and linux to unwind problems yourself, then it's not an issue.

 

Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.