RedPill Tinycore loader


Recommended Posts

@pocopico forgive me that my first post is about how to make it run in Proxmox.

 

These commands are tested on PVE7.1 and can be used as is. If you want a different id, make sure to modify the value before pasting everyhing into the PVE root shell:

# set vm id
id=930

# create image directory, download and uncomporess
mkdir -p /var/lib/vz/images/${id}
curl --location https://github.com/pocopico/tinycore-redpill/raw/main/tinycore-redpill.img.gz --output /var/lib/vz/images/${id}/tinycore-redpill.img.gz
gzip --decompress /var/lib/vz/images/${id}/tinycore-redpill.img.gz --keep

# create vm
qm create ${id} \
  --args "-drive 'if=none,id=synoboot,format=raw,file=/var/lib/vz/images/${id}/tinycore-redpill.img' -device 'qemu-xhci,addr=0x18' -device 'usb-storage,drive=synoboot,bootindex=5'" \
  --cores 2 \
  --cpu host \
  --machine q35 \
  --memory 2048 \
  --name DSM \
  --net0 virtio,bridge=vmbr0 \
  --numa 0 \
  --onboot 0 \
  --ostype l26 \
  --scsihw virtio-scsi-pci \
  --sata0 local-lvm:vm-${id}-disk-0,discard=on,size=100G,ssd=1 \
  --sockets 1 \
  --serial0 socket \
  --serial1 socket \
  --tablet 1

# create disk for sata0
pvesm alloc local-lvm ${id} vm-${id}-disk-0 100G

 

The last comand allocates 100GB on the local thin-lvm partition. Once the commandes are done, just start the vm

 

Update: virtio network driver works in tc ootb (of course you need to use bridge that is connected to a network with dhcp to see an ip)

 

Update2: you definitly want to follow @pocopico's suggestion and open a terminal once TinyCore is started, change the password for the user tc with `passwd tc` and then do the rest of the action using a comfortable ssh connection :)

Edited by haydibe
  • Like 1
Link to post
Share on other sites

I made a test run and of course forgot to change the user_config.json 🤦‍♂️.

 

After rebooting tc with `sudo reboot`, grub showed the expected bootloader, Synology Assistent did find the new instance, but of course It immediatly threw an error.

Then I rebooted into tc, but ./rploader.sh is gone.  

I did a shutdown with `sudo exitcheck.sh shutdown` this time.

 

Time to re-fetch the image and start over :)

 

Update: I added --keep to the gzip command, so that whenever the image needs to be replaced, it can be simply extracted again - just make sure the vm was shutdown before! Otherwise the extracted image wil be corrupt!

 

Edited by haydibe
Link to post
Share on other sites

I had to prefix update with sudo in order to work`sudo ./rploader.sh update now` to update the script. 

 

This time I used my own user_config.json (of course I replaced my real sn and mac1 with dummy values):

{
    "extra_cmdline": {
        "vid": "0x46f4",
        "pid": "0x0001",
        "sn": "1510LWN123456",
        "mac1": "00123456789A",
        "SasIdxMap": "0",
        "SataPortMap": "66",
        "DiskIdxMap": "0600"
    },
    "synoinfo": {
        "internalportcfg": "0xffff",
        "maxdisks": "16"
    },
    "ramdisk_copy": {}
}

 

And directly run the build `./rploader.sh build apollolake-7.0.1-42218 static` and it worked like a charm.

Then I rebooted again with `sudo reboot`.

 

This time the installation went fine :)

 

After setting up the admin account, the "Storage Manager" popped up. The 100gb sata block device was recognized as drive 1 and I could create a volume.

The only annoyance is Synology's drive compatibility warning they build-in into DSM. It can be easily skipped.

 

For the first time setup on Promox, it works like a charm.

 

Though, follow up modifications are not possible yet, as ./rploader.sh disappeared again.

 

 

 

 

Edited by haydibe
Link to post
Share on other sites

I have a mirror problem 
 

 Adding new extension from https://mirrors.llilii.cn/redpill/extensions/enhancements/boot-wait/config/rpext-index.json...
[#] Downloading remote file https://mirrors.llilii.cn/redpill/extensions/enhancements/boot-wait/config/rpext-index.json to /home/tc/redpill-load/custom/extensions/_new_ext_index.tmp_json
curl: (22) The requested URL returned error: 502 Bad Gateway
Warning: Transient problem: HTTP error Will retry in 1 seconds. 5 retries

Command run 

tc@box:~$ ./rploader.sh build bromolow-7.0.1-42218

 

Edited by Aigor
Link to post
Share on other sites

I had a problem with the extension urls containing quotes:

 

[#] Downloading remote file "https://raw.githubusercontent.com/pocopico/rp-ext/master/e1000e/rpext-index.json" to /home/tc/redpill-load/custom/extensions/_new_ext_index.tmp_json
[#] /usr/local/bin/curl --location --fail --progress-bar --retry 5 --output /home/tc/redpill-load/custom/extensions/_new_ext_index.tmp_json "https://raw.githubusercontent.com/pocopico/rp-ext/master/e1000e/rpext-index.json"
curl: (6) Could not resolve host: "https
Warning: Transient problem: timeout Will retry in 1 seconds. 5 retries left.

 

 

and added a trimming in: redpill-load/include/file.sh

 

# Downloads remote file to a specific file path
#
# Args: $1 URL to download from | $2 destination file | $3 hard fail on error [1 to do so]
rpt_download_remote()
{
    pr_info "Downloading remote file %s to %s" "${1}" "${2}"
    local out;
    local trimmed="${1%\"}"
    trimmed="${trimmed#\"}"
    pr_info "${CURL_PATH} --location --fail --progress-bar --retry 5 --output "${2}" ${trimmed}"
    out=$("${CURL_PATH}" --location --fail --progress-bar --retry 5 --output "${2}" ${trimmed})
    if [ $? -ne 0 ]; then
      if [[ "${3}" -eq 1 ]]; then
        pr_crit "Failed to download %s to %s\n\n%s" "${1}" "${2}" "${out}"
      else
        return 1
      fi
    fi
}

 

 

 

 

 

Link to post
Share on other sites
53 minutes ago, tester said:

I had a problem with the extension urls containing quotes:

 

[#] Downloading remote file "https://raw.githubusercontent.com/pocopico/rp-ext/master/e1000e/rpext-index.json" to /home/tc/redpill-load/custom/extensions/_new_ext_index.tmp_json
[#] /usr/local/bin/curl --location --fail --progress-bar --retry 5 --output /home/tc/redpill-load/custom/extensions/_new_ext_index.tmp_json "https://raw.githubusercontent.com/pocopico/rp-ext/master/e1000e/rpext-index.json"
curl: (6) Could not resolve host: "https
Warning: Transient problem: timeout Will retry in 1 seconds. 5 retries left.

 

 

and added a trimming in: redpill-load/include/file.sh

 


# Downloads remote file to a specific file path
#
# Args: $1 URL to download from | $2 destination file | $3 hard fail on error [1 to do so]
rpt_download_remote()
{
    pr_info "Downloading remote file %s to %s" "${1}" "${2}"
    local out;
    local trimmed="${1%\"}"
    trimmed="${trimmed#\"}"
    pr_info "${CURL_PATH} --location --fail --progress-bar --retry 5 --output "${2}" ${trimmed}"
    out=$("${CURL_PATH}" --location --fail --progress-bar --retry 5 --output "${2}" ${trimmed})
    if [ $? -ne 0 ]; then
      if [[ "${3}" -eq 1 ]]; then
        pr_crit "Failed to download %s to %s\n\n%s" "${1}" "${2}" "${out}"
      else
        return 1
      fi
    fi
}

 

 

 

 

 

 

Nice catch, i have updated the rploader script so when possible test again after updating the script with :

 

# rploader.sh update now 

 

 

Link to post
Share on other sites

Still have problem with extension 

[#] Checking runtime for required tools... [OK]
[#] Adding new extension from https://mirrors.llilii.cn/redpill/extensions/enhancements/boot-wait/config/rpext-index.json...
[#] Downloading remote file https://mirrors.llilii.cn/redpill/extensions/enhancements/boot-wait/config/rpext-index.json to /home/tc/redpill-load/custom/extensions/_new_ext_index.tmp_json
curl: (22) The requested URL returned error: 502 Bad Gateway
Warning: Transient problem: HTTP error Will retry in 1 seconds. 5 retries
Warning: left.
curl: (22) The requested URL returned error: 502 Bad Gateway#
Warning: Transient problem: HTTP error Will retry in 2 seconds. 4 retries
Warning: left.
curl: (22) The requested URL returned error: 502 Bad Gateway      #              #              #
Warning: Transient problem: HTTP error Will retry in 4 seconds. 3 retries
Warning: left.
curl: (22) The requested URL returned error: 502 Bad Gateway                           #              #              #             #
Warning: Transient problem: HTTP error Will retry in 8 seconds. 2 retries
Warning: left.
curl: (22) The requested URL returned error: 502 Bad Gateway                                                              #              #            #          #
Warning: Transient problem: HTTP error Will retry in 16 seconds. 1 retries
Warning: left.
curl: (22) The requested URL returned error: 502 Bad Gateway                                                                                               #          #        #       #

[!] Failed to download https://mirrors.llilii.cn/redpill/extensions/enhancements/boot-wait/config/rpext-index.json to /home/tc/redpill-load/custom/extensions/_new_ext_index.tmp_json



*** Process will exit ***

[!] Failed to add "https://mirrors.llilii.cn/redpill/extensions/enhancements/boot-wait/config/rpext-index.json" as an extension:


,

*** Process will exit ***
[!] Failed to install unknowno.boot-wait bundled extension - see errors above

*** Process will exit ***
FAILED : Loader creation failed check the output for any errors

 

Link to post
Share on other sites

@Aigor Is it safe to assume that you use your own configuration files? I have never seen that URL in any of my attempts...

 

If I run with the default packaged ones (+updating the script itself of course), none of my downloads points to those urls...

[#] Extension thethorgroup.boot-wait index is already up to date
[#] Updating thethorgroup.virtio extension...
[#] Downloading remote file https://github.com/jumkey/redpill-load/raw/develop/redpill-virtio/rpext-index.json to /home/tc/redpill-load/custom/extensions/_new_ext_index.tmp_json
############################################################################################################################################ 100.0%

 

 

Are you sure it's not a problem of your configuration or the repo you are pointing to?

 

Update: the problem originates in https://github.com/unknown-o/redpill-load/blob/master/bundled-exts.json. as unknown-o's repository is used bromolow-7.0.1-42218

 

Edited by haydibe
Link to post
Share on other sites

I receive the same error and I manually replace the entries in:

 

redpill-load/bundled-exts.json

 

with:

 

{
    "unknowno.virtio": "https://raw.githubusercontent.com/jumkey/redpill-load/develop/redpill-virtio/rpext-index.json",
    "unknowno.boot-wait": "https://raw.githubusercontent.com/pocopico/rp-ext/main/redpill-boot-wait/rpext-index.json"
}

 

@pocopico: yes the fix works
 

I still have to find out how to configure everything so that it boots in Virtualbox or esxi 6.7 .. I get many problems on my esxi 6.7 on Gen8 Micro ...

But I just started to figure out how to configure redpill so I will need time and find good examples ..

Edited by tester
  • Like 1
  • Thanks 1
Link to post
Share on other sites
20 minutes ago, haydibe said:

@Aigor Is it safe to assume that you use your own configuration files? I have never seen that URL in any of my attempts...

 

If I run with the default packaged ones (+updating the script itself of course), none of my downloads points to those urls...


[#] Extension thethorgroup.boot-wait index is already up to date
[#] Updating thethorgroup.virtio extension...
[#] Downloading remote file https://github.com/jumkey/redpill-load/raw/develop/redpill-virtio/rpext-index.json to /home/tc/redpill-load/custom/extensions/_new_ext_index.tmp_json
############################################################################################################################################ 100.0%

 

 

Are you sure it's not a problem of your configuration or the repo you are pointing to?

 

That's strange, i use config from tinycore images without any modification 

 

Link to post
Share on other sites

The rp-load repo for bromollow 7.0.1 in custom_config.json causes the issue. From the past I know how impossible it is to keep track of all the different repositories. So I guess we will have to use our own configuration there. 

 

I started to scp the user_config.json to tc already. Now I am going to do the same for the custom_config.json.

 

The issue itself is unrelated weather the rploader works or not. The same will happen with rp-helper, if the same custom_config.json is used to build an image. 

Edited by haydibe
Link to post
Share on other sites
41 minutes ago, Aigor said:

That's strange, i use config from tinycore images without any modification 

 

It is not :) I used apollolake 7.0.1., which points to jumkey's repo, while you used bromolow 7.0.1, which points to unknown-o's repo. 

When I tried bromollow 7.0.1 as well, I had the same problem as you. Make sure to bring your own custom_config.json.

 

Assuming you have a health configuration for custom_config.json and user_config.json, you can copy over "good files" from a ssh host to tc: 

1. in tc: open a terminal

2. in tc (in terminal): passwd  # to change the password 

3. in tc (in terminal): ifconfig  # to see the ip of tc

4. ssh host: scp custom_config.json tc@ip-of-tc:~/custom_config.json

5. ssh host: scp user_config.json tc@ip-of-tc:~/user_config.json

6. ssh host: ssh tc@ip-of-tc

7. ssh-host (while connected to tc): sudo ./rploader.sh update now # wasn't working without sudo for me.

8. ssh-host (while connected to tc): ./rploader.sh build bromolow-7.0.1-42218 static

9. ssh-host (while connected to tc): ./rploader.sh clean now # required, otherwise the next command will fail

10. ssh-host (while connected to tc): filetool.sh -b sdb3 # persist tc configuration back to tc partition. Your tc partition might be on a different disk.. change sdb to your device if needed.

11. ssh-host (while connected to tc): sudo reboot

Edited by haydibe
  • Like 2
Link to post
Share on other sites

Extension handling is not implemented so far, except listing detected devices.

You will have to do this manualy. Something like this should do the trick:
 

./rploader.sh download bromolow-7.0.1-42218
./redpill-load/ext-manager.sh add https://raw.githubusercontent.com/pocopico/rp-ext/master/qla2xxx/rpext-index.json
./rploader.sh build bromolow-7.0.1-42218 static

 

If I am not mistaken, @pocopico is researching on how to auto detect required drivers and add extensions for them automaticly. This might explain why the "ext" action is not simply wired to ext-manager, like it is in rp-helper. 

  • Like 1
Link to post
Share on other sites
15 minutes ago, haydibe said:

... whoever gets the idea to build a web ui: make sure the runtime it needs is available/works on tiny core and fits into the bootloader image.

 

 

I'm not sure that is a good idea to build web ui as a part of bootloader. Its completely different parts. I mean we can use any os as a way to load wizard of generating the bootloader image.

Link to post
Share on other sites

The beauty of the tinycore image is that it actualy is supposed to be used on the target system, allowing to detect the required drivers right away - configure, test, repeat -> until things are sorted out.  No need to "burn" an usb-stick on each attempt or to copy vmdk files or img files to hypervisiors.

 

 

 

 

 

 

 

Link to post
Share on other sites

Hi all,

 

I find all great ideas ! I like the webui but maybe it’s overkill we’ll see how it goes. 
 

Loader capacity is not a big issue cause In tinycore you can download software upon request, same like I do for compiling the modules. The command is  tce-load -iw xxx software 

 

As for the extensions, there is a function on version 0.2.2, that tries to figure out the class/pciid and then tries to match the extensions listed on my repo and automatically adds them. I’m still missing the manual add function though but It in my plans to do that soon.

 

 

 

 

Edited by pocopico
Link to post
Share on other sites
6 hours ago, Aigor said:

I would like to add this extension 
 


https://raw.githubusercontent.com/pocopico/rp-ext/master/qla2xxx/rpext-index.json

But i'm not able to understand howto, 


If it’s listed with lspci then it should be added automatically. Can you share the lspci -nnq ?

Link to post
Share on other sites
1 hour ago, pocopico said:


If it’s listed with lspci then it should be added automatically. Can you share the lspci -nnq ?

 

It's a VM with access to Qlogic Passthrough

0000:00:00.0 Host bridge [0600]: Intel Corporation 440BX/ZX/DX - 82443BX/ZX/DX Host bridge [8086:7190] (rev 01)
0000:00:01.0 PCI bridge [0604]: Intel Corporation 440BX/ZX/DX - 82443BX/ZX/DX AGP bridge [8086:7191] (rev 01)
0000:00:07.0 ISA bridge [0601]: Intel Corporation 82371AB/EB/MB PIIX4 ISA [8086:7110] (rev 08)
0000:00:07.1 IDE interface [0101]: Intel Corporation 82371AB/EB/MB PIIX4 IDE [8086:7111] (rev 01)
0000:00:07.3 Bridge [0680]: Intel Corporation 82371AB/EB/MB PIIX4 ACPI [8086:7113] (rev 08)
0000:00:07.7 System peripheral [0880]: VMware Virtual Machine Communication Interface [15ad:0740] (rev 10)
0000:00:0f.0 VGA compatible controller [0300]: VMware SVGA II Adapter [15ad:0405]
0000:00:11.0 PCI bridge [0604]: VMware PCI bridge [15ad:0790] (rev 02)
0000:00:15.0 PCI bridge [0604]: VMware PCI Express Root Port [15ad:07a0] (rev 01)
0000:00:15.1 PCI bridge [0604]: VMware PCI Express Root Port [15ad:07a0] (rev 01)
0000:00:15.2 PCI bridge [0604]: VMware PCI Express Root Port [15ad:07a0] (rev 01)
0000:00:15.3 PCI bridge [0604]: VMware PCI Express Root Port [15ad:07a0] (rev 01)
0000:00:15.4 PCI bridge [0604]: VMware PCI Express Root Port [15ad:07a0] (rev 01)
0000:00:15.5 PCI bridge [0604]: VMware PCI Express Root Port [15ad:07a0] (rev 01)
0000:00:15.6 PCI bridge [0604]: VMware PCI Express Root Port [15ad:07a0] (rev 01)
0000:00:15.7 PCI bridge [0604]: VMware PCI Express Root Port [15ad:07a0] (rev 01)
0000:00:16.0 PCI bridge [0604]: VMware PCI Express Root Port [15ad:07a0] (rev 01)
0000:00:16.1 PCI bridge [0604]: VMware PCI Express Root Port [15ad:07a0] (rev 01)
0000:00:16.2 PCI bridge [0604]: VMware PCI Express Root Port [15ad:07a0] (rev 01)
0000:00:16.3 PCI bridge [0604]: VMware PCI Express Root Port [15ad:07a0] (rev 01)
0000:00:16.4 PCI bridge [0604]: VMware PCI Express Root Port [15ad:07a0] (rev 01)
0000:00:16.5 PCI bridge [0604]: VMware PCI Express Root Port [15ad:07a0] (rev 01)
0000:00:16.6 PCI bridge [0604]: VMware PCI Express Root Port [15ad:07a0] (rev 01)
0000:00:16.7 PCI bridge [0604]: VMware PCI Express Root Port [15ad:07a0] (rev 01)
0000:00:17.0 PCI bridge [0604]: VMware PCI Express Root Port [15ad:07a0] (rev 01)
0000:00:17.1 PCI bridge [0604]: VMware PCI Express Root Port [15ad:07a0] (rev 01)
0000:00:17.2 PCI bridge [0604]: VMware PCI Express Root Port [15ad:07a0] (rev 01)
0000:00:17.3 PCI bridge [0604]: VMware PCI Express Root Port [15ad:07a0] (rev 01)
0000:00:17.4 PCI bridge [0604]: VMware PCI Express Root Port [15ad:07a0] (rev 01)
0000:00:17.5 PCI bridge [0604]: VMware PCI Express Root Port [15ad:07a0] (rev 01)
0000:00:17.6 PCI bridge [0604]: VMware PCI Express Root Port [15ad:07a0] (rev 01)
0000:00:17.7 PCI bridge [0604]: VMware PCI Express Root Port [15ad:07a0] (rev 01)
0000:00:18.0 PCI bridge [0604]: VMware PCI Express Root Port [15ad:07a0] (rev 01)
0000:00:18.1 PCI bridge [0604]: VMware PCI Express Root Port [15ad:07a0] (rev 01)
0000:00:18.2 PCI bridge [0604]: VMware PCI Express Root Port [15ad:07a0] (rev 01)
0000:00:18.3 PCI bridge [0604]: VMware PCI Express Root Port [15ad:07a0] (rev 01)
0000:00:18.4 PCI bridge [0604]: VMware PCI Express Root Port [15ad:07a0] (rev 01)
0000:00:18.5 PCI bridge [0604]: VMware PCI Express Root Port [15ad:07a0] (rev 01)
0000:00:18.6 PCI bridge [0604]: VMware PCI Express Root Port [15ad:07a0] (rev 01)
0000:00:18.7 PCI bridge [0604]: VMware PCI Express Root Port [15ad:07a0] (rev 01)
0000:02:00.0 USB controller [0c03]: VMware USB1.1 UHCI Controller [15ad:0774]
0000:02:02.0 USB controller [0c03]: VMware USB2 EHCI Controller [15ad:0770]
0000:02:03.0 SATA controller [0106]: VMware SATA AHCI controller [15ad:07e0]
0000:02:05.0 SATA controller [0106]: VMware SATA AHCI controller [15ad:07e0]
0000:03:00.0 Serial Attached SCSI controller [0107]: VMware PVSCSI SCSI Controller [15ad:07c0] (rev 02)
0000:0b:00.0 Fibre Channel [0c04]: QLogic Corp. ISP2532-based 8Gb Fibre Channel to PCI Express HBA [1077:2532] (rev 02)
0000:13:00.0 Ethernet controller [0200]: Intel Corporation 82574L Gigabit Network Connection [8086:10d3]
0001:07:00.0 SATA controller [0106]: Marvell Technology Group Ltd. 88SE9235 PCIe 2.0 x2 4-port SATA 6 Gb/s Controller [1b4b:9235] (rev 11)
0001:08:00.0 SATA controller [0106]: Marvell Technology Group Ltd. 88SE9235 PCIe 2.0 x2 4-port SATA 6 Gb/s Controller [1b4b:9235] (rev 11)
0001:09:00.0 SATA controller [0106]: Marvell Technology Group Ltd. 88SE9235 PCIe 2.0 x2 4-port SATA 6 Gb/s Controller [1b4b:9235] (rev 11)
0001:0a:00.0 SATA controller [0106]: Marvell Technology Group Ltd. 88SE9235 PCIe 2.0 x2 4-port SATA 6 Gb/s Controller [1b4b:9235] (rev 11)

 

Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.