RedPill - the new loader for 6.2.4 - Discussion


Recommended Posts

9 hours ago, snowfox said:

Have I installed the network card driver correctly?   Why can't I see the 10G network card in the network interface?Only see the 1G network card!

1Gnic.jpg

10Gnic.jpg

There are two things that come in mind :

 

1. 10Gb driver is not loaded correctly

2. /etc/synoinfo.conf does not have the support_10g=y

 

log in with ssh and verify the above 

Link to post
Share on other sites

Today we will break the tradition a bit and respond to all questions first. Since the forum has a limit of 50 quoted posts we have to split our response into multiple posts ;) The last post will be with the actual code updates so if you're reading it and not seeing a post with updates - just wait - we're writing as you're reading this.

 

------------------------------------------

 

On 8/29/2021 at 4:21 PM, coint_cho said:

Oh I've rechecked my config and somehow it automatically reverted back to e1000 for some reason.. changed it to e1000e and I'm getting an IP and currently have installed DiskStation on it, but right now another road block unfortunately. 

 

image.thumb.png.c67a17f47407c05473d5e2f7a2378a40.png

I've seen other people with the same issues here so.. it's a 128GB SATA disk mounted in VMWare. 

It's fixed in the today's relese :) [will be up on GH after the update post]

 

 

On 8/29/2021 at 4:25 PM, haydibe said:

@ThorGroup thank you once again for the update and adressing all issues that come up!

 

Would you mind to provide some insights about where `"DiskIdxMap": "0C", "SataPortMap": "1", "SasIdxMap": "0"` actualy are ment to be configured in the user_config.json? According the redpill-load README.md it should be a k/v structure specified in "synoinfo". Though so far everyone seems to add them in "extra_cmdline".


Is this the intended usage?


...
"synoinfo": {
  "DiskIdxMap": "OC",
  "SataPortMap": "1",
  "SasIdxMap": "0"
},
...

 

Afterall, those parameters are not only usefull for those that use sata_dom boot, but also for those that use the usb boot and want to bring order into the drive ordering.

 

Note: I believe SasIdxMap is not required if no sas controller is passed into the vm, on the other side it doesn't seem to make any harm.

The difference between synoinfo and cmdline is subtle but important. The synoinfo defines the properties of the platform and is mainly used to configure the DSM itself (e.g. how many drives it sees in the UI, does it allow for creation of the SHR etc). These settings can often be changed and everything will work. For example some models don't support VMM not because it wouldn't run but because running anything besides a hello world on it will exhaust all the resources. Cmdline parameters in the other hand configure the kernel directly. Changing them to something inappropriate can often lead to the sysem not booting at all or not seeing disks or not being able to get an IP address from DHCP (e.g. when you set a wrong MAC).

 

The DiskIdxMap, SataPortMap, and SasIdxMap are all cmdline parameters. Putting them in the synoinfo section will produce no results. They must be put in cmdline as they're read by the SCSI driver in the kernel. There's no connection between synoinfo and cmdline reading routines at all. As for how to configure them that short thread explains this very well:

 

Setting SasIdxMap does nothing if you have no SAS controller. These options are internally stored in variables inside of the kernel but only used when the actual drive is connected to a given controller. Without that they have no effect.

 

 

On 8/29/2021 at 4:57 PM, gadreel said:

@ThorGroup as always thanks for taking the time to reply to all those posts and assist all of us.

 

Now that the loader works for most (maybe all) users in order to make testing easier and efficient I have a stupid question to ask. When you add new commits to your repositories and we generate the new bootloader with haydbibe's docker solution in order to test properly do we have to format the disks/delete and recreate the virtio sata disks and start the installation procedure all over again or just boot with the newly generated bootloader from the previous installation and if it works then all is good?

You only need to replace the boot image. In fact internally we simply have a drive in proxmox with "sata0: file=/our/data/store/image.img". We just replace the file and restart the VM to boot from a new one. Due to some QEmu peculiarities it's recommended to shut down the VM, wait few seconds, and start it again (as restart SOMETIMES may not read the full image again for some unknown reasons).

 

 

On 8/29/2021 at 5:48 PM, coint_cho said:

Ok, after a few trial and errors, currently I've identified that DS918+ can't be booted and installed since the bootloader will be counted in DiskStation together and it will format that vdk as well(DS918+ only has USB boot option), while the DS3615XS can be booted from both USB/SATA(Storage doesn't work and will only show not compatible unfortunately), selecting booting from SATA doesn't prompt me to format my bootloader vdk image, anyone knows any workaround for this? (I'm using VMWare Workstation 16). However, it can be booted from USB and installed in Proxmox with no issues(Referring to DS918+ Apollo Lake).

The 918+ now has an experimental SATA support ;)

 

 

On 8/29/2021 at 6:34 PM, dodo-dk said:

I have read the q35 is better than i440fx. So I switch to q35 but with that, the boot takes longer and my Volume2 is crashed.

I doesn't found all Harddrives. I have one Virtual and 5 Passtrough Disks. I have switched back to i440fx and all works, thanks god, Volume2 is back.

 

Do I need mapping or something with the q35 settings?

As haydibe mentioned you need to configure the mapping. Q35 is better because it's a newer platform + supports things like PCI-Ex passthrough. Once you configure the proper DiskIdxMap and SataPortMap it will see all disks (and the array will not be crashed as it will not see disks as missing).

This whole problem exists because disks beyond 26th are starting to be named e.g. /dev/sdaa, /dev/sdab, /dev/sdac... and this schema is not supported by the OS. To our knowledge there are no devices from syno with >26 disks without a SAS controller.

 

 

On 8/29/2021 at 9:37 PM, haydibe said:

Actualy I have never seen a QEMU XML. Haven't sumbled accross it on Proxmox so far, no such files are location in /etc or /lib.

I have yet to understand if Proxmox allows that level of configuration for storage.

 

Update: seems Unraid uses libvirt as a wrapper arround QEMU/KVM. Proxmox directly instructs QEMU/KVM (Warning: do not try to intall libvirt on Proxmox, it might break your installation!).

These XMLs are a standardized way to use libvirt. Proxmox went its own way as both were developed kind of in the same time. Additionally, what most of the people don't know, configuration "files" in proxmox aren't files - they're stored in a synchronized SQLite database and just presented as a filesystem ;) This allows for efficient cluster synchronization.

 

 

On 8/29/2021 at 9:45 PM, gadreel said:

 

Thanks for replying. I will leave it as is if it will not cause any issues...

Because if I remember correct maybe I read something somewhere (I could be very wrong) that DS918+ cannot take more than 12 drives, but maybe that has nothing to do with those numbers.

We didn't try ourselves but you can attempt to set synoinfo "maxdisks" parameter beyond 16. Theoretically it should be able to host 26 drives if mapping in cmdline is set properly.

 

 

On 8/30/2021 at 12:15 AM, T-REX-XP said:

@ThorGroupDo you have an example how to use the PMU emulation?? Any examples ?? Thanks in advance.

 

As I saw, now all communication with PMU will be mirrored to /dev/ttyS1 so we can easily find the output of the PMU when some sort of FAN or beep settings has been changed in the DSM settings. For example Fan speed set to Quite mode


[ 2250.254824] <redpill/pmu_shim.c:324> Got 1 bytes from PMU: reason=1 hex={2d} ascii="-"
[ 2250.476426] <redpill/pmu_shim.c:324> Got 1 bytes from PMU: reason=1 hex={6b} ascii="k"
[ 2250.493676] <redpill/pmu_shim.c:249> Unknown 1 byte PMU command with signature hex="6b" ascii="k"

It give us the possibility to connect Arduino and try to build the our own PMU with hardware copy button))

That's the goal - we want to provide an interface which simply sends commands to an arduino-like device which can serve that. The fan control is... strange. It seems to always send "k" no matter what. There is probably something we've missed (or maybe it goes through the mfgBIOS ioctl? either way both outputs need to be combined).

 

 

On 8/30/2021 at 12:50 AM, yanjun said:

 

@ThorGroup I put the full logs of my boot stage.

 

In addition, this hba card works normally on Jun’s 1.04b loader, and the bios type is OVMF (UEFI). I know this does not mean it is a redpill problem, it may also be a compatibility problem between SEABIOS and HBA card, but so far, I don't know how to solve it

That could be an issue with the card being seen as a proper SAS device. After quickly asking around none of have any free LSI card running in a server to test but we will look into it. Can you put a GH issue in the kernel module repo? It probably needs a small kick similar to VirtIO SCSI - currently as you see the SCSI driver attempts to add a disk with a "/dev/" name (literally empty name) which should and will fail. The problem lies somewhere in the syno modifications to the SCSI driver.

 

 

On 8/30/2021 at 12:56 AM, tocinillo2 said:

@ThorGroupthanks for the update! My request about uefi support it's not about performance, it' because a lot of apollolake and apollolake refresh mainboards doesn't support legacy boot (or CSM) like J4105, J4125, J5040, J5005 and new models.

 

Again, thanks for this fanstatic work! 

That is a good point. We will take this into consideration then and offer a hybrid-boot version supporting BIOS and EFI in a single image, with the default recommendation of a BIOS boot.

 

 

On 8/30/2021 at 2:50 AM, mcdull said:

I have some issue in migrating.

I do have some images with facial recognition when old version in Jun's loader still works. But now it is must not work with DSM7 (I uses AMD cpu) and no GPU at all.

 

Now the synology photos keep telling me that its trying to migrate from old photos and moments app, and keep waiting the 30000+ photos to complete.

Any suggestions for me?

Someone mentioned the problem with facial recognition & v7. There should be a separate thread for that as this is an issue with the photo app and its checking of S/N (it has to be a real one or the binary of photo app must be patched). This is out of scope of the loader part ;)

 

 

On 8/30/2021 at 6:01 AM, apriliars3 said:

I have the same issue. Any idea that how can fix it? I test it with VMWare.

See the next post with updates - now ESXi is supported with SMART emulation.

 

 

On 8/30/2021 at 1:59 PM, scoobdriver said:

do you know if there are any models which support Sata DoM and HW transcoding please?

With this update you can use 918+ [will be on GH after the update post below]. We are looking into one of the models with native support for nVidia GPUs so that transcoding can work with either Intel QS or NVENC. While there are hacks to run the nVidia driver on other platforms its cumbersome and possibly unstable (as, putting it mildly, nV drivers are .... picky). We want to go that route as people with AMD cannot use Intel QS and cards like Quadro P200 are cheap, low power (peak of like 25-30W), and have plenty of enc/dec power (IIRC 7 streams at once).

 

 

On 8/30/2021 at 4:44 PM, hoangminh88 said:

 

 I checked: We can use Balena Atcher to burn USB

Yes, on Windows it's a pretty good tool. On macOS or Linux the dd is the simplest and already present in the system ;)

 

 

On 8/30/2021 at 4:49 PM, scoobdriver said:

Wish I could try it , but I'm using ESXi and there is no Sata Dom support , so It won't load . (So using 3615+ DSM 7.0 beta ) 

 

Does DS918+ 7.0 have the i915.ko ? 

Now [will be on GH after the update post below] the support for ESXi & SATA boot on 918+ so you should be good to go.

 

 

On 8/30/2021 at 5:40 PM, Orphée said:

Comparing with Jun's loader 6.2.3 :

 

image.thumb.png.ad947f2af1e44b373007973bf62c84cb.png

 

 

dmesg_70_ssd_sas.txt 216.2 kB · 5 downloads

 

What we actually do is qemu-img on Linux + vmkfstools - ESXi is very picky as to VMDKs. Older versions worked great but ESXi 6 and 7 often refuse to properly use VMDKs created by something else than VMWare products. This is why "vmkfstools -i input.vmdk output.vmkd -d thin" works and actually makes an image which works properly with VMWare.

 

 

On 8/30/2021 at 5:43 PM, psychoboi32 said:


please play with it i tried several times but it not loaded by any means even i installed esxi on spare disk to get it working end result is same not loaded

edit :- you got loaded wow i didn’t get that far means i want to make image which preloads that so i didn’t add any drive just lsi sas card and drives not detected at all

Sent from my iPhone using Tapatalk

Ah, yes... it seems like new syno SCSI driver sees the disks as SAS instead of SATA. You have to play with SasIdxMap then... or we can add a small patch like with VirtIO SCSI to make sure they're treated like SATA. However, DSM should theoretically handle it just fine for data disks (but not for the boot disk). Once we get a free LSI to test we will look into that. Can you post an issue on GH in the kernel module repo?

 

 

On 8/30/2021 at 5:53 PM, Orphée said:

I tried under ubuntu VM but it failed for me... the converted vmdk was not working. Whereas V2V converter did the job.

If you use ESXi try doing "vmkfstools -i input.vmdk output.vmkd -d thin" on the ESXi box after qemu-img - for some weird reason newer versions of ESXi refuse to even see a normal VMDK if it wasn't made by their tools...

 

 

On 8/30/2021 at 6:05 PM, pocopico said:

Hi, 

 

the same happens for virtual disks as well but this workaround works fine for now

 

https://github.com/RedPill-TTG/redpill-lkm/issues/14#issuecomment-903683483

Now, with SMART emulation, that workaround is no longer needed :)

A new release will be posted on GH as soon as we finish fighing with the forum edit to post all posts.

 

 

On 8/30/2021 at 11:30 PM, jforts said:

Hi, I try the redpill-loader with baremetal, I cannot access to any serial port, how to access the console from a monitor and keyboard?

Because the screen froze at Booting the kernel...

The issue with baremetal is that you cannot see any output or console if you don't have a serial port. The syno kernel is built without VGA support, so you only see the output from preboot as kernel simply doesn't initialize GPU (so it's not really frozen per se but just not initialized after GRUB passes the control to the kernel).

 

 

On 8/31/2021 at 5:12 AM, Amoureux said:

@ThorGroup I have some errors after install DSM7.0 on my hardware.

 


[    2.574597] redpill: module verification failed: signature and/or required key missing - tainting kernel
[    3.243540] <redpill/override_symbol.c:404> Failed to locate vaddr for sys_call_table using kallsyms - falling back to memory search
[    7.654945] intel-lpss: probe of 0001:00:16.0 failed with error -22
[    7.661824] intel-lpss: probe of 0001:00:18.0 failed with error -22
[    7.668735] intel-lpss: probe of 0001:00:19.0 failed with error -22
[    7.675644] intel-lpss: probe of 0001:00:19.2 failed with error -22
[   16.797306] ahci: probe of 0001:01:00.0 failed with error -22
[   16.803420] ahci: probe of 0001:00:12.0 failed with error -22
[   17.338180] xhci_hcd 0001:00:15.0: init 0001:00:15.0 fail, -16
[   17.344410] xhci_hcd: probe of 0001:00:15.0 failed with error -16
[   17.755953] <redpill/bios_shim.c:207> Symbol #29 in mfgBIOS "apollolake_synobios" {synobios_record_scsi_error_event}<ffffffffa00ba480>
  [   17.794780] <redpill/bios_shim.c:207> Symbol #32 in mfgBIOS "apollolake_synobios" {synobios_disk_reset_fail_report}<ffffffffa00ba580>
  [   17.832428] <redpill/bios_shim.c:207> Symbol #35 in mfgBIOS "apollolake_synobios" {synobios_sata_error_report}<ffffffffa00ba690>
  [   17.891199] <redpill/bios_shim.c:207> Symbol #39 in mfgBIOS "apollolake_synobios" {synobios_error_fs_btrfs_event}<ffffffffa00ba7e0>
  [   17.903689] <redpill/bios_shim.c:207> Symbol #40 in mfgBIOS "apollolake_synobios" {synobios_error_fs_event}<ffffffffa00ba830>
  [   17.952685] <redpill/bios_shim.c:207> Symbol #44 in mfgBIOS "apollolake_synobios" {synobios_error_oom_event}<ffffffffa00baa50>
  [   18.418432] <redpill/bios_shim.c:207> Symbol #83 in mfgBIOS "apollolake_synobios" {fan_fail}<ffffffffa00c4800>
  [   18.535314] <redpill/bios_shim.c:207> Symbol #93 in mfgBIOS "apollolake_synobios" {fan_fail}<ffffffffa00c4c40>
  [   18.647725] <redpill/bios_shim.c:207> Symbol #103 in mfgBIOS "apollolake_synobios" {fan_fail}<ffffffffa00c5080>
  [   18.761576] <redpill/bios_shim.c:207> Symbol #113 in mfgBIOS "apollolake_synobios" {fan_fail}<ffffffffa00c54c0>
  [   18.881884] <redpill/bios_shim.c:207> Symbol #123 in mfgBIOS "apollolake_synobios" {fan_fail}<ffffffffa00c5900>
  [   18.995030] <redpill/bios_shim.c:207> Symbol #133 in mfgBIOS "apollolake_synobios" {fan_fail}<ffffffffa00c5d40>
  [   19.112728] <redpill/bios_shim.c:207> Symbol #143 in mfgBIOS "apollolake_synobios" {fan_fail}<ffffffffa00c6180>
  [   19.226639] <redpill/bios_shim.c:207> Symbol #153 in mfgBIOS "apollolake_synobios" {fan_fail}<ffffffffa00c65c0>
  [   19.339744] <redpill/bios_shim.c:207> Symbol #163 in mfgBIOS "apollolake_synobios" {fan_fail}<ffffffffa00c6a20>
  [   20.754211] usb 1-12: device descriptor read/64, error -71
  [   20.979213] usb 1-12: device descriptor read/64, error -71
  [   21.336220] usb 1-12: device descriptor read/64, error -71
  [   21.570232] usb 1-12: device descriptor read/64, error -71
  [   22.743253] usb 1-12: device not accepting address 6, error -71
  [   23.293258] usb 1-12: device not accepting address 7, error -71
  [   32.515336] systemd[1]: Cannot add dependency job for unit SynoLedBrightness.service, ignoring: Unit SynoLedBrightness.service is masked.
[   32.528516] systemd[1]: Cannot add dependency job for unit pkg-install-high-priority-builtin.service, ignoring: Unit pkg-install-high-priority-builtin.service failed to load: No such file or directory.
[   32.547485] systemd[1]: Cannot add dependency job for unit pkg-install-low-priority-builtin.service, ignoring: Unit pkg-install-low-priority-builtin.service failed to load: No such file or directory.
  [   34.688838] igb: probe of 0001:02:00.0 failed with error -5
[   34.694790] igb: probe of 0001:03:00.0 failed with error -5
  [   35.247539] i915 0000:00:02.0: Direct firmware load for i915/kbl_dmc_ver1_04.bin failed with error -2
  [   35.257396] i915 0000:00:02.0: Falling back to user helper
  [   36.132828] <redpill/bios_shim.c:207> Symbol #29 in mfgBIOS "apollolake_synobios" {synobios_record_scsi_error_event}<ffffffffa064b480>
  [   36.171822] <redpill/bios_shim.c:207> Symbol #32 in mfgBIOS "apollolake_synobios" {synobios_disk_reset_fail_report}<ffffffffa064b580>
  [   36.209524] <redpill/bios_shim.c:207> Symbol #35 in mfgBIOS "apollolake_synobios" {synobios_sata_error_report}<ffffffffa064b690>
  [   36.261526] <redpill/bios_shim.c:207> Symbol #39 in mfgBIOS "apollolake_synobios" {synobios_error_fs_btrfs_event}<ffffffffa064b7e0>
  [   36.274110] <redpill/bios_shim.c:207> Symbol #40 in mfgBIOS "apollolake_synobios" {synobios_error_fs_event}<ffffffffa064b830>
  [   36.323244] <redpill/bios_shim.c:207> Symbol #44 in mfgBIOS "apollolake_synobios" {synobios_error_oom_event}<ffffffffa064ba50>
  [   36.778878] <redpill/bios_shim.c:207> Symbol #83 in mfgBIOS "apollolake_synobios" {fan_fail}<ffffffffa0655800>
  [   36.891358] <redpill/bios_shim.c:207> Symbol #93 in mfgBIOS "apollolake_synobios" {fan_fail}<ffffffffa0655c40>
  [   37.004279] <redpill/bios_shim.c:207> Symbol #103 in mfgBIOS "apollolake_synobios" {fan_fail}<ffffffffa0656080>
  [   37.118604] <redpill/bios_shim.c:207> Symbol #113 in mfgBIOS "apollolake_synobios" {fan_fail}<ffffffffa06564c0>
  [   37.232185] <redpill/bios_shim.c:207> Symbol #123 in mfgBIOS "apollolake_synobios" {fan_fail}<ffffffffa0656900>
  [   37.345777] <redpill/bios_shim.c:207> Symbol #133 in mfgBIOS "apollolake_synobios" {fan_fail}<ffffffffa0656d40>
  [   37.459209] <redpill/bios_shim.c:207> Symbol #143 in mfgBIOS "apollolake_synobios" {fan_fail}<ffffffffa0657180>
  [   37.573346] <redpill/bios_shim.c:207> Symbol #153 in mfgBIOS "apollolake_synobios" {fan_fail}<ffffffffa06575c0>
  [   37.687032] <redpill/bios_shim.c:207> Symbol #163 in mfgBIOS "apollolake_synobios" {fan_fail}<ffffffffa0657a20>
  [   42.673054] i915 0000:00:02.0: Failed to load DMC firmware i915/kbl_dmc_ver1_04.bin. Disabling runtime power management.

1968850318_2021-08-3021_00_26.thumb.png.cb4bce30668463eeb8f1d35ccabda172.png

These errors are perfectly normal. The one about syscall table relates to the kernel not exporting the symbol and RP having to manually scan the memory for it. Rest of the errors about USB and GPU are there because we're faking existence of a hardware which doesn't really exists, so drivers are trying to initialize it and realize it's not there (but the OS still thinks it's there - working or not ;)).

 

 

On 8/31/2021 at 6:13 AM, maxhartung said:

 

For Proxmox what is the recommended platform ? Also SATA vs USB boot ?

Q35 for platform. As for USB vs SATA it doesn't really matter. SATA is easier to configure as you can do it from the GUI, while virtual USB requires editing of the VM config from the terminal (the officially support that, just the GUI is not there).

 

 

On 8/31/2021 at 5:49 PM, maxhartung said:

 

I don't think so. I will try to install Proxmox again on my laptop and see if I can get this running. For Proxmox I think I need the SATA mappings. 

 

Proxmox:

I get file is corrupted...

 

I used these config

 

{

"extra_cmdline": {

"pid": "0x1666",

"vid": "0x0951",

"sn": "REMOVED",

"mac1": "0687940FE75F"

},

"synoinfo": {

"DiskIdxMap": "OC",

"SataPortMap": "1",

"SasIdxMap": "0"

},

"ramdisk_copy": {}

}

 

Logs: https://hastebin.com/ofehukocev.sql

Options you put in synoinfo belong in extra_cmdline. Besides, that if you're getting a file corrupted error on proxmox try shutting down the VM with "Stop", wait few second, start it again. If it still doesn't work you probably have a corrupted file (which we saw happening here a few times) - just regenerate the loader again.

 

 

On 8/31/2021 at 9:02 PM, taiziccf said:

why do you need all these SATA port mapping if you are using proxmox????

 


If you're running QEmu-based virtualization (e.g. Proxmox or VirtualBox) set vid to 0x46f4 and pid to 0x0001

 

image.thumb.png.c4f610a141bfc59f7828a31cd33fc4ba.png

Syno kernel doesn't enumerate disks starting from /dev/sda, so you first disk lands on like 7th position. If you don't care and you don't have a lot of disks you don't have to confiure the mapping. As for the vid/pid these are needed when you're booting from a virtual USB drive on Proxmox (and 0x46f4/ 0x0001 are just the ids which QEmu hardcoded in their source).

 

 

On 8/31/2021 at 10:24 PM, maxhartung said:

 

Just to clarify. These settings are for Proxmox, right ? For PID/VID should I use the one from my USB stick or these ones:

 

If you're running QEmu-based virtualization (e.g. Proxmox or VirtualBox) set vid to 0x46f4 and pid to 0x0001 ?

 

Curently, I am using the ones from USB like in the picture (attached)

Screenshot 2021-08-31 at 17.23.22.png

If you're using a real/physical USB drive you need to pass its parameters. In your case vid/pid is 0930/6545 (proxmox shows it in the dropdown).

 

 

On 9/1/2021 at 4:37 AM, tocinillo2 said:

I'm able to boot DS918+ with UEFI in my J4125 perfectly, detects it and I can start installation process without problem....but the install stop at 99% :( Any ideas?

 

Baremetal ;)

Isn't 99% a reboot? What happens after reboot?

 

 

On 9/1/2021 at 6:02 AM, jarugut said:

I'm creating a new bootloader for DS3615 6.2.4 with the last github content, for my sorprise I've received the following error.

 

image.thumb.png.c3fffb0f7bfceaba118481a4f56698e1.png

 

It seems like in the config.json file is configured to apply different patch but the original file not exist anymore.

Is this correct? I can copy  the original file from other old copy from my computer but I prefer to ask before do it!

 

 

Hm, files seems to be correct in the repos. You can try scrapping both pulls and re-pulling both repos. Previously we used symlinks which was a nightmare. It looks like you still have an old file with symlinks-based approach (and you stepped into an issue with them).

 

 

On 9/1/2021 at 7:23 PM, maxhartung said:

For one SATA Controller and one disk, there is some mapping to be done ?

Nope, it's just your drive will be shown in a semi-random place and not as a 1st one. The mapping really makes a difference only when you have like 8-9 drives and you go beyond the limit (since it will start with 1st hdd at like 7th position). The disk map is more of a "cherry on the top" so it looks properly in the UI.

 

 

On 9/1/2021 at 8:53 PM, gadreel said:

@jumkey do you know why the loader does not work if you update to 7.0.1 from DSM? It requires re-patching or something?

Yes, it requires a new kernel patch for new version as offsets change. We were tinkering with making it an automatic patcher but it's still a risk after every version. With the rate of changes in new DSM versions now we're not sure if developing an auto-patching in the preboot makes sense as likely with new version something else has to be updated aswell.

 

 

On 9/1/2021 at 9:07 PM, dodo-dk said:

@jumkey

I´m not a Windows User but thank you for your work.

How to generate the bsp file?

I have changed the config.json and changed to the "NEW" init patch. IMG Build works but with old BSP (41222) it reboots when it wants to load the Kernel.

DS3615 7.0.1-42214

You absolutely can never ever use old bsp files on a different version of the file. bsp file is like a diff but binary and using it on a different version causes random bytes to be patches in the kernel image. New bsp file can usually be generated by using php scripts from the tools folder. However, for normal use we don't expect non-developers to be doing this as you should blindly generate a bsp and call it a day - there are some checks to verify in a disassembler. The fact that a patch is generated doesn't mean it will work properly.

 

 

On 9/1/2021 at 9:26 PM, kennysino said:

could you pls post the loader image here?thanks

We will strongly advise to never post full loaders, especially not on the forum if we want it to exist. The loader image contains proprietary syno code and cannot be distributed in any shape or form without risking a copyright strike. This is why we developed a tool to build a loader as the tool doesn't contain any proprietary code and the kernel patches are for a GPL kernel so...

 

 

On 9/1/2021 at 11:06 PM, gadreel said:

If it was not clear what jumkey said, if you look into red-pill loader repo you will find inside the config folder that bsp file.

Currently the bsp file for 7.0.1 is not there. Either ThorGroup has to update the repo or build it yourself if you know how to...

As Jumkey mentioned in his post, everything it's in the ThorGroup repos how to do all that. 

The tools are all there ;) If you run the loader in patch mode without the bsp it will ask you to patch the kernel and rebuild everything. It will even leave the files so you can simply call bsdiff on two files and call it a day. There's a partial documentation for that in the load repo.
We didn't touch 7.0.1 as we were busy with the code but we don't expect any problems with 7.0.1 - it simply needs a new binary patch file.

 

On 9/2/2021 at 4:19 AM, jarugut said:

 

Hi after copy the patches files on the correct folders inside the last github repo and execute the original procedure to generate the bootloader I've received the following error.

 

Somebody do you knows the possible reason?

 

image.thumb.png.68faef73a4b26301790fd46a94a631df.png

Can you try with debug mode enabled? (add BRP_DEBUG=1 before the command so it becomes BRP_DEBUG=1 ./build_loader....). It will print much more information why it couldn't repack the ramdisk.

 

 

On 9/2/2021 at 11:27 AM, jforts said:

Okay, I managed to telnet into baremetal, notice that my SATA Controller is able to detect by RedPill.

The stuck at error 55% is due to there is no /dev/synoboot2. Anyone face this problem before? 

 

 

It doesn't see your bootloader source. Are you using USB or SATA? If it's USB you probably set a wrong VID/PID or connected the USB to a USB3 port (which most of the time will not work due to drivers issues, as 3.0 is a very complex spiderweb in comparison to 2.0).

 

 

On 9/2/2021 at 12:00 PM, jforts said:

I formatted another USB facing the same result. May I know how to make sure the USB is loaded correctly?

 

I also notice another error:


Jan  1 00:00:35 raidtool: disk/disk_port_to_container.c:38 Unknown diskType
Jan  1 00:00:35 raidtool: partition_path_enum.c:29 Failed to parsing device: name
Jan  1 00:00:35 raidtool: external/external_disk_port_check.c:158 expected sdx or /dev/sdx, not ram, not match any port type
Jan  1 00:00:35 raidtool: disk/disk_port_to_container.c:38 Unknown diskType

Is it my USB does not mapped??

That is actually unrelated. This is something which happens all the time, including on real DS hw. The "sdx" refers to like /dev/sdX or /dev/sd<anythingHere>, not the actual sdx drive. You can check the dmesg log. Do "dmesg | grep usb_boot_shim" and it will show you what it saw and if it was shimmed, or maybe the usb stick disconnected or something else.

 

 

On 9/2/2021 at 3:10 PM, pocopico said:

@jumkey @ThorGroup Could/should we add telnetd at the initrd patch to enable telnet by default for troubleshooting ? Normaly the install will start telnetd right after an installation failure but there are cases where in bare metal this will not happen. E.g. error with the sata mapping 

That should actually be automatic and built-in into syno scripting. If it doesn't work can you post an issue in the GH loader generator repo, so we can look at it? Maybe it changed in v7.

 

 

On 9/2/2021 at 3:33 PM, maxhartung said:

 

 

I have the same issue with 1 SATA port with EliteBook 840 G2 (PCIe/M2 slot disabled from BIOS, anyway didn't make a difference if enabled).  The mapping didn't help

 

{

"extra_cmdline": {

"pid": "0x1666",

"vid": "0x0951",

"sn": "XXXXXXXX",

"mac1": "fc3fdb87064f"

},

"synoinfo": {

"DiskIdxMap": "00",

"SataPortMap": "1",

"SasIdxMap": "0"

},

"ramdisk_copy": {}

}

 

USB drive:

 

Product ID: 0x1666

Vendor ID: 0x0951  (Kingston Technology Company)

The *Map parameters should be in extra_cmdline. We explained this in details in a response to below.

 

 

On 9/2/2021 at 10:29 PM, maxhartung said:

 

Yes, it does, it works out of the box, without any patches required. Just the PID/VID and is ready to go. There is any documentation regarding how mapping should be done ?

 

What each of the options means ? Someone stated here that should be inside extra_cmd_line section instead of synoinfo, what is the right way ?

 

I need 1 controller with 1 or 2 ports.

See
https://xpenology.com/forum/topic/44285-proxmox-hba-passthrough-second-virtual-controller-not-detected/

 

 

 

On 9/2/2021 at 10:39 PM, WiteWulf said:

Hi folks, I've been sitting and patiently keeping up with this thread since shortly after it started, keenly aware that this is in early development stages, not beta, not intended for use with production data and only for people who know how to compile, test and feedback, etc. 😀

 

I set up a build environment and successfully compiled everything a week or so ago, but didn't want to test it out yet as the vast majority of talk on here seems to be around virtualised installs, but I'm running bare metal on an HP Gen8 Microserver, upgraded with an Intel Xeon E3-1265L 2.50GHz 4-Core (Ivy Bridge).

 

Has anyone successfully used RedPill on a baremetal Microserver?

Is RedPill intending to support baremetal, or just virtualisation platforms?

 

If I need to migrate my install to esxi I'm gonna need a big HDD to back up all my data first 😬

If you're planning to migrate it's better to use Proxmox, ESPECIALLY on microserver. We actually have quite a few of these little cute boxes. If you want to migrate you can leave harddrives as-is and just boot the proxmox out of a SATA drive (you can use the ODD port, it should be on the side of the motherboard and IIRC it's blue and supports only 1.5Gb/s transfer speed). We will be playing with bare metal but it's currently about the drivers. However, gen 8 should work as the ethernet drivers are already included in the DSM.

That being said bare metal support is planned as well :)

 

 

On 9/2/2021 at 11:34 PM, jhoughten said:

The thing about the extra_cmd_line is that it just adds that stuff to the linux /zimage command that starts.

You can actually edit this without rebuilding the image.

When the GRUB screen comes up, you can hit the e key to edit the command.

I added DiskIdxMap=00 SataPortMap=1 SasIdxMap=0 to the end of the command - after syno_port_thaw=1

Yup, this is exactly how we test things. If you want to edit the loader and you're on linux you can do "losetup -pF image.img" and do a "mount /dev/loop0p1 /mnt" - then you have a normal grub config file in /mnt. The loader generator does the same thing really. Since it's in bash and with a lot of comments you can take a peek how it does things pretty easily.

 

 

On 9/3/2021 at 12:08 AM, WiteWulf said:

To be honest, I think it's time I bit the bullet and went virtual. I've been running xpenology for years now and have lost count of the number of problems I've had caused by running baremetal :D

It's really just the time involved in backing everything up, rebuilding the machine, then restoring data that's holding me back. It's just so much simpler to do an in-place upgrade when it works, and not have to reconfigure all the services (mainly Plex) that are running on it.

Some of our folks needed a LOT of convincing to go VM... some are still grumpy about that. But the technology went really far and virtualization nowadays is way less problematic than bare metal, while also being very efficient in terms of performance. But as we said before ESXi is probably not the greatest choice - Proxmox works much better. If you have any questions regarding it on microserver feel free to ping us: these are amazing machines to this day. Some of ours run with 10 disks :D [for uninitated: by default they have 4 slots]

 

 

On 9/3/2021 at 1:21 AM, Aigor said:

I'v lost some updates, there is some recap to know "state of the art" of bootloader? 
Thanks 

Hm, we don't have a centralized place with list of things which are working/not. However, if something isn't an issue on GH it's probably working as we try to add issue we find there as well. Currently, after today's updates [one of my colleagues is writting that post as I'm answering questsions], we can say it's a beta-which-we-don't-call-a-beta :D

 

 

On 9/3/2021 at 1:24 AM, pigr8 said:

ok figured out, but the disks connected to the mpt2sas are detected as sas -_- so dsm refuses to use them 😕

 


DiskStation> insmod /usr/lib/modules/mpt2sas.ko
[   36.102592] mpt2sas version 20.00.00.00 loaded
[   36.104746] scsi0 : Fusion MPT SAS Host
[   36.106132] I/O scheduler elevator not found
[   36.109234] mpt2sas0: 64 BIT PCI BUS DMA ADDRESSING SUPPORTED, total mem (4022016 kB)
[   36.183709] mpt2sas0: MSI-X vectors supported: 1, no of cores: 4, max_msix_vectors: -1
[   36.188214] mpt2sas 0000:13:00.0: irq 73 for MSI/MSI-X
[   36.195944] mpt2sas0-msix0: PCI-MSI-X enabled: IRQ 73
[   36.197702] mpt2sas0: iomem(0x00000000fd3f0000), mapped(0xffffc90008540000), size(65536)
[   36.200643] mpt2sas0: ioport(0x0000000000006000), size(256)
[   36.324504] mpt2sas0: Allocated physical memory: size(7445 kB)
[   36.326292] mpt2sas0: Current Controller Queue Depth(3305), Max Controller Queue Depth(3432)
[   36.329512] mpt2sas0: Scatter Gather Elements per IO(128)
[   36.390490] mpt2sas0: LSISAS2008: FWVersion(16.00.00.00), ChipRevision(0x03), BiosVersion(00.00.00.00)
[   36.395798] mpt2sas0: Dell 6Gbps SAS HBA: Vendor(0x1000), Device(0x0072), SSVID(0x1028), SSDID(0x1F1C)
[   36.399919] mpt2sas0: Protocol=(Initiator,Target), Capabilities=(TLR,EEDP,Snapshot Buffer,Diag Trace Buffer,Task Set Full,NCQ)
[   36.405305] mpt2sas0: sending port enable !!
[   36.411397] mpt2sas0: host_add: handle(0x0001), sas_addr(0x590b11c027f7ff00), phys(8)
[   36.417293] mpt2sas0: detecting: handle(0x000d), sas_address(0x4433221104000000), phy(4)
[   36.419977] mpt2sas0: REPORT_LUNS: handle(0x000d), retries(0)
[   36.422124] mpt2sas0: TEST_UNIT_READY: handle(0x000d), lun(0)
[   36.425202] I/O scheduler elevator not found
[   36.427956] I/O scheduler elevator not found
[   36.431578] scsi 0:0:0:0: Direct-Access     WDC      WD20PURZ-85GU6Y0         0A80 PQ: 0 ANSI: 6
[   36.435117] scsi 0:0:0:0: SATA: handle(0x000d), sas_addr(0x4433221104000000), phy(4), device_name(0x50014ee20f84b83c)
[   36.438294] scsi 0:0:0:0: SATA: enclosure_logical_id(0x590b11c027f7ff00), slot(7)
[   36.441957] scsi 0:0:0:0: atapi(n), ncq(y), asyn_notify(n), smart(y), fua(y), sw_preserve(y)
[   36.445386] scsi 0:0:0:0: serial_number(     WD-WCC4M6XXDE3R)
[   36.447318] scsi 0:0:0:0: qdepth(32), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1)
[   36.450226] <redpill/sata_boot_shim.c:305> Probing SCSI device using sd_probe_shim
[   36.453332] <redpill/sata_boot_shim.c:307> sd_probe_shim: new SCSI device connected - it's not a SATA disk, ignoring
[   36.456873] <redpill/sata_boot_shim.c:321> Handing over probing from sd_probe_shim to sd_probe
[   36.459910] sd 0:0:0:0: Attached scsi generic sg0 type 0
[   36.463739] sd 0:0:0:0: [sas1] 3907029168 512-byte logical blocks: (2.00 TB/1.81 TiB)
[   36.466779] sd 0:0:0:0: [sas1] 4096-byte physical blocks
[   36.474036] sd 0:0:0:0: [sas1] Write Protect is off
[   36.474739] mpt2sas0: port enable: SUCCESS
[   36.477888] sd 0:0:0:0: [sas1] Mode Sense: 7f 00 10 08
DiskStation> [   36.482099] sd 0:0:0:0: [sas1] Write cache: enabled, read cache: enabled, supports DPO and FUA
[   36.505347]  sas1: p1 p2 p3 p4
[   36.515222] sd 0:0:0:0: [sas1] Attached SCSI disk

 

It will be a problem with booting from SAS but isn't DSM seeing them as data disks? Which LSI card are you running? Can you put an issue on GH in the kernel module repo for this, so we don't forget?

 

 

On 9/3/2021 at 1:26 AM, Dreadnought said:

 

or Parallels, what I am using.

Is Parallels also lacking SMART? It should work with the new release from today. We didn't test specifically on Parallels but the code we wrote isn't specific to ESXi (and it even works with a IDE => SATA adapter which we tested for fun).

 

 

On 9/3/2021 at 7:21 AM, maxhartung said:

 

Thanks a lot for these tips. Unfortunately still not working:

 

Tried: DiskIdxMap=00 SataPortMap=4  -> We have detected errors on the hard drives 1,3,4

Tried: DiskIdxMap=00 SataPortMap=3  -> We have detected errors on the hard drives 1,3

Tried: DiskIdxMap=00 SataPortMap=2 -> We have detected errors on the hard drives 1

Tried: DiskIdxMap=00 SataPortMap=1 -> No drives detected

Tried: DiskIdxMap=00 SataPortMap=0 -> not loading/no network

 

Tried: DiskIdxMap=00 SataPortMap=1 SasIdxMap=0 -> No drives detected

Tried: DiskIdxMap=00 SataPortMap=2 SasIdxMap=0 -> We have detected errors on the hard drives 1

 

Tried: DiskIdxMap=01 SataPortMap=1 -> No drives detected

Tried: DiskIdxMap=01 SataPortMap=2 -> We have detected errors on the hard drives 2

Do you have a serial console by any chance? If so can you post the full dmesg output?

 

 

On 9/3/2021 at 4:06 PM, Aigor said:

Do this method works with other drivers?  I'm thinking to network card's driver other then intel 
 

Yes, you can add any driver like that really. You may need to modify the patch for init file to do an insmod. It's not a solution for a long-term and easy-to-use loader but if we use it to add an ethernet driver as well.

 

 

On 9/3/2021 at 4:10 PM, sebg35 said:

Preview For messages :

 


Sep  3 07:48:26 raidtool: external/external_disk_port_check.c:158 expected sdx or /dev/sdx, not ram, not match any port type
[...]
Sep  3 07:48:25 synodiskport: external/external_disk_port_check.c:158 expected sdx or /dev/sdx, not md, not match any port type
Sep  3 07:48:26 raidtool: space_error_log.c:47 SpaceCommand:command="/sbin/mdadm --zero-superblock /dev/sda1" Error=""
Sep  3 07:48:26 raidtool: raid_internal_lib.c:90 [Info] Clean RAID superblock on '/dev/sda1'
Sep  3 07:48:26 kernel: [  727.226044] md1: detected capacity change from 2147418112 to 0
Sep  3 07:48:26 kernel: [  727.226759] md: md1: set sda2 to auto_remap [0]
Sep  3 07:48:26 kernel: [  727.227301] md: md1 stopped.
Sep  3 07:48:26 kernel: [  727.227656] md: unbind<sda2
[...]
Sep  3 07:57:25 updater: updater.c:3211 orgBuildNumber = 41890, newBuildNumber=41890
Sep  3 07:57:25 updater: util/updater_util.cpp:86 fail to read company in /tmpRoot//etc.defaults/synoinfo.conf
Sep  3 07:57:25 updater: updater.c:7029 ==== Start flash update ====
Sep  3 07:57:25 updater: updater.c:7033 This is X86 platform
Sep  3 07:57:25 updater: boot/boot_lock.c(228): failed to mount boot device /dev/synoboot2 /tmp/bootmnt (errno:2)
Sep  3 07:57:25 updater: updater.c:6494 Failed to mount boot partition
Sep  3 07:57:25 updater: updater.c:3118 No need to reset reason for v.41890
Sep  3 07:57:25 updater: updater.c:7636 Failed to accomplish the update! (errno = 21)
Sep  3 07:57:25 install.cgi: ninstaller.c:1546 Executing [/tmpData/upd@te/updater -v /tmpData > /dev/null 2>&1] error[21]
Sep  3 07:57:27 install.cgi: ninstaller.c:123 Mount partion /dev/md0 /tmpRoot
Sep  3 07:57:27 install.cgi: ninstaller.c:1515 Moving updater for configuration upgrade...cmd=[/bin/mv -f /tmpData/upd@te/updater /tmpRoot/.updater > /dev/null 2>&1]
Sep  3 07:57:27 install.cgi: ninstaller.c:152 umount partition /tmpRoot
Sep  3 07:57:27 install.cgi: ErrFHOSTCleanPatchDirFile: After updating /tmpData/upd@te...cmd=[/bin/rm -rf /tmpData/upd@te > /dev/null 2>&1]
Sep  3 07:57:27 install.cgi: ErrFHOSTCleanPatchDirFile: Remove /tmpData/upd@te.pat...
Sep  3 07:57:27 install.cgi: ErrFHOSTDoUpgrade(1794): child process failed, retv=-21
Sep  3 07:57:27 install.cgi: ninstaller.c:1811(ErrFHOSTDoUpgrade) err=[-1]
Sep  3 07:57:27 install.cgi: ninstaller.c:1815(ErrFHOSTDoUpgrade) retv=[-21]
Sep  3 07:57:27 install.cgi: install.c:409 Upgrade by the manual patch fail.
Sep  3 07:57:27 install.cgi: install.c:678 Upgrade by the uploaded patch /tmpData/@autoupdate/upload.pat fail.
Jan  1 00:00:00 install.cgi: ninstaller.c:152 umount partition /tmpData

 

Your problem lies in lack of /dev/synoboot. Previously if you were using ESXi and you had to use a physical USB flash drive. Since this release you can experiment with SATA boot. Your log shows you've booted with USB boot but you have no usb drives in your system.

 

 

On 9/3/2021 at 11:00 PM, WiteWulf said:

The problem is that when the Gen8 boots it completely ignores the USB stick the redpill image on it and attempts to netboot. [...] I just can't figure out why the Gen8 is refusing to boot from it.

Try booting from the port INSIDE of the microserver. Some of the ports on the microserver are finicky for booting as HPE BIOS is strange at times.

 

 

On 9/4/2021 at 12:44 AM, WiteWulf said:

*edit* this is an interesting thread on the topic, although few years (and ESXi revisions) old now:

https://homeservershow.com/forums/topic/14778-esxi-65-ahci-bad-write-performance/

ESXi doens't play well with older hardware. Proxmox will be a better choice here ;)

 

 

On 9/4/2021 at 4:06 AM, haydibe said:

 

Oh boy🙄 This will leave plenty of fun for LSI9211 owner that want to migrate from (jun's) 6.2.3 to (redpill) 7.0. Must be a behavior of the mpt2sas driver itself - almost looks like they change from /dev/sd? to /dev/sas? at one point and the new drivers happen to be created from sources after that change? Afaik Bromolow DSM6.2.3 uses 3.10.105. and the change logs of https://mirrors.edge.kernel.org/pub/linux/kernel/v3.x/ChangeLog-3.10.106https://mirrors.edge.kernel.org/pub/linux/kernel/v3.x/ChangeLog-3.10.107 or https://mirrors.edge.kernel.org/pub/linux/kernel/v3.x/ChangeLog-3.10.108 (which is what the bromolow DSM7.0 kernel sources base on) indicate nothing that would point to such a change (or anything helpfull).. But what caused the different detection then?

 

I expect lvm/mdam raids to identify the drives belong to an array by device path (/dev/sd?) and not by the physical controller they are attached to - thus a switch from /dev/sd? to /dev/sas? will result in a broken array?! I hope I realy got this wrong and some sort of magic that relies on disk-uuid, serialnumber or whatever is used to self-heal the situation.

They (syno) heavily modify the drivers and the kernel. The array will be fine if the DSM will be able to recognize the SAS devices as the arrays are recognized and built using metadata on disks itself. Where the devices are doesn't matter for the raid subsystem, only the DSM tools care. On a normal Linux box you can have the drive be under /dev/test123 and it will work ;) This is why you should never clone 1:1 disk with arrays on the same system which mounts them.

 

 

 

-------

 

to be continued in the next post due to forum limits

 

 

  • Like 4
  • Thanks 5
Link to post
Share on other sites

Continuation of questions starting from mid-page 37 (forum doesn't allow for more than 50 quotes per post):

 

 

On 9/4/2021 at 7:27 AM, jumkey said:

change CONFIG_SYNO_SAS_DEVICE_PREFIX="sas" to CONFIG_SYNO_SAS_DEVICE_PREFIX="sd"? if you want?


root@vultr:~/build/linux-4.4.x-apollolake-25426# grep -r 'SYNO_SAS_DEVICE_PREFIX' ./
./synoconfigs/broadwellntbap:CONFIG_SYNO_SAS_DEVICE_PREFIX="sas"
./synoconfigs/Kconfig.devices:config SYNO_SAS_DEVICE_PREFIX
./synoconfigs/broadwellnk:CONFIG_SYNO_SAS_DEVICE_PREFIX="sas"
./synoconfigs/broadwellntb:CONFIG_SYNO_SAS_DEVICE_PREFIX="sas"
./synoconfigs/coffeelake:CONFIG_SYNO_SAS_DEVICE_PREFIX="sas"
./synoconfigs/purley:CONFIG_SYNO_SAS_DEVICE_PREFIX="sas"
./drivers/scsi/sd.c:                    error = syno_sd_format_numeric_disk_name(CONFIG_SYNO_SAS_DEVICE_PREFIX, synoidx, gd->disk_name, DISK_NAME_LEN);
./drivers/scsi/sd.c:                    error = syno_sd_format_numeric_disk_name(CONFIG_SYNO_SAS_DEVICE_PREFIX, synoidx, gd->disk_name, DISK_NAME_LEN);
root@vultr:~/build/linux-4.4.x-apollolake-25426#

You cannot change that - that will require rebuilding of the kernel which isn't possible (as syno doesn't share compilable sources of the kernel).

 

 

On 9/4/2021 at 4:39 PM, pocopico said:


what if you create links of the eg /dev/sas1 to /dev/sda with # ln -s /dev/sas1 /dev/sda

You theoretically can use mknod but this causes problems with v7 (synoboot used to be done this way on 918+). Stay tuned: we will figure this out ;)

 

 

On 9/4/2021 at 10:27 PM, haydibe said:

So it is not just the drive order that is irrelevant, but it's also the device path (and thus the controller type) that is irrevant. 

I was aware of the first part, but not of the second.

 

Good to know :)

Yeah, Linux and md doesn't care - only the DSM has some hardcoded assumptions for paths which we need to emulate ;)

 

 

On 9/4/2021 at 10:46 PM, pkdick1 said:

 

Hello, thank you very much for this file. May I ask you a question ?: I have myself an Asrock J4150-ITX, do you think that using you image, I will be able to migrate from DSM 6.23 to DSM 7.01 ? I also thought that it was also necessary to put a legit serial nember to make the baremetal XPENOLOGY device working: could you give me your point about this ?

 

I admit that I start to long a bit about this wonderful development, so I am close to take the risk to use you image...

 

Thank you in advance,

You don't need a real S/N to run it. Some services (e.g. faces detection in photos) need it but rest of the things don't. What's important is not using the same S/N for multiple people because sooner than later they will start blacklisting them.

 

 

On 9/4/2021 at 10:59 PM, maxhartung said:

In order to work on ESXI what changes are required ? 

Today's update should make ESXi working. [will be posted on GH after we finish writing all of the posts]

 

 

On 9/5/2021 at 12:12 AM, Drones said:

I'm trying this solution, but no luck.

How i can change PID&VID on ready-made image (post tocinillo2)?

I think my problem is with wrong PID&VID.

Do not use ready-made images. You don't know what's in them nor we can help with those. Additionally, posting them here is a substantial risk as, unlike the generator, full images contain copyrighted materials which cannot be distributed legally. We will not go into details but the RP project started in a different thread... with different repos... by a different author... none of that is accessible anymore. Add 2 and 2 yourself.

 

 

On 9/5/2021 at 12:14 AM, haydibe said:

Incorporated the addition from @taiziccf for chchia's  apollolake-7.0.1-42214, and added @jumkey's bromolow-7.0.1-42214 from his develop branch.

 

You guys are moving at an incredible pace :)

 

 

 

redpill-tool-chain_x86_64_v0.6.1.zip 7.47 kB · 133 downloads

We only see a one big problem here - an EFI is being added on the side but nobody opens a PR for it. Instead, there are random zips flying around. Nothing against you (your help with docker is amazing!). However, soon we will not be able to help with issues as some of them stem from people running a modified version grabbed from an attachment...

 

 

On 9/5/2021 at 4:27 AM, mcdull said:

After using the 7.0.1RC (918) on proxmox, I got quite a lot of call trace in log, and the system is very slow.  Anyone experiencing the same?


[ 2160.666146] INFO: task kworker/u8:4:14489 blocked for more than 120 seconds.
[ 2160.668620]       Tainted: P           OE   4.4.180+ #42214
[ 2160.670135] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 2160.671611] kworker/u8:4    D ffff8802613a3850     0 14489      2 0x00000000
[ 2160.672895] Workqueue: writeback wb_workfn (flush-btrfs-2)
[ 2160.674311]  ffff8802613a3850 00000000812c380a ffffffff818114c0 ffff8800367b6600
[ 2160.675579]  0000000000000000 ffff88027dc16300 7fffffffffffffff ffffffff81576880
[ 2160.676493]  ffff8802613a3970 ffff8802613a3860 ffffffff81576126 ffff8802613a38d8
[ 2160.677916] Call Trace:
[ 2160.678311]  [<ffffffff81576880>] ? bit_wait+0x60/0x60
[ 2160.679061]  [<ffffffff81576126>] schedule+0x26/0x70
[ 2160.679679]  [<ffffffff81578cfe>] schedule_timeout+0x16e/0x280
[ 2160.680306]  [<ffffffff810b836a>] ? ktime_get+0x3a/0xa0
[ 2160.682031]  [<ffffffff81576880>] ? bit_wait+0x60/0x60
[ 2160.682702]  [<ffffffff81575871>] io_schedule_timeout+0xa1/0x110
[ 2160.683374]  [<ffffffff81576896>] bit_wait_io+0x16/0x60
[ 2160.683942]  [<ffffffff8157663e>] __wait_on_bit_lock+0x4e/0xd0
[ 2160.684637]  [<ffffffff8113124b>] __lock_page+0xab/0xb0
[ 2160.685314]  [<ffffffff8108e850>] ? prepare_to_wait_event+0x100/0x100
[ 2160.685922]  [<ffffffffa08c150d>] extent_write_cache_pages.isra.46.constprop.68+0x34d/0x480 [btrfs]
[ 2160.686800]  [<ffffffff8108e351>] ? __wake_up+0x41/0x50
[ 2160.687517]  [<ffffffff81082c15>] ? update_curr+0xa5/0x130
[ 2160.688139]  [<ffffffffa08c28bb>] extent_writepages+0x4b/0x60 [btrfs]
[ 2160.688829]  [<ffffffffa08a00b0>] ? btrfs_submit_direct+0x940/0x940 [btrfs]
[ 2160.689620]  [<ffffffffa089c8b6>] btrfs_writepages+0x26/0x30 [btrfs]
[ 2160.690432]  [<ffffffff8113ea7b>] do_writepages+0x2b/0x80
[ 2160.690986]  [<ffffffffa08bd876>] ? clear_state_bit+0x156/0x1e0 [btrfs]
[ 2160.691691]  [<ffffffff811c34aa>] __writeback_single_inode+0x4a/0x380
[ 2160.692445]  [<ffffffff811c3c69>] writeback_sb_inodes+0x1b9/0x530
[ 2160.693183]  [<ffffffff811c4044>] __writeback_inodes_wb+0x64/0xb0
[ 2160.693901]  [<ffffffff811c434a>] wb_writeback+0x22a/0x310
[ 2160.694571]  [<ffffffff811c49f2>] wb_workfn+0x162/0x360
[ 2160.695197]  [<ffffffff81073ceb>] worker_run_work+0x9b/0xe0
[ 2160.695860]  [<ffffffff811c4890>] ? inode_wait_for_writeback+0x30/0x30
[ 2160.696723]  [<ffffffff8106b2e3>] process_one_work+0x1e3/0x4f0
[ 2160.697559]  [<ffffffff8106b61e>] worker_thread+0x2e/0x4b0
[ 2160.698251]  [<ffffffff8106b5f0>] ? process_one_work+0x4f0/0x4f0
[ 2160.698850]  [<ffffffff810700f5>] kthread+0xd5/0xf0
[ 2160.699369]  [<ffffffff81070020>] ? kthread_worker_fn+0x160/0x160
[ 2160.699981]  [<ffffffff81579fef>] ret_from_fork+0x3f/0x80
[ 2160.700547]  [<ffffffff81070020>] ? kthread_worker_fn+0x160/0x160
[ 2160.701382] INFO: task SYNO.API.Auth.T:16585 blocked for more than 120 seconds.
[ 2160.702204]       Tainted: P           OE   4.4.180+ #42214
[ 2160.702807] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 2160.703679] SYNO.API.Auth.T D ffff880235643c50     0 16585  10537 0x00000000
[ 2160.704476]  ffff880235643c50 000000008182ed60 ffff880274970000 ffff8802643b8cc0
[ 2160.705284]  ffff88025e54d760 ffff8802643b8cc0 ffff88025e54d764 00000000ffffffff
[ 2160.706104]  ffff88025e54d768 ffff880235643c60 ffffffff81576126 ffff880235643c70
[ 2160.706929] Call Trace:
[ 2160.707196]  [<ffffffff81576126>] schedule+0x26/0x70
[ 2160.707682]  [<ffffffff815763b9>] schedule_preempt_disabled+0x9/0x10
[ 2160.708367]  [<ffffffff81577b6c>] __mutex_lock_slowpath+0x8c/0x100
[ 2160.709101]  [<ffffffff8119add9>] ? lookup_fast+0xc9/0x320
[ 2160.709633]  [<ffffffff81577bf2>] mutex_lock+0x12/0x30
[ 2160.710157]  [<ffffffff8119c8fb>] walk_component+0x21b/0x330
[ 2160.710773]  [<ffffffff8119db24>] path_lookupat+0xb4/0x230
[ 2160.711384]  [<ffffffff811a2d0a>] filename_lookup+0x9a/0x100
[ 2160.711974]  [<ffffffff8117ca4d>] ? kmem_cache_alloc_trace+0x13d/0x150
[ 2160.712665]  [<ffffffff811a2a23>] ? getname_flags+0x53/0x190
[ 2160.713253]  [<ffffffff811a2e23>] user_path_at_empty+0x33/0x40
[ 2160.713815]  [<ffffffff8118b0ff>] SyS_access+0x8f/0x2a0
[ 2160.714307]  [<ffffffff81579c4a>] entry_SYSCALL_64_fastpath+0x1e/0x8e
[ 2160.714866] INFO: task synoappnotify:16640 blocked for more than 120 seconds.
[ 2160.715496]       Tainted: P           OE   4.4.180+ #42214
[ 2160.715977] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 2160.716882] synoappnotify   D ffff880035693c60     0 16640      1 0x00000004
[ 2160.717741]  ffff880035693c60 000000008119a7e5 ffffffff818114c0 ffff8802732ccc80
[ 2160.718576]  ffff88025e54d760 ffff8802732ccc80 ffff88025e54d764 00000000ffffffff
[ 2160.719414]  ffff88025e54d768 ffff880035693c70 ffffffff81576126 ffff880035693c80
[ 2160.720268] Call Trace:
[ 2160.720550]  [<ffffffff81576126>] schedule+0x26/0x70
[ 2160.721168]  [<ffffffff815763b9>] schedule_preempt_disabled+0x9/0x10
[ 2160.721904]  [<ffffffff81577b6c>] __mutex_lock_slowpath+0x8c/0x100
[ 2160.722627]  [<ffffffff81577bf2>] mutex_lock+0x12/0x30
[ 2160.723246]  [<ffffffff811a0614>] path_openat+0x444/0x1a40
[ 2160.723882]  [<ffffffff811e458a>] ? flock_lock_inode+0xda/0x280
[ 2160.724579]  [<ffffffff811e47ee>] ? locks_remove_flock+0xbe/0xc0
[ 2160.725298]  [<ffffffff811a390e>] do_filp_open+0x7e/0xc0
[ 2160.725897]  [<ffffffff811ab2b1>] ? dput.part.25+0x91/0x200
[ 2160.726549]  [<ffffffff811b1c6b>] ? __alloc_fd+0x3b/0x170
[ 2160.727187]  [<ffffffff81189b4f>] do_sys_open+0x1af/0x240
[ 2160.727715]  [<ffffffff8118bf3f>] SyS_openat+0xf/0x20
[ 2160.728242]  [<ffffffff81579c4a>] entry_SYSCALL_64_fastpath+0x1e/0x8e
[ 2160.728924] INFO: task SYNO.API.Auth.T:19202 blocked for more than 120 seconds.
[ 2160.729745]       Tainted: P           OE   4.4.180+ #42214
[ 2160.730348] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 2160.731201] SYNO.API.Auth.T D ffff880231103c50     0 19202  10537 0x00000000
[ 2160.731980]  ffff880231103c50 000000008182ed60 ffffffff818114c0 ffff88026c4f3300
[ 2160.732827]  ffff88025e54d760 ffff88026c4f3300 ffff88025e54d764 00000000ffffffff
[ 2160.733753]  ffff88025e54d768 ffff880231103c60 ffffffff81576126 ffff880231103c70
[ 2160.734605] Call Trace:
[ 2160.734819]  [<ffffffff81576126>] schedule+0x26/0x70
[ 2160.735273]  [<ffffffff815763b9>] schedule_preempt_disabled+0x9/0x10
[ 2160.735812]  [<ffffffff81577b6c>] __mutex_lock_slowpath+0x8c/0x100
[ 2160.736580]  [<ffffffff8119add9>] ? lookup_fast+0xc9/0x320
[ 2160.737218]  [<ffffffff81577bf2>] mutex_lock+0x12/0x30
[ 2160.737791]  [<ffffffff8119c8fb>] walk_component+0x21b/0x330
[ 2160.738406]  [<ffffffff8119db24>] path_lookupat+0xb4/0x230
[ 2160.738980]  [<ffffffff811a2d0a>] filename_lookup+0x9a/0x100
[ 2160.739615]  [<ffffffff8117ca4d>] ? kmem_cache_alloc_trace+0x13d/0x150
[ 2160.740399]  [<ffffffff811a2a23>] ? getname_flags+0x53/0x190
[ 2160.741083]  [<ffffffff811a2e23>] user_path_at_empty+0x33/0x40
[ 2160.741734]  [<ffffffff8118b0ff>] SyS_access+0x8f/0x2a0
[ 2160.742330]  [<ffffffff81579c4a>] entry_SYSCALL_64_fastpath+0x1e/0x8e
[ 2180.147079] <redpill/pmu_shim.c:324> Got 1 bytes from PMU: reason=1 hex={2d} ascii="-"
[ 2180.297936] <redpill/pmu_shim.c:324> Got 1 bytes from PMU: reason=1 hex={75} ascii="u"
[ 2180.298917] <redpill/pmu_shim.c:253> Executing cmd OUT_FAN_HEALTH_ON handler cmd_shim_noop+0x0/0x2d [redpill]
[ 2180.298917] <redpill/pmu_shim.c:42> vPMU received OUT_FAN_HEALTH_ON using 1 bytes - NOOP

 

Do you have details how you triggered that error? We didn't look into 7.0.1 yet.

 

 

On 9/5/2021 at 5:44 AM, matyyy said:

This is my first post on the forum so it would be appropriate to say hello! Hello XPenlogy!

Great job RedPill - I was able to get DSM 7.0 running on an HP Microserver Gen 8 in EXSI - the problem was only with the disk but the solution from https://github.com/RedPill-TTG/redpill-lkm/issues/14 helped. The only thing left is the pesky message that the disk needs attention, does anyone have a solution for that?

It should be working with the new release today ;) [will be posted on GH soon after we finish all posts]

 

 

On 9/5/2021 at 10:40 AM, volitation said:

How does DS918+ support SATA boot like DS3615xs?

It does now - today's update adds this.

 

 

On 9/5/2021 at 3:18 PM, progressives said:

@haydibe what is error msg?

WARNING: Symbol version dump /opt/redpill-lkm/kernel-bromolow-6.2.4-25556/Module.symvers
           is missing; modules will have no dependencies and modversions.

That is perfectly normal when compiling modules against kernel sources which weren't compiled in that directory. In essence, it means that the kernel cannot determine dependencies between multiple modules (which is important for e.g. distributing a driver for Ubuntu, but here it doesn't matter at all).

 

 

On 9/5/2021 at 4:37 PM, titoum said:

i started the latest docker and great works as usual :-)

 

wondering why i get this warning:

 


/opt/redpill-lkm/config/runtime_config.c:168:47: warning: passing argument 2 of 'validate_nets' discards 'const' qualifier from pointer target type
     valid &= validate_nets(config->netif_num, config->macs);
                                               ^
/opt/redpill-lkm/config/runtime_config.c:80:20: note: expected 'char (**)[13]' but argument is of type 'char (* const*)[13]'
 static inline bool validate_nets(const unsigned short if_num, mac_address *macs[MAX_NET_IFACES])
                    ^

 

for the following user_config (obfuscated value of course but i kept the length of it :-) )

 


Using user_config.json:
{
  "extra_cmdline": {
    "pid": "XXXXXX",
    "vid": "XXXXXX",
    "sn": "-",
    "mac1": "000000000000",
    "mac2": "000000000000"
  },
  "synoinfo": {},
  "ramdisk_copy": {}
}

 

Fixed in today's release - that warning is actually an old GCC artifact but we worked around it with a slightly different syntax.

 

 

On 9/5/2021 at 8:22 PM, Rikk said:

Hello to all,

   I am trying to test the loader on my current hardware and I am stuck to a stupid step : my system didn't find the USB key to boot on it. With the previous setup (Jun's loader with 3615xs model, 6.2.3 DSM) it was working well.

 

My hardware is :

Gen8 HP microserver with Xeon E3-1220L CPU update

6Gb of RAM

 

I am using le last env (redpill-tool-chain_x86_64_v0.7.2) to build the image with MAC OS big sur platform., and docker installed.

 

My bromolow_user_config.json:

content is (xx and yy are replacing real values and pid/vid are the one from my USB key😞

{
  "extra_cmdline": {
    "pid": "0xFFF7",
    "vid": "0x203A",
    "sn": "yyyyyyyyyyyyy",
    "mac1": "xxxxxxxxxxxx"
  },
  "synoinfo": {},
  "ramdisk_copy": {}
}

 

Commands to build the image:

cd ~/Desktop/redpill-tool-chain_x86_64_v0.7.2
sudo chmod 777 redpill_tool_chain.sh

sh ./redpill_tool_chain.sh build bromolow-7.0.1-42214
sh ./redpill_tool_chain.sh auto bromolow-7.0.1-42214
 

The image is written on USB key with BalenaEtcher

 

Did I am missing something ?

 

Help will be appreciate.

 

Thanks !

 

Rikk

 

 

Try the port INSIDE of the case. HPE bios on microserver is strange sometimes.

 

 

On 9/5/2021 at 10:35 PM, pkdick1 said:

One last question: I need to updated the PID/VID of my USB stick in order to be recognized during install. What about the serial number and the mac address you put in you grub.cfg file ? Are they mandatory for the install ? (I know that the serial number is necessary for DS Video to use hardward transcoding, but what about the mac address?).

 

Thank you very much for your answers.

S/N must be unique (can be generated but must be unique); MAC is needed for DHCP. You should not edit the grub config manually.

 

 

On 9/6/2021 at 4:19 AM, maxhartung said:

My Proxmox install seem broken for some reason:

 

I uploaded the img file to via web browser (local home -> ISO Images)

 

/var/lib/vz/template/iso/image.img

 

But when I try to start the VM:

 

kvm:  -device qemu-xhci,addr=0x18 -drive id=synoboot,file=/var/lib/vz/template/iso/image.img,if=none,format=raw -device usb-storage,id=synoboot,drive=synoboot,bootindex=5:

 

Could not open ' -device qemu-xhci,addr=0x18 -drive id=synoboot,file=/var/lib/vz/template/iso/image.img,if=none,format=raw -device usb-storage,id=synoboot,drive=synoboot,bootindex=5': No such file or directory

 

I checked the file via shell and it's there.

 

This is the args in the conf:

 

args: -device 'qemu-xhci,addr=0x18' -drive 'id=synoboot,file=/var/lib/vz/template/iso/image.img,if=none,format=raw' -device 'usb-storage,id=synoboot,drive=synoboot,bootindex=5'

 

root@home:~# ls /var/lib/vz/template/iso/
image.img
root@home:~# 

 

Config:

 

args: -device 'qemu-xhci,addr=0x18' -drive 'id=synoboot,file=/var/lib/vz/template/iso/image.img,if=none,format=raw' -device 'usb-storage,id=synoboot,drive=synoboot,bootindex=5'

balloon: 0
boot: order=sata0;net0
cores: 2
machine: q35
memory: 4096
name: DSM
net0: virtio=F2:9D:7D:2E:7E:61,bridge=vmbr0
numa: 0
ostype: l26
sata0: local-lvm:vm-100-disk-0,size=120G
scsihw: virtio-scsi-pci
smbios1: uuid=27d90b08-fa4c-4b43-85fc-f40200747f72
sockets: 1
vmgenid: 6e85c112-295e-4432-b661-7f91af624a80

You need to put it as a disk image, not an ISO image. These are quite different - that's one. Second you have a syntax error somewhewre but the forum replaced bunch of characters. You need to post in a code block tag so the forum doesn't replace anything.

 

 

On 9/6/2021 at 4:53 PM, mcdull said:

I tried 3615 image in my old qnap.  But it does not boot properly.

In GRUB> whenever I press enter, it will reboot by itself.

 

What CPU do you have? If it just hard-reboots it means the CPU just crashed, which usually means that it doesn't support some instructions.

 

 

On 9/6/2021 at 7:39 PM, WiteWulf said:

FYI, if you need to set the partition as active on your USB stick and you're using macOS, it's non-obvious how to do this. Here's a howto:

 

https://hackintoshmumbai.com/how-to-set-up-your-partition-active-using-fdisk-in-mac-os-x/

If you dd the raw image (dd if=file.img of=/dev/rdiskX => make sure you sure rdisk and not disk) you don't need to manually modify the partition. Before you do that you should check the disk with diskutil list to get the number and unmount all existing partitions using diskutil unmount (not eject from the Finder).

 

 

On 9/6/2021 at 10:26 PM, pocopico said:

To extract : 

 

mkdir rd

cd rd

cat ../rd.gz | cpio -idm
 

To repack the rd, in rd directory :

 

find | cpio -o -H newc > ../rd.gz.new

 

 

This is actually posted in the research repo. However, we strongly advise against manually tinkering with the rd.gz file - the loader generator can copy files where needed.

 

 

On 9/6/2021 at 10:34 PM, pocopico said:

 

Not very difficult .. 

 

In order to compile your modules for 3.10.108 download standard kernel sources :

 

https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/snapshot/linux-3.10.108.tar.gz

 

or just clone latest stable and 

 

# git checkout v3.10.108

 

copy the .config from the bromolow/apollolake sources to the folder you have the kernel sources that you downloaded, make menuconfig and select the modules you want to compile and the end run make modules.

 

Pay attention to the module dependencies (run modinfo xxx.ko ) as you might need modules to be loaded before that. e.g. module tg3.ko (tigon) depends on libphy.ko, so you need to first load libphy.ko and then tg3.ko. Not all modules have dependencies, but some do.

 

 

It's better to compile modules against synology's source published on sourceforge as they modify structs. Building against vanilla kernel sources may have catastrophic consequences, especially if you're trying to build some driver.

 

 

On 9/6/2021 at 11:12 PM, pocopico said:

@Orphée One problem that is not yet solved or released by @ThorGroup , is that we don’t have a solution for adding the extra modules in one batch like with Jun loader.

 

I’am assuming though that for most installations, adding few required kernel modules in rd could be possible. You can also start telnetd at boot to help troubleshoot sata port errors or anything that even fails to start installation.

 

Lines to add in rd.gz on /etc/rc line 321 could be

 

  echo "Starting early telnetd for troubleshooting "
        /sbin/inetd
        if [ `ps -ef |grep -i inetd |grep sbin | wc -l` -eq 1 ] ; then
               echo "Inetd has started succesfully"
       fi

 

We're working on a solution for drivers. This is currently what's stopping us from formally calling it a beta :)

 

 

On 9/7/2021 at 2:30 AM, abesus said:

Hi everyone,

 

Ive successfully built and use loader for 980+. Unfortunatelly, I'm using Xeon E3-1265L V4 and there were many errors in system logs. System was hanging-up every few hours. It looks like DS918+ is using some new CPU/GPU instructions that are missing in my old xeon.

After that I've tried DS3615xs loader. DSM 7.0 is in beta stage so I've tried 6.2.4. Everything is working great except moments. How can I find valid SN/MAC combination for ds3615xs?

 

Seba

Yes.... Intel decided to add a single instruction (MOVBE) to only the Atom line... then a few years later they've added it to the Xeons. Go figure... that's why 918+ requires a newer CPU with MOVBE support. If you search for "MOVBE intel cpu" you will find many topics not related to xpenology as this is a much bigger problem.

As for the SN/MAC: the mac needs to match your real ethernet adaptor. As for the S/N you can google "xpenology s/n generator" and there are tools prepared by the community.

 

 

On 9/7/2021 at 2:15 PM, titoum said:

is there a way to get the info when it fails at 56% ?

 

all is going smooth until the installation start and stop at that steps :s

 

thx

You need to use a serial console on your hypervisor, login as root:[no password] and check the /var/log/messages - it will say what went wrong somewhere on the bottom.

 

 

On 9/7/2021 at 2:42 PM, abesus said:

So in my case, software for DS918+ should work on my CPU. That would be great (I've working set of SN and MAC) :)

Did you check /var/log/messages? I've got a lot logs like:

kernel: DMAR: DRHD: handling fault status reg 3

kernel: DMAR: [DMA Read] Request device [some numbers] fault addr some address...

I can't remember exactly but if it can be helpful I can build loader again and check.

Did you try updating the BIOS? We've seen such errors on Debian on an Asus motherboard and updating the BIOS solved the issue.

 

 

On 9/7/2021 at 3:19 PM, Null said:

Why is my control center blank

屏幕截图 2021-09-07 151757.png

That is a bug in the DSM itself (JS error on the page...). It happens on a real hardware as well... albeit not always hmmm. However the only broken page is the Info Center => General.

 

 

On 9/7/2021 at 4:19 PM, jumkey said:

from https://gist.github.com/Izumiko/26b8f221af16b99ddad0bdffa90d4329


  "synoinfo": {
    "supportsystemperature": "no",
    "supportsystempwarning": "no"
  },

Yay! A nice fix ;) We will add this to the main loader generator.

 

 

On 9/7/2021 at 5:05 PM, sirdrug said:

(...)

[  127.785579] md1: detected capacity change from 0 to 2147418112
[  128.446373] EXT4-fs (md0): couldn't mount as ext3 due to feature incompatibilities
[  128.466030] EXT4-fs (md0): mounted filesystem with ordered data mode. Opts: (null)
[  128.554349] <redpill/rtc_proxy.c:222> Got an invalid call to rtc_proxy_set_auto_power_on
[  128.559532] EXT4-fs (md0): couldn't mount as ext3 due to feature incompatibilities
[  128.578612] EXT4-fs (md0): mounted filesystem with ordered data mode. Opts: (null)

 

 

These errors are normal. If you look at them just below it the array is mounted. It jus first tries one thing, failes, and then tries another thing and succeeds.

 

 

On 9/7/2021 at 5:20 PM, nemesis122 said:

Has anyone the loader for Bromolow 3615 and can upload for me ?

thank you

 

You can generate it with the loader generator. Please, do not use premade images.

 

 

On 9/7/2021 at 5:21 PM, sirdrug said:

In /var/log/messages
Sep  7 09:13:13 updater: updater.c:7045 ==== Start flash update ====
Sep  7 09:13:13 updater: updater.c:7049 This is X86 platform
Sep  7 09:13:13 updater: boot/boot_lock.c(228): failed to mount boot device /dev/synoboot2 /tmp/bootmnt (errno:2)
Sep  7 09:13:13 updater: updater.c:6510 Failed to mount boot partition
Sep  7 09:13:13 updater: updater.c:3134 No need to reset reason for v.42214 
Sep  7 09:13:13 updater: updater.c:7652 Failed to accomplish the update! (errno = 21)
Sep  7 09:13:13 install.cgi: ninstaller.c:1546 Executing [/tmpData/upd@te/updater -v /tmpData > /dev/null 2>&1] error[21]
Sep  7 09:13:15 install.cgi: ninstaller.c:123 Mount partion /dev/md0 /tmpRoot
Sep  7 09:13:15 install.cgi: ninstaller.c:1515 Moving updater for configuration upgrade...cmd=[/bin/mv -f /tmpData/upd@te/updater /tmpRoot/.updater > /dev/null 2>&1]
Sep  7 09:13:15 install.cgi: ninstaller.c:152 umount partition /tmpRoot
Sep  7 09:13:15 install.cgi: ErrFHOSTCleanPatchDirFile: After updating /tmpData/upd@te...cmd=[/bin/rm -rf /tmpData/upd@te > /dev/null 2>&1]
Sep  7 09:13:15 install.cgi: ErrFHOSTCleanPatchDirFile: Remove /tmpData/upd@te.pat...
Sep  7 09:13:15 install.cgi: ErrFHOSTDoUpgrade(1794): child process failed, retv=-21
Sep  7 09:13:15 install.cgi: ninstaller.c:1811(ErrFHOSTDoUpgrade) err=[-1]
Sep  7 09:13:15 install.cgi: ninstaller.c:1815(ErrFHOSTDoUpgrade) retv=[-21]
Sep  7 09:13:15 install.cgi: install.c:409 Upgrade by the manual patch fail.
Sep  7 09:13:15 install.cgi: install.c:678 Upgrade by the uploaded patch /tmpData/@autoupdate/upload.pat fail.

 

any ideas?

Your vid/pid combination is most likely invalid, or you connected the usb flash drive to a USB3 port. Try doing "dmesg | grep usb_boot_shim" - it will tell you what drives it saw and if any was shimmed as boot disk.

 

 

On 9/7/2021 at 5:22 PM, WiteWulf said:

I had a couple more odd reboots of my system this morning. They always seem to occur shortly after a reboot. I'd restarted the system after updating firmware on the iLO card in my system, and the server rebooted almost immediately after it had booted into DSM (I think it managed about 3 minutes uptime), then rebooted again shortly afterwards. Looking in /var/log/messages for entries just before it reboots each time there's nothing that stands out, other than that it's typically doing stuff you'd expect after the system had just rebooted: starting docker containers and registering certificates for a couple of webhosts I have on the server

Are you sure your RAM is ok? Are you running ECC there? Check the iLO log as well. Try disabling HPET as well.

 

 

On 9/7/2021 at 5:46 PM, WiteWulf said:

Question: I have an HP NC360T PCIe NIC card in my system. I'm only using one interface, so have only declared one of them in my user_config.json. Is this a problem?

 

I notice in Control Panel in DSM that two interfaces are detected, and both have the same MAC address (the one I declared for the interface that I'm actually using). Likewise, running 'ifconfig' from the cli reports two interfaces with the same MAC address. I know that an OSI 7-layer network stack shouldn't have an issue with multiple interfaces on a system having the same MAC address (it's standard on Sun machines, for example), but will DSM complain about this?

 

Next time I rebuild the boot stick I'll be sure to declare both interfaces, but am just curious for now.

You should define both. DSM is... hacky... in terms of ethernet interfaces. All of them should be accounted for with different MAC addresses.

 

 

On 9/7/2021 at 6:34 PM, Franks4fingers said:

I have tried this using both UEFI and Legacy boot from within the BIOS and get the same issue. Does anyone have any tips to get past this point?

Are you able to connect to the machine using a serial port?

 

On 9/7/2021 at 6:49 PM, Franks4fingers said:

 

That line in my grub.cfg file is as follows:

 

linux /zImage netif_num=1 earlycon=uart8250,io,0x3f8,115200n8 syno_hdd_powerup_seq=1 syno_hdd_detect=0 elevator=elevator root=/dev/md0 loglevel=15 log_buf_len=32M mac1=123456789012 vid=0x0781 pid=0x5572 console=ttyS0,115200n8 sn=xxxxxxxxxxxxx earlyprintk syno_port_thaw=1 HddHotplug=0 withefi syno_hw_version=DS918+ vender_format_version=2 

 

Did you add the DiskIdxMap= SataPortMap= SasIdxMap= values to the end of that line?

 

Also, how did you derive what the SataPortMap value needed to be?

 

Cheers for the help.

Please add them to your user config under extra_cmdline and rebuild the loader image.

 

 

On 9/7/2021 at 8:51 PM, nemesis122 said:

Maybe its releated to this and you need 3615 Bromolow

 

Correct, this table still applies. This is due to how syno builds their kernels and how Intel artificially limits instruction sets on some CPUs [fun fact, some older Xeons have the MOVBE in the microcode but it's disabled]

 

 

On 9/7/2021 at 8:57 PM, shibby said:


i have 4-gen CPU (Haswell). I can install DS918+ 6.2.3 (jun). I can install DS918+ 7.0.1 on fresh HDD using redpill. I can even upgrade from 6.2.3 (jun) to 6.2.4 (redpill) using DS918+ image.

But i can`t upgrade from 6.2.3/6.2.4 to 7.x 😕

So what exactly fails with the upgrade? Are you able to recover the log?

 

 

On 9/7/2021 at 10:05 PM, MastaG said:

Trying to build and run it on my HP Gen8.

I first built redpill.ko using redpill-lkm, that was quite easy.

For the record I used gcc 4.9.2 to build it.

 

Using redpill-load I created the following user_config.json:


{
  "extra_cmdline": {
    "vid": "0x0951",
    "pid": "0x1665",
    "sn": "xxx",
    "mac1": "xxx",
    "mac2": "xxx"
  }
}

I copied redpill.ko from repill-lkm to: ext/rp-lkm/redpill-linux-v3.10.108.ko

And finally: sudo ./build-loader.sh 'DS3615xs' '7.0-41222'

 

The image is successfully created, but the first partition doesn't have the bootabled flag:


fdisk -l ./redpill-DS3615xs_7.0-41222_b1631017171.img 
Schijf ./redpill-DS3615xs_7.0-41222_b1631017171.img: 128 MiB, 134217728 bytes, 262144 sectoren
Eenheid: sectoren van 1 * 512 = 512 bytes
Sectorgrootte (logisch/fysiek): 512 bytes / 512 bytes
In-/uitvoergrootte (minimaal/optimaal): 512 bytes / 512 bytes
Schijflabeltype: dos
Schijf-ID: 0xf110ee87

Apparaat                                      Op.  Begin  Einde Sectoren Grootte ID Type
./redpill-DS3615xs_7.0-41222_b1631017171.img1       2048 100351    98304     48M 83 Linux
./redpill-DS3615xs_7.0-41222_b1631017171.img2     100352 253951   153600     75M 83 Linux
./redpill-DS3615xs_7.0-41222_b1631017171.img3     253952 262143     8192      4M 83 Linux

 

So I used fdisk on the image to set the first partition "active", so it can actually boot on legacy systems.

The developers should fix this I guess.

 

After writing it to USB it does seem to try and boot, but it just shows: Booting the kernel.

It hangs for like 20 or 30 seconds, then it reboots. (see screenshot).

 

Any ideas?

 

 

Schermafdruk van 2021-09-07 16-03-44.png

In order to see any logs you must attach a serial/COM port. Syno kernel doesn't support VGA output (so as soon as the grub passes the control to the kernel nothing is passed to the GPU).

 

 

On 9/7/2021 at 10:27 PM, MastaG said:

Yeah it's annoying that the kernel doesn't print any console output tot the iLO console either.

I've tried adding console=tty0 console=tt1, console=ttyS1 etc.. no dice.

It just sits there and reboots all the time :(

If you boot with ttyS0 iLO should be able to access that port. When you log in to the iLO using ssh (use the same credentials as for the web) there's a virtual serial port emulator (vsp command IIRC from memory). Connect to it before booting and you should start seeing messages right after the screen says "Booting the kernel".

 

 

On 9/7/2021 at 10:42 PM, sirdrug said:

 

I updated my synology, but I still can't change the serial number to a valid one, who can tell me where else to change it?

2021-09-07_17-40-37.png

You can put it in your user_config.json and it should work... how did you get that S/N?

 

 

On 9/7/2021 at 10:55 PM, MastaG said:

So I got a little further.

 

I've now build the redpill.ko kernel module using the toolchain stated here: https://github.com/RedPill-TTG/redpill-lkm/pull/16/commits/b3f263c65198d94014f60f52f58bc6be00d1d8f8#

 

So basically:

make LINUX_SRC=~/.toolchain/bromolow-DSM-7.0-toolkit/build CROSS_COMPILE=~/.toolchain/bromolow-DSM-7.0-toolchain/x86_64-pc-linux-gnu/bin/x86_64-pc-linux-gnu-

and also:

strip -S redpill.ko

 

Now using the redpill.ko built by the DSM 7.0 toolchain (and stripped) my Microserver Gen8 doesn't reboot any longer.

It now just sits there at the "Booting the kernel." screen.

 

However I can't see any DHCP being requested and I still don't have any console output.

Any ideas?

Do not use strip on kernel modules - there are many problems with that. Some options will strip e.g. modinfo which will make the module useless for the kernel.

 

 

On 9/7/2021 at 11:06 PM, WiteWulf said:

Oh, that's cool, didn't realise that! That opens up the possibility of getting a better/additional HBA to go in there at some point...

Yes, the built-in card was broken only for a short while. 6.2.3 restored the driver. We didn't test if v7 removed it again (hopefully not).

 

 

On 9/7/2021 at 11:33 PM, imdgg said:

 

1.open ssh service

2.connect to your dsm

3.mount boot partion. Just like this: mount /dev/vda1 /mnt  #my device is /dev/vda1

4.vim /mnt/boot/grub/grub.cfg , search "sn",change the serial number

 

my dsm 7.0.1 runs on qnap 453b-mini

 

If your loader is at /dev/vda1 then something is very wrong... it shouldn't be. It should be at /dev/synobootN. Editing the grub.cfg manually is not (and will not be) supported.

 

 

On 9/8/2021 at 1:03 AM, nemesis122 said:

Which  VID and PID i have to set when i load/install in VMM VirtualmaschineManager ?

 

VMM doesn't allow for virtual USB drive. You need to use SATA boot. However, if you're using VMM you should probably install the official VDSM image.

 

 

On 9/8/2021 at 2:57 AM, MastaG said:

Thanks bro,

That would be great 👍

If you can confirmed the internal NIC isn't supported any longer, then perhaps you can share your card with us.

 

I guess it's a small price to pay to have the latest DSM working :)

NC360T is a good and solid card, even outside of the DSM ecosystem. It's a HP product but it's cheap and readily available (even new). We have probably 30+ of these cards in different servers.

 

 

On 9/8/2021 at 11:45 AM, Franks4fingers said:

Do you have the link for the .pat file that you used for the fresh install?

After you build the loader the correct PAT file will be in the "cache" folder. You don't need to download it manually from syno.

 

 

On 9/8/2021 at 3:58 PM, titoum said:

may be @ThorGroup could enhance first topic with some extra info ?

 

link to latest version of haydibe docker image ?

 

how to get ride of the blank info center


sudo su - root

sed -i 's/supportsystemperature="yes"/supportsystemperature="no"/g' /etc.defaults/synoinfo.conf
sed -i 's/supportsystempwarning="yes"/supportsystempwarning="no"/g' /etc.defaults/synoinfo.conf

 

acpi throttle


sudo su - root
vi /usr/local/etc/rc.d/S99Powersaving.sh and insert according to number of core
 
Script-content:

#!/bin/sh

echo powersave > /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor
echo powersave > /sys/devices/system/cpu/cpu1/cpufreq/scaling_governor
echo powersave > /sys/devices/system/cpu/cpu2/cpufreq/scaling_governor
echo powersave > /sys/devices/system/cpu/cpu3/cpufreq/scaling_governor

chmod +x /usr/local/etc/rc.d/S99Powersaving.sh
sh /usr/local/etc/rc.d/S99Powersaving.sh

 

We will like to but we don't have permissions to edit old posts :D Who should we contact on the form regarding that?

 

 

On 9/8/2021 at 4:12 PM, p33ps said:

 

Synology says that the "face detection" feature which is implemented in lower versions (<7.0) for Moments app is different that the new one, which needs higher hardware to run smoothly. I hope not to affect to xpenology newer versions, but my current experience with 7.0 is quite bad. It consumes too many resources and does not detect any face after hours.

v7 should be considered a beta (even if syno says it's stable) on any hardware, including that sold by synology. There are nasty bugs in it unrelated to xpenology.

 

 

On 9/8/2021 at 4:16 PM, WiteWulf said:

I made some progress with the spontaneous reboots I've been seeing since migrating to 6.2.4 but spun it off into a separate topic as it's not entirely relevant to this thread:

 

That smells like an ancient docker version in the DSM itself... we saw this before and they fixed the issue after like 6-7 months. It may be the case again.

 

 

On 9/8/2021 at 7:48 PM, tossp said:

The kernel keeps reporting this information. What action should I take?

 


[80085.526427] <redpill/rtc_proxy.c:226> Mocking auto-power SET on RTC
[80085.532878] BUG: unable to handle kernel NULL pointer dereference at           (null)
[80085.540936] IP: [<ffffffff812fd1e6>] memcpy_erms+0x6/0x10
[80085.546473] PGD 363fb9067 PUD 3d1d07067 PMD 0
[80085.551076] Oops: 0000 [#713] SMP
[80085.554569] Modules linked in: fuse nfnetlink pci_stub vfio_pci vfio_virqfd vfio_iommu_type1 vfio vhost_net(O) kvm_intel kvm irqbypass xt_ipvs ip_vs_rr ip_vs xt_mark br_netfilter bridge stp aufs macvlan veth xt_addrtype tun cmac ctr ccm ipt_MASQUERADE xt_REDIRECT nf_nat_masquerade_ipv4 xt_nat nf_nat_redirect xt_policy xfrm6_mode_transport xfrm4_mode_transport xfrm6_mode_tunnel xfrm4_mode_tunnel xfrm6_mode_beet xfrm4_mode_beet deflate authencesn ipcomp6 ipcomp xfrm6_tunnel xfrm4_tunnel tunnel6 esp6 esp4 ah6 ah4 xfrm_ipcomp af_key xfrm_user xfrm_algo l2tp_ppp l2tp_core ppp_deflate bsd_comp ppp_mppe echainiv xt_TCPMSS iptable_mangle pppox ppp_async ppp_generic slhc 8021q vhost_scsi(O) vhost(O) tcm_loop(O) iscsi_target_mod(O) target_core_user(O) target_core_ep(O) target_core_multi_file(O) target_core_file(O)
[80085.628492]  target_core_iblock(O) target_core_mod(O) syno_extent_pool(PO) rodsp_ep(O) udf isofs loop nfsv3 nfs quota_v2 quota_tree synoacl_vfs(PO) iptable_nat nf_nat_ipv4 nf_nat raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xt_conntrack xt_mac xt_geoip(O) compat_xtables(O) nf_conntrack_ipv6 ip6table_filter ip6_tables xt_recent xt_iprange xt_limit xt_state xt_tcpudp xt_multiport xt_LOG nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables x_tables nfsd btrfs ecryptfs zstd_decompress zstd_compress xxhash xor raid6_pq lockd grace rpcsec_gss_krb5 auth_rpcgss sunrpc aesni_intel glue_helper lrw gf128mul ablk_helper apollolake_synobios(PO) hid_generic usbhid hid usblp i915 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops cfbfillrect cfbcopyarea cfbimgblt drm drm_panel_orientation_quirks
[80085.702288]  iosf_mbi fb fbdev video backlight button uhci_hcd ehci_pci ehci_hcd r8168(O) i2c_algo_bit openvswitch gre nf_defrag_ipv6 nf_conntrack igb(O) e1000e(O) sg dm_snapshot dm_bufio crc_itu_t crc_ccitt psnap p8022 llc hfsplus md4 hmac sit tunnel4 ipv6 flashcache_syno(O) flashcache(O) syno_flashcache_control(O) dm_mod arc4 crc32c_intel cryptd sha256_generic ecb aes_x86_64 authenc des_generic ansi_cprng cts md5 cbc cpufreq_powersave cpufreq_performance acpi_cpufreq processor cpufreq_stats vxlan ip6_udp_tunnel udp_tunnel zram ip_tunnel etxhci_hcd usb_storage xhci_pci xhci_hcd usbcore usb_common redpill(OE) [last unloaded: apollolake_synobios]
[80085.760520] CPU: 3 PID: 4856 Comm: scemd Tainted: P      D    OE   4.4.180+ #42214
[80085.768258] Hardware name: System manufacturer System Product Name/B150M-PLUS, BIOS 3806 05/18/2018
[80085.777496] task: ffff88004055d280 ti: ffff88012a65c000 task.ti: ffff88012a65c000
[80085.785166] RIP: 0010:[<ffffffff812fd1e6>]  [<ffffffff812fd1e6>] memcpy_erms+0x6/0x10
[80085.793225] RSP: 0018:ffff88012a65fbb0  EFLAGS: 00010246
[80085.798660] RAX: ffff88012a65fbf0 RBX: ffffffffffffffff RCX: 0000000000000134
[80085.805956] RDX: 0000000000000134 RSI: 0000000000000000 RDI: ffff88012a65fbf0
[80085.813252] RBP: ffff88012a65fbc0 R08: 000000000000000a R09: 00000000000085d6
[80085.820550] R10: 0000000000000005 R11: ffffffff819fe16d R12: ffff88012a65fbf0
[80085.827831] R13: ffff8800abd51a40 R14: ffff88042027c048 R15: ffff8800abd51a40
[80085.835135] FS:  00007f5fe7413700(0000) GS:ffff880433d80000(0000) knlGS:0000000000000000
[80085.843377] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[80085.849279] CR2: 0000000000000000 CR3: 0000000104946000 CR4: 00000000003606f0
[80085.856569] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[80085.863891] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[80085.871196] Stack:
[80085.873241]  ffffffffa000875c 00007f5fe7412320 ffff88012a65fe88 ffffffffa06a452c
[80085.880894]  ffff88012a65fc68 ffff88012a65fc60 ffff88012a65fcf0 0000000000000001
[80085.888512]  0000000100000001 00000000007f1700 0000000000000000 0000000000000000
[80085.896131] Call Trace:
[80085.898639]  [<ffffffffa000875c>] ? rtc_proxy_set_auto_power_on+0x70/0x76 [redpill]
[80085.906437]  [<ffffffffa06a452c>] synobios_ioctl+0x54c/0x1e50 [apollolake_synobios]
[80085.914252]  [<ffffffff81131bd9>] ? unlock_page+0x59/0x60
[80085.919748]  [<ffffffff8115e837>] ? do_wp_page+0x287/0x650
[80085.925356]  [<ffffffff8121ecf9>] ? ext4_getattr+0x19/0x90
[80085.930980]  [<ffffffff81194a20>] ? cp_new_stat+0x110/0x120
[80085.936674]  [<ffffffff811a70d2>] do_vfs_ioctl+0x4d2/0x960
[80085.942248]  [<ffffffff811a7601>] SyS_ioctl+0xa1/0xb0
[80085.947395]  [<ffffffff81579c4a>] entry_SYSCALL_64_fastpath+0x1e/0x8e
[80085.953945] Code: 90 90 90 90 90 eb 1e 0f 1f 00 48 89 f8 48 89 d1 48 c1 e9 03 83 e2 07 f3 48 a5 89 d1 f3 a4 c3 66 0f 1f 44 00 00 48 89 f8 48 89 d1 <f3> a4 c3 0f 1f 80 00 00 00 00 48 89 f8 48 83 fa 20 72 7e 40 38
[80085.974293] RIP  [<ffffffff812fd1e6>] memcpy_erms+0x6/0x10
[80085.979909]  RSP <ffff88012a65fbb0>
[80085.983480] CR2: 0000000000000000
[80085.986852] ---[ end trace 487f34b257519703 ]---

 

Fixed in the new release (bug in memory handling).

 

 

On 9/8/2021 at 8:49 PM, mcdull said:

seems 918 is in many way better than 3615

It's a shame that Intel artificially limited MOVBE instruction to Atoms and then enabled it again only in new Xeons. That's actually the only reason why 918+ doesn't work on older CPUs.

 

 

On 9/8/2021 at 9:09 PM, abesus said:

Hello again.

I've got question: how can I add additional param to kernel. I want to pass intel_iommu=off.

Is user_config.json a good place for that?

Yes, add them as key:value pairs to "extra_cmdline" key in your user_config.json

 

 

On 9/9/2021 at 12:39 AM, MastaG said:

any idea where I can get the patched kernel sources for linux-3.10.108 used in 7.0.1-42214 ?
the toolchain only contains the pre-compiled kernel headers, not the full source...

You can't get them - syno didn't publish them yet... the only way to compile modules is to do it with their toolchain sources.

 

--------------------------------------------------------

[to be continued in a next post due to forum limits]

  • Like 3
  • Thanks 9
Link to post
Share on other sites

This post contains responses to questions from mid-page 47 only and was split due to forum's limitations. To see release updates jump to a post on the bottom of page 55:

 

 

----------------------------------------------

 

On 9/9/2021 at 3:46 AM, djvas335 said:

Check post from pocopico on page 41 on how to compile

 

Link to kernel sources

 

https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/snapshot/linux-3.10.108.tar.gz

DO NOT use vanilla kernel sources to build modules for the DSM unless you know what you're getting into. We cannot stop you of course but you should know it's like playing with fire over a tank of gasoline.

 

Let us explain. Syno, despite the version number attached in the kernel, modifies source heavily. So their 4.4.123 can be anything. They back port a lot of things, they break the ABI by modifying structs in the middle etc... original/vanilla kernel sources are mostly useless in dealing with DSM. Building a module against them and loading it is like playing a russian roulette - it may work sometimes but you never know what's going to happen tomorrow. This is especially crazy if you try to build your HBA driver (i.e. you're trusting your data to a driver built with incompatible sources... one shifted struct and you can lose all your data on all disks pretty easily).

 

If you wanna play with something not important like network card and you check the kernel source for that module in the previous syno GPL releases and confirm it wasn't modified you can try. But again, this is a very risky endeavour.

 

 

On 9/9/2021 at 5:34 AM, pocopico said:

Just for testing purposes, It looks that it is possible to install 918+ 7.0.1 in vmware. Steps include linking the detected SATA loader to /dev/synoboot like that 

 

# ln -s /dev/sdXX       /dev/synoboot
# ln -s /dev/sdXX1      /dev/synoboot1
# ln -s /dev/sdXX2     /dev/synoboot2
# ln -s /dev/sdXX3      /dev/synoboot3

 

where sdXX is your loader vmdk disk whose disk name can be found running

 

# fdisk -l 

 

Once you have it installed there will be no need to fake it again as md raid devices are discovered and mounted during boot and are not related to the loader. Anyway you may relink using the same commands. 
 

*** edit : you need to relink after boot, I will do some more testing 

 

Again, wait for developers to release a final version before installing on your production systems.

 

 

This is an old hack which should never be used for v7. New version of the RP solves the problem on ESXi. This hack is a very very very bad idea on v7. There are some new self-destruction routines which check that. DO NOT symlink synoboot, DO NOT use mknod to fake it. This MUST be done through the kernel. You can ignore all warnings we made but please, don't ignore this one.

 

 

On 9/9/2021 at 6:59 AM, hannibal1969 said:

Anybody has a step by step tutorial? Found it very clear instructions.

The link you posted is actually a quite good summary for macOS :)

 

 

On 9/9/2021 at 10:36 AM, Franks4fingers said:

I am still plagued with the issue of hitting 56% and then the install bombing out with the error message of:

 

Failed to Install DSM.

Failed to install the file.  The file is probably corrupted.

 

I have tried a different USB key and changed over the PID and VID accordingly to see if that made a difference but no joy. 

 

Does it make a difference if I choose UEFI boot or Legacy Boot within the BIOS?

 

Are there any other specific BIOS settings that could be causing the continual failure and could be worth changing? 

 

When I now try the install it comes up with my original NAS being migratable which I presume is due to finding a partial install from the failures at 55%.  This particular desktop machine that I am using as a test bed doesn't have anything installed already so I am trying to do a brand new clean install using the DSM 7.01RC from the Synology DL Center. I will wipe the partition on the HDD so it looks like a new disk and try again.

 

Any pointers would be appreciated as this is doing my melon in.

 

Cheers

You should use BIOS boot rather than EFI if possible. Also, 7.0.1 is not supported yet. Check the log over serial console. For the PAT they're always left in cache folder so you don't need to re-download them from syno.

 

 

On 9/9/2021 at 10:52 AM, ghtester said:

'they should' is absolute correct but they don't care much about it for years... They'll do it sometimes but it'll take some (and usually a quite long) time...

...and this is painful. They ignore GPL left and right. They recently posted sources for v7.... and pulled them after like an hour. There's a question posted on their forum on SF and they're not responding.

 

 

On 9/9/2021 at 1:02 PM, dogbig said:

Faces cannot be enabled
Are any files missing?

Faces detection isn't working as it requires a REAL serial number (or someone to patch the new binary, this is outside of the scope of the loader).

 

 

On 9/9/2021 at 1:16 PM, Franks4fingers said:

Using putty I am able to telnet to the diskstation and I am prompted for a username and password.

 

Anyone know what the default is for telnet?

Login is always root:[empty password] for preboot environment. Later on when the system boots this obviously doesn't work for security reasons.

 

 

On 9/9/2021 at 5:34 PM, MastaG said:

That didn't work for me.

 

I believe it's something like this:



mkdir rd
cd rd
xz -dc < ../rd.gz | cpio -idmv
(add drivers)
find . 2>/dev/null | cpio -o -c -R root:root | xz -9 --format=lzma > ../rd.new.gz
cd ../
rm -Rf rd
mv rd.gz rd.old.gz
mv rd.new.gz rd.gz

Still I can't seem to get any network interfaces.

Also the tg3.ko and libphy.ko where not inside the rd.gz archive.. while I did add it to the config.json file...

 

The iLO VSP console only print up to the point where it says:

[    2.167768] <redpill/uart_swapper.c:428> ### LAST MESSAGE BEFORE SWAP ON "OLD" PORT ttyS1<=>ttyS0

 

From that point on I'm blind...

 

 

The format of packing depends on the version. Check dsm-research repo under ramdisk document as syno broke the kernel. In general, the rd.gz shouldn't be tinkered with manually without proper knowledge.

 

 

On 9/9/2021 at 10:29 PM, flyride said:

 

That is what FixSynoboot is intended to correct (and does exactly what you show above)

https://xpenology.com/forum/topic/28183-running-623-on-esxi-synoboot-is-broken-fix-available/

 

Synoboot devices are not required after install, until you want to do an upgrade.  The loader storage is always modified by an upgrade.  I realize that is getting ahead of things with regard to RedPill, but that's how it works.

As we mentioned above FixSynoboot is not part of the RP for a good reason. Synoboot is actually being checked in v7 after DSM is booted.

 

 

On 9/10/2021 at 2:58 AM, ct85msi said:

Found a bug, when I run the integrated benchmark tool on any disk, from Storage Manager, the raid array is corrupted. The benchmark never finishes and when I stop it, the array will resync.

 



20253.145546] md_error: sdd2 is being to be set faulty
[20253.145836] raid1: Disk failure on sdd2, disabling device. 
               	Operation continuing on 3 devices
[20253.182475] RAID1 conf printout:
[20253.182659]  --- wd:3 rd:16
[20253.182815]  disk 0, wo:0, o:1, dev:sda2
[20253.183027]  disk 1, wo:0, o:1, dev:sdb2
[20253.183239]  disk 2, wo:0, o:1, dev:sdc2
[20253.183466]  disk 3, wo:1, o:0, dev:sdd2
[20253.188464] RAID1 conf printout:
[20253.188648]  --- wd:3 rd:16
[20253.188804]  disk 0, wo:0, o:1, dev:sda2
[20253.189016]  disk 1, wo:0, o:1, dev:sdb2
[20253.189227]  disk 2, wo:0, o:1, dev:sdc2
[20254.150881] md: unbind<sdd2>
[20254.161450] md: export_rdev(sdd2)
[21149.167214] md: bind<sdd2>
[21149.196351] RAID1 conf printout:
[21149.196559]  --- wd:3 rd:16
[21149.196714]  disk 0, wo:0, o:1, dev:sda2
[21149.196925]  disk 1, wo:0, o:1, dev:sdb2
[21149.197136]  disk 2, wo:0, o:1, dev:sdc2
[21149.197347]  disk 3, wo:1, o:1, dev:sdd2
[21149.197912] md: md1: current auto_remap = 0
[21149.198229] md: recovery of RAID array md1
[21149.198487] md: minimum _guaranteed_  speed: 10000 KB/sec/disk.
[21149.198804] md: using maximum available idle IO bandwidth (but not more than 600000 KB/sec) for recovery.
[21149.199308] md: using 128k window, over a total of 2097088k.
[21229.131072] perf interrupt took too long (2510 > 2500), lowering kernel.perf_event_max_sample_rate to 50000
[21232.348680] md: md1: recovery done.
[21232.363216] md: md1: current auto_remap = 0
[21232.396664] RAID1 conf printout:
[21232.396863]  --- wd:4 rd:16
[21232.397018]  disk 0, wo:0, o:1, dev:sda2
[21232.397229]  disk 1, wo:0, o:1, dev:sdb2
[21232.397440]  disk 2, wo:0, o:1, dev:sdc2
[21232.397691]  disk 3, wo:0, o:1, dev:sdd2

 

Are there any other errors during the benchmark? In dmesg or /var/log/messages?

 

 

On 9/10/2021 at 3:01 AM, jhoughten said:

Also, it seems like the maxdrives and internalportcfg settings in /etc/synoinfo.conf keep reverting to their original settings.

I have made the change in /etc/synoinfo.conf and /etc.defaults/synoinfo.conf.

The usbportcfg setting is retained, but the others revert back to maxdrives=16 and internalportcfg=0xffff

 

 

You should not edit these files by hand... user_config.json contains a provisioning for this and the documentation explains why you cannot edit these files like that (TL;DR: dsm overrides them and they need to be fixed on every reboot, it is done automatically by RP).

 

 

On 9/10/2021 at 3:03 AM, flyride said:

I believe Jun's loader patches these on each boot.  This may need to be added to RedPill.

Small correction: RedPill supports it from the beginning - it is described in the documentation in the redpill-load repository.

 

 

On 9/10/2021 at 6:30 AM, flyride said:

SataPortMap, DiskIdxMap, etc are not in /etc/synoinfo.conf, it's part of options passed to the Linux kernel via grub, i.e. it's just assembling the argument string from the options you provide.  I don't think that particular RedPill provisioning option will help with /etc/synoinfo.conf parameters.

 

Just to be clear, /etc.defaults/synoinfo.conf is copied to /etc/synoinfo.conf each boot.  And some of the /etc.defaults/synoinfo.conf parameters are "reset" by DSM on upgrades or on each boot, hence the Jun loader patch.

Again, this is also supported and documented. It's the "extra_cmdline" parameter in user_config.json which ensures options are placed in all kernel boot cmdlines/params and if needed override some default ones etc.

 

 

On 9/10/2021 at 7:06 AM, flyride said:

Got it, so maybe it does actually modify the DSM file... I didn't look that closely before.

 

Unlikely that maxdisks=32 will work and will probably be overridden by DSM as it starts.  I think 24 is the practical maximum.

To our knowledge the absolute max is 26 as DSM doesn't support /dev/sdaa /dev/sdab /dev/sdac etc. naming. Since there's only 26 letters you can only use 26 disks. Their eboxes work differently and that's how on a real DS you can get >26 drives.

 

 

On 9/10/2021 at 7:19 AM, jhoughten said:

However DSM interprets that value is beyond me.  With Jun's loader on DSM 6.2.3, maxdisks=6, but the storage manager shows 5 used slots and 5 available slots.

It must be some combination of the maxdisks and the internalportcfg settings.

 

These new settings on 7.0 have storage manager showing disks 1, 17 and 18.  Disk 17 is the loader USB device, even though I do have synoboot, synoboot1, and synoboot2 in the /dev directory.

This is related to the Map parameters. Jun's loader used a hack to "hide" the loader image - RP doesn't need it. You need to configure mapping properly. This is a good description on how it works:

 

 

On 9/10/2021 at 11:19 AM, mcdull said:

AFAIK, the valid mac is just needed to be input to the grub, but the actual hardware mac dose not matter, can you confirm?

I used it under proxmox so I acutally put the valid mac into virtual machine as well.

It actually does matter for some things - with invalid MAC the DHCP sometimes doesn't work. That's why it must match. You can try specifying that you have 2 network interfaces and putting your real MAC into one and the one matching S/N into the other.

If you're using proxmox however you can put that MAC in the virtual network interface on proxmox and that way you will have a matching boot setting with the actual card MAC. This is actually something we do as we have many VMs on many hosts to boot the kernel but just one user_config.

 

 

On 9/10/2021 at 2:06 PM, ressof said:

Hi

 

Im trying to install DSM 7.01 apollolake 918+ on ESXI.

 

When running fdisk before installing PAT file I get this:



Disk /dev/sda: 120 GB, 128849018880 bytes, 251658240 sectors
15665 cylinders, 255 heads, 63 sectors/track
Units: sectors of 1 * 512 = 512 bytes

Disk /dev/sda doesn't contain a valid partition table
Disk /dev/synoboot: 128 MB, 134217728 bytes, 262144 sectors
1008 cylinders, 5 heads, 52 sectors/track
Units: sectors of 1 * 512 = 512 bytes

Device       Boot StartCHS    EndCHS        StartLBA     EndLBA    Sectors  Size Id Type
/dev/synoboot1    0,32,33     6,62,56           2048     100351      98304 48.0M 83 Linux
Partition 1 has different physical/logical start (non-Linux?):
     phys=(0,32,33) logical=(7,4,21)
Partition 1 has different physical/logical end:
     phys=(6,62,56) logical=(385,4,44)
/dev/synoboot2    6,62,57     15,205,62       100352     253951     153600 75.0M 83 Linux
Partition 2 has different physical/logical start (non-Linux?):
     phys=(6,62,57) logical=(385,4,45)
Partition 2 has different physical/logical end:
     phys=(15,205,62) logical=(976,3,36)
/dev/synoboot3    15,205,63   16,81,1         253952     262143       8192 4096K 83 Linux
Partition 3 has different physical/logical start (non-Linux?):
     phys=(15,205,63) logical=(976,3,37)
Partition 3 has different physical/logical end:
     phys=(16,81,1) logical=(1008,1,12)

and the logs when trying to install the PAT file from serial console



[  183.720933] <redpill/rtc_proxy.c:37> MfgCompatTime raw data: sec=20 min=3 hr=6 wkd=5 day=10 mth=8 yr=121
[  183.723631] <redpill/rtc_proxy.c:95> Writing BCD-based RTC
[  183.724954] RTC time set to 2021-09-10  6:03:20 (UTC)
[  186.800143] md: bind<sda1>
[  186.801637] md/raid1:md0: active with 1 out of 16 mirrors
[  186.802992] md0: detected capacity change from 0 to 2549940224
[  189.811875] md: bind<sda2>
[  189.813208] md/raid1:md1: active with 1 out of 16 mirrors
[  189.814563] md1: detected capacity change from 0 to 2147418112
[  190.198325] EXT4-fs (md0): couldn't mount as ext3 due to feature incompatibilities
[  190.209674] EXT4-fs (md0): mounted filesystem with ordered data mode. Opts: (null)
[  190.240130] <redpill/rtc_proxy.c:222> Got an invalid call to rtc_proxy_set_auto_power_on
[  190.246693] EXT4-fs (md0): couldn't mount as ext3 due to feature incompatibilities
[  190.257611] EXT4-fs (md0): mounted filesystem with ordered data mode. Opts: (null)
[  190.272865] EXT4-fs (md0): couldn't mount as ext3 due to feature incompatibilities
[  190.283741] EXT4-fs (md0): mounted filesystem with ordered data mode. Opts: (null)
[  196.595667] ext2: synoboot2 mounted, process=updater
[  196.599505] synoboot2 unmounted, process=updater
[  196.607208] ext2: synoboot2 mounted, process=updater
[  196.609221] synoboot2 unmounted, process=updater
[  196.616749] ext2: synoboot2 mounted, process=updater
[  196.619137] synoboot2 unmounted, process=updater
[  196.625799] ext2: synoboot2 mounted, process=updater
[  198.179513] synoboot2 unmounted, process=updater
[  203.191011] ext2: synoboot2 mounted, process=updater
[  203.250955] synoboot2 unmounted, process=updater
[  203.261041] ext2: synoboot1 mounted, process=updater
[  203.265540] vfat: synoboot1 mounted, process=updater
[  203.268169] FAT-fs (synoboot1): Volume was not properly unmounted. Some data may be corrupt. Please run fsck.

Can this be fixed?

Regenerate loader image. It is corrupted according to the last error message. Keep in mind we don't support 7.0.1 yet.

While these errors don't look fatal having a broken filesystem is another complication which may cause problems (and FAT is very prone to being damaged... to the point that syno itself keeps two copies).

 

 

On 9/10/2021 at 5:03 PM, Polanskiman said:

Oh boy, have I been sleeping!!!. I got some serious reading to do.

Haha, nice to see an admin here. It's just 55 pages ;)

 

 

On 9/10/2021 at 5:58 PM, pocopico said:

 

Right after that, the system is supposed to extract the ramdisk. The issue can be that you have a big or corrupted rd.gz.

 

I noticed that only the uncompressed image gets extracted successfully on some systems for some reason

 

***edit and the reason is explained on the loader Documentation for developers :

 

https://github.com/RedPill-TTG/redpill-load/blob/master/FOR_DEVS.md

 

 

 

There's a detailed description of the problem in the ramdisk document in dsm-research repository. TL;DR: syno broke the kernel routines for decompression... [a minute of silence as we don't want to swear in public].
This is the reason why we advise against using versions needing editing of the image by hand. RD.gz compression type is just one of the stupid little quirks to take care of - we automate that to not think about that ourselves.

 

 

On 9/10/2021 at 6:09 PM, pocopico said:

@ThorGroup @jumkey Should/could someone create a compiled module repository in GH ? This way we can offload this thread from module requests

We deliberately didn't do this as we explained before in this thread. Current release is intended for developers and should not be used by regular users which aren't able to run a bash script. We will provide a precompiled modules when (very soon at this point ;)) the project reaches at least a beta state. One person was even nice enough to provide a PR on GH to automate the build (THANKS!).

 

The overwhelming number of posts in this thread are from people who downloaded a random image and started modifying configs inside it without even reading the documentation. When we ask for any logs there's usually silence which doesn't help us solve the underlying issue.

 

 

On 9/10/2021 at 10:45 PM, abesus said:

Hi,

I've question: Is such log normal or is something bad in my system:

 

 

Yes, that's normal - if you didn't physically do "rmmod apollolake_synobios" you're good. This is a debug message which is very helpful if a kernel panic happens right after "synobios: unload". mfgBIOS module doesn't handle being unloaded manually.

 

 

On 9/10/2021 at 11:22 PM, p33ps said:

Could you please be more specific about this?

Add it to the extra_cmdline in user_config.json and rebuild the loader image.

 

 

On 9/11/2021 at 12:22 AM, stigma86 said:

Hi everyone.

 

I'm a satisfied user, but I was thinking of trying DMS 7.
My Mobo is the Asrock J4005B-ITX + 2x4GB 2400 RAM +  WD Purple 2TB HDD+ etc HW's and my System is DMS 6.2.3 (918+, Loader 1.04B loader, Pendrive)

I will be using a new Pendrive and HDD because if the installation will be fail, I can restore the current system(DMS 6.2.3)

I'm reading the topic, and I have some questions:
 - This is a developer loader: Google Drive link => this is a good link or I'm a noob😀
 This is a ,,step by step codex"?
  - Download the loader
  - mount OFSmount and replace the SN (918+) PID and VID
  - Download the latest path file : 918+ => DSM 7.0.1 (Version: 7.0.1-42214)
  - I continue the installation on the web interface and install the 7.01 image, that I'm writing

No, this isn't any official build. If you want to try playing with it you should use the official loader builder (with or without @haydibe docker). However, keep in mind that as of right now the loader is still in the alpha stage (however, beta will be coming soon with precompiled binaries of the module).

 

Sharing full loader images is something which will get people into troubles (as we explained before).

 

 

On 9/11/2021 at 2:48 AM, haydibe said:

 Let's say I share the reservation that things might become ugly to maintain and new problems might be introduced.

I must admit I am not realy keen to spend time on research about what would needed to be done.

 

Personaly I prefer to wait for the official redpill implementation as well :)

THANK YOU!!

 

This is exactly the message we're trying to send - there are hacks, we can load custom drivers, we do that on dev machines, we also do other weird things like restoring snapshots with memory and taking them mid-load or using hardware breakpoints... but this isn't something which a non-kernel developer should ever try. It's like playing with fire. However, ultimately we want a robust solution which does work for most/all of the cases instead of hacking the ramdisk and modifying files to do an insmod... and what if you put a driver which is slightly too large? (ramdisk has a strict limit). Well, the kernel is gonna boot with a partial ramdisk and everything will start collapsing with random errors. This is a nightmare to debug... ask us how we know :D

 

We're all for PRs and other contributions. We will definitely incorporate the improvement from posts here (e.g. the temperature breaking general tab) but it's way harder when other users start to (often unaware and confused) use other mods posted as attachments here and begin reporting bugs as RP bugs. Your docker image is always nicely documented and robust contributing to a great value here. However, some other mods* asking people to mount images and edit menu entries for grub ends up in broken installations as it skips the whole process and all the checks left in place for a reason.

 

There are very good reasons why we're not sharing complete images and why we spend many many many hours developing the generator. The RP started in a different thread, its code was shared in different GH repos, the initial work wasn't done by us... and it's now unavailable. Full loader images contain way more than the GPL kernel. They contain syno's proprietary and copyrighted binaries. Distributing them is... problematic at best.

 


*for the record here: nothing personal against anybody, we don't even remember nicknames half of the time as there are so many posts. We don't mind if someone publishes a modified version. It's just that it's incredibly hard for us to support modifications which are published as binaries with no instructions, a PR, and directly contradict the process we develop for building images. We love contributions, we run RP as an open-source initiative for a reason! We only try to limit confussion of less experienced folks :)

 

 

 

On 9/11/2021 at 6:07 AM, icaroscherma said:

Do you guys plan to work on a DS920+ release instead of 918+? Or it's just too much work?

 

I'm asking because I figured that DS Video won't work with HWA transcoding for hevc because it's based off the 918+ (Intel UHD500 - j3355/j3455, not UHD600 - j4105/j4125), so the DS Video authentication doesn't even check for hardware acceleration on hevc.

It would be a nice update from my baremetal jun's DSM6.2.3 918+ to a DSM7 920+. ❤️ 

We're thinking about looking at one of the nvidia-based ones as they support both nvidia and intel quick sync. Especially that on an AMD system you can get something like Quadro P200 for cheap and have 7 streams of HVEC with little CPU usage.

 

 

On 9/11/2021 at 8:43 AM, mcdull said:

 I bet an AMD variant first will bring more benefit.  I vote for 1621+

This is one of our big things - with AMD being so affordable and so powerful now... it will be great to have support for it. The biggest roadblock are S/N generation methods. The generators we've seen from the community support 3615/918/3617.

 

Is there any description on how these S/N are generated? How these values were discovered?

 

 

On 9/12/2021 at 12:16 AM, buggy25200 said:

Hello,

I have a misunderstanding on how to get a .bsp patch for DSM 7.0.1 rc,
This is what i did:

1) Extract zimage with the extract-vmlinux


2) Patch vmlinuz with patch-ramdisk-check.php



#php patch-ramdisk-check.php vmlinuz vmlinuz.1
Generating patch for vmlinuz
Found .rodata at 81 60 00 00 in vmlinuz
Found "3ramdisk corrupt" at ff fd bd 0f in vmlinuz
LE arg addr: 0ebd6d81
Found pritk arg @ ff cc 62 2b
Found printk MOV @ ff cc 62 28
OK - patching 7417 (JZ) to eb17 (JMP) @ 13394470
Patched 2 bytes in memory
Saving stream to vmlinuz.1 ...
DONE!


3) Then patch vmlinuz.1 with patch-boot_params-check.php
 



php patch-boot_params-check.php vmlinuz.1 vmlinuz.2
Generating patch for vmlinuz.1
Found .init.text at ff cc 30 00 in vmlinuz.1
Looking for f() candidates...
Analyzing f() candidate @ ff cf 0e 78
[?] Found possible f() @ ff cf 0e 78
        [+] Found LOCK-OR#0 sequence @ ff cf 0f c2 => f0 80 0d 36 84 22 00 01 [RIP+(dec)15832056]
        [+] Found LOCK-OR#1 sequence @ ff cf 10 18 => f0 80 0d e0 83 22 00 02 [RIP+(dec)15832056]
        [+] Found LOCK-OR#2 sequence @ ff cf 10 56 => f0 80 0d a2 83 22 00 04 [RIP+(dec)15832056]
        [+] Found LOCK-OR#3 sequence @ ff cf 10 69 => f0 80 0d 8f 83 22 00 08 [RIP+(dec)15832056]
        [+] All 4 LOCK-OR PTR offsets equal - match found!
Patching OR to AND @ file offset (dec)13569988
Patching OR to AND @ file offset (dec)13570074
Patching OR to AND @ file offset (dec)13570136
Patching OR to AND @ file offset (dec)13570155
Saving stream to vmlinuz.2 ...
DONE!

 



ls -l vmlinuz.2
-rw-r--r-- 1 root root 14174320 11 sept. 17:54 vmlinuz.2

 

4)



bsdiff  vmlinuz vmlinuz.2 patch_sort.bsp
ls -l patch_sort.bsp
-rw-r--r-- 1 root root 180 11 sept. 17:58 patch_sort.bsp

 

I don't understand why my bsp file is much smaller than the one you made

 



ls -l zImage-001-3615xs-42214-ramdisk-and-header.bsp
-rwxr-x--- 1 jim root 164235  3 sept. 19:10 zImage-001-3615xs-42214-ramdisk-and-header.bsp

 

Can you explain steps I am missing ?

 

Thank you all of you for this great work !

The redpill-load has a mode where you can repack the kernel and let it continue. The resulting bsp can be bigger or smaller depending on the compression results (as the bsdiff should be run on compressed image). As long as it's a proper diff the size doesn't matter here. The REAL changes are just a few bytes - the resulting bsp is bigger than that due to compression and bsdiff header.

For details as to how to "be guided" to build a new platform look at variables on the top of build-loader.sh file. You need to set BRP_LINUX_PATCH_METHOD=repack and few others (set this one - it will tell you which ones are missing) and then try to build an image of a platform which has no .bsp file. It will go up to the kernel patching, stop and let you patch the kernel. Then you restart the same process and it will detect patched kernel and recompress it etc. In the end it will leave these two compressed files for you to bsdiff :)

We want to have proper docs for this but it's all about time and number of hands :D

 

 

8 hours ago, pocopico said:

There are two things that come in mind :

 

1. 10Gb driver is not loaded correctly

2. /etc/synoinfo.conf does not have the support_10g=y

 

log in with ssh and verify the above 

Does it even matter what DSM shows in the settings for network speed? For VirtIO it shows Unknown or 10/100 while everything is working at few Gb/s. We will simply try iperf3 test and check if the speed is really <=1Gb/s between DSM and some other host and then try to tinker with it. If it's >1Gb/s it doesn't matter if the panel shows 10/100/1000 as this isn't really read directly from the interface or driver.

 

 

1 hour ago, jumkey said:

sorry but l didn't find best way to change cp /efi/. /efi

Hm, can you describe what you've changed to make the EFI working? (sorry if you did and we've missed it in the sea of posts) We don't want to unnecessarily duplicate the work if you made it already :)
We think it just needs the new template image for the GRUB which supports EFI boot as the syno-specific files are copied anyway (whether GRUB supports EFI boot or not).

 

 

1 hour ago, maxhartung said:

 

 

No, the laptop doesn't have serial console (HP Elitebook 840 G2). I will try to find a way to attach a serial console in some way. Thank you.

Hm, that can be tricky then as AFAIK syno kernel doesn't support serial over USB. Are you able to see it connecting to the network at all? (e.g. on your router's DHCP leases/clients table).

 

 

48 minutes ago, cferra said:

@ThorGroup I will gladly donate an LSI sas2 HBA if necessary.  Feel free to PM me here or on chat.

Thank you for the offer ;) We have a few LSI cards but currently none of them is in any of the servers which we can play with. It just happens that for pure HBA we usually use cheap SATA cards and not good LSI controllers (as they need flashing in IT mode and not all of them play nice with SMART etc).

 

 

28 minutes ago, p3t3 said:

have a question, dsm 6.2.4 loader can support 6.2.4-25556 update 1 and 2?

Good question. Can you try unpacking the update PAT (use tar -xvf FILE.pat) and seeing if there's a different zImage inside? If there isn't you're good to go to just update.

Edited by ThorGroup
  • Like 2
  • Thanks 3
Link to post
Share on other sites

New release update! 💊

 

This is the biggest release to date and ESXi users will be especially happy to see it! ;)


SATA boot support for 918+ (and others)
Remember how we said it's nearly impossible to add SATA boot support for 918+ as the kernel physically lacks the code to support SATA DOM on that platform? Well, we've said NEARLY impossible as we had an idea but didn't want to disappoint anyone if it failed. It didn't.

 

With this release we're introducing a SATA boot support for platforms which natively lack the support for it. This is mainly aimed towards hypervisors without emulated USB support (looking at you ESXi). How it works is rather crazy: we're taking the SATA drive and masking it as a USB drive just for a short while to fool the SCSI driver into giving the drive a correct type (but not long enough to cause problem with reading the data). We weren't sure if this can even work or how good it will work. It turns out it does work, and it looks to be stable.

 

However, due to the nature of this code we're only recommending (at least for now) using it when there's no other way to boot. A physical USB flash drive passed through to a VM will be a better choice. Keep in mind this does not apply to native SATA DOM (e.g. on 3615) - this one is as stable, if not even more, than USB. You can easily distinguish which mode is used by looking at boot parameter "synoboot_satadom": native DOM will be synoboot_satadom=1 while this emulated one will be enabled with "synoboot_satadom=2". Technically you can actually use this emulated mode on platforms supporting native SATA DOM (not that there's a reason).

 

For details see the following commits:

 

 

Disk SMART emulator
One of the biggest issues reported was the inability to create pools on ESXi (or any VMWare products). This bug was traced on GH as well: https://github.com/RedPill-TTG/redpill-lkm/issues/14 . There were multiple theories and the community stepped up and came up with workarounds. However, this state wasn't sustainable for a long time so we went digging.

 

Oh boy. v7 made a lot of changes in how disks are handled. One of the biggest changes is that now all drives used by the system in pools must support SMART (and actually multiple parts of it). Without that things will break horribly, even if a workaround is used to create a pool. The pools creation worked on Proxmox only because QEmu actually emulates a minimal (but sufficient for basic operation) SMART support for all its virtual drives.

 

With this release we implemented a full-blown emulated support for SMART on any drive which doesn't support it. The module makes everything glow green and report no problems. Even the health tab and automatic SMART tests will not cause any problems and appear as they were really executed. Obviously the emulated SMART has no diagnostic value and the drive should be monitored externally. The SMART emulation goes beyond being useful under ESXi. We've identified three main scenarios where the SMART data is/can be missing:

  • running under ESXi (it doesn't support SMART at all)
  • running VirtIO disks connected to SCSI ports (see below)
  • using RAID cards which don't work in a real HBA mode

 

With the commit https://github.com/RedPill-TTG/redpill-lkm/commit/d032ac48c00f78b11516f2f2112970bc51bf68be out of the way we can officially confirm full support for ESXi. We didn't close the issue on GH yet awaiting feedback.

 


Added support for SCSI VirtIO
..and there's more disk updates in this release!

Proxmox (and all other VirtIO-enabled hypervisors like QEmu, libvirt [used in e.g. unRAID], or VirtualBox) allow the drives to be passed as either SATA or SCSI. There are technical differences between them. However, the main one while actually using it is that you can add up to 6 SATA devices (0-5) while SCSI allows for up to 31 (0-30).

 

This release adds the support for VirtIO SCSI drives. This includes SMART emulation described above as well, as QEmu doesn't provide SMART for non-SATA devices. As SATA devices ultimately use SCSI protocol and share the vast majority of the protocol we simply convince the OS to treat VirtIO SCSI drives as they were SATA. This isn't really a hack. The official syno kernel does the same thing BUT only for "NEXTKVMX64" platform (VDSM?). We simply retrofitted that feature to work on all platforms. You can also use an SCSI drive for booting and mix SATA & SCSI disks as well.

 

The core functionality can be found in the commit: https://github.com/RedPill-TTG/redpill-lkm/commit/2ce06db3ef69b6ea28c49754c101fd0832abec04

 


SCSI drives monitoring subsystem
This is more of a dev-only functionality, but lies in the core of all the features described above. In short, Linux kernel contains a mechanism for publishing and subscribing to events. Unfortunately it's a rather new and niche subsystem. The SCSI driver, as one of the oldest parts of the Linux kernel, doesn't support the notifications. In order to react to new drives being connected to the system we developed an extension for the SCSI driver adding that functionality. We will probably pursue upstreaming of these changes, so one day they can land in the kernel natively.

 

For details see the following commits:


Multiple build modes & hiding from the system
The previous version of the module was always built in the dev mode. In addition to being heavy at whooping 4MB, due to the debugging information included, it also contained extensive logging and multiple exported symbols. With this release made substantial progress in hiding the module from the OS.

Now the module can be built in three different modes:

  • dev: all debug symbols are included, standard debug messages are printed. This is essentially the current mode as it is. It's still the default one when running make without parameters
  • test: module is stripped from debug symbols and displays only warnings, errors, critical and BUG messages. This means the output is significantly reduced while still allowing to catch problems occurring. This is the recommended mode to run on test installations. However, when reporting bugs we will prefer to get the output from the dev build as it contains much more information of what lead to the bug.
  • prod: module is fully stripped from debug symbols, contains absolutely no logging, and in addition it makes an effort to hide itself from the OS. It will not be visible in the lsmod, it will not publish its symbols in /proc/kallsyms and once loaded it cannot be unloaded. If it crashes in this state the stack trace will point to an anonymous kernel address rather than to a loaded module.

For details see the following commits:


Why the correct compiler is so important?
We had a discussion here some time ago about using different GCC versions and playing with different configurations. We've got a tangible example now, which made our blood boil for a good whole day. The code which is native to the kernel triggered a bug in the GCC itself: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=63567

 

In short this problem wasn't present in earlier versions of the GCC, but it is affecting GCC >4 and <5. This is a serious problem as the module is written with C99 in mind and the bug occurs when code is compiled in any mode besides C89. Well, normally you would upgrade your GCC. However, Linux <3.18 will NOT compile even with GCC 5.0 - the absolute latest one is 4.9.x.
The workaround we've used was to extract the offending piece of code to a separate compilation unit compiled in C89 mode where rest of the code is still being compiled in [sane] C99 mode.

 

We're leaving this story as a warning how compiler version can have big effects on even code which is written in a clean manner. In the C world, despite so many years of advancements, there are still monsters in the closet like that ;)

 

 

--------------------------------------------------------------------------------------------------------

 

 

Assorted smaller changes & fixes we've made:

 

Fix RTC scheduling crash
When auto power-on scheduling was modified in any way the action triggered on platforms with no native RTC support (e.g. 918+) was causing a soft-crash in the RTC proxy. It was reported in https://github.com/RedPill-TTG/redpill-lkm/issues/17 and fixed in https://github.com/RedPill-TTG/redpill-lkm/commit/29637db757d8ad0240e161136db814d5bfd31c33

 

 

Fix SATA DOM shimming on older HDDs
Some older HDDs (and possibly some real SATA DOMs) don't implement so-called CAP16 command used to query the capacity. Due to a bug in our capacity estimation code the CAP10 fallback wasn't triggered for disks without CAP16 support. Now this is fixed and capacity should be detected for all drives properly.
Related commit: https://github.com/RedPill-TTG/redpill-lkm/commit/524690549cb7cad96de50cc350a9e13f520f53fa

 

 

Fix execve() debugging crashes
Did you know that Linux launches few thousand binaries every time it boots? We didn't either. Enabling (a very noisy yet helpful) execve() debug mode caused some crashes. It was really about edge cases in arguments parsing. As we are working with the code remembering the Linux code from 1992 some things slip through the cracks. With the commit https://github.com/RedPill-TTG/redpill-lkm/commit/1a6665bc71f7a73b02a3c150aad746d359caf9a4 the debugger shouldn't cause crashes. If you experience it let us know via an issue.

 

 

GCC warnings are now gone
To avoid confession (which was reported here and on GH) while making the code overall better we went ahead and fixed all GCC warnings. One of the main ones was the validate_nets cropping up on all platforms: https://github.com/RedPill-TTG/redpill-lkm/commit/0a91024b5ed4e51382015d098f0c4ed04417700f

 

 

Transition to new override symbol is complete
As mentioned in the previous post new, more stable and faster, override symbol logic has been implemented. Now it's used across the board. Old method has been completely removed. In the process we've also found a rare set of circumstances where the override could fail completely and crash the kernel. This has been fix as well.

For details see the following commits:

 

More consistent logging
To make debugging & reporting from users easier we unified how each shim/submodule reports its state. This also serves as a general clean-up of these logs.

For details see the following commits:

 

Build env. improvements

  •  We've validated and enabled concurrent build for the LKM. Concurrent builds speed up the build times of the kernel module drastically. Commit: https://github.com/RedPill-TTG/redpill-lkm/commit/363f6d4d6459f05f7b49c0e72c8e4e3dddf5690e
  •  VMDKs are build in addition to raw images in the mass build script, see: https://github.com/RedPill-TTG/redpill-load/commit/24894ae306648353e71214839e0aac8317e05816 - the only issue here is that these VMDKs doesn't seem to be recognized natively by ESXi (at least not 7.0.2) and require one extra command on the ESXi itself. If anyone knows the solution working on Linux we will be glad to hear it. Due to the closed-source nature of ESXi we have no idea why ESXi dislikes perfectly valid VMDKs here. Oddly enough they work perfectly in other VMWare products. Go figure.
  • The attach/mount race condition which randomly cropped up for some people (e.g. https://github.com/RedPill-TTG/redpill-load/issues/10 ) started happening on one of our member's machines as well. The underlying issue is simply the timing: losetup on linux does NOT wait for the partitions scan, so it's possible that the mount command executes after /dev/loopX has been created but before the kernel had a chance to scan partitions. This issue, funny enough, usually happens when listening to music in Spotify. The fix was dead-simple: https://github.com/RedPill-TTG/redpill-load/commit/cecf8416ee0219f534b976e0879bf7e2923827f7
  • CLion should be happy again with the code - we've fixed special UAPI include path (https://github.com/RedPill-TTG/redpill-lkm/commit/4b5905039815647508ae7e0305715f1b0bfff923 ) as well as duplicated defs (https://github.com/RedPill-TTG/redpill-lkm/commit/8347a4116217191e31711e72d7d849464e348064 )

 

Edited by ThorGroup
  • Thanks 15
Link to post
Share on other sites
2 hours ago, ThorGroup said:

 

On 9/6/2021 at 4:53 PM, mcdull said:

I tried 3615 image in my old qnap.  But it does not boot properly.

In GRUB> whenever I press enter, it will reboot by itself.

 

What CPU do you have? If it just hard-reboots it means the CPU just crashed, which usually means that it doesn't support some instructions.

 

mine is D425, probably require DS713+ image for maximum compatibility of instruction set.

Link to post
Share on other sites
25 minutes ago, ThorGroup said:

DO NOT use vanilla kernel sources to build modules for the DSM unless you know what you're getting into. We cannot stop you of course but you should know it's like playing with fire over a tank of gasoline.

 

Let us explain. Syno, despite the version number attached in the kernel, modifies source heavily. So their 4.4.123 can be anything. They back port a lot of things, they break the ABI by modifying structs in the middle etc... original/vanilla kernel sources are mostly useless in dealing with DSM. Building a module against them and loading it is like playing a russian roulette - it may work sometimes but you never know what's going to happen tomorrow. This is especially crazy if you try to build your HBA driver (i.e. you're trusting your data to a driver built with incompatible sources... one shifted struct and you can lose all your data on all disks pretty easily).

 

If you wanna play with something not important like network card and you check the kernel source for that module in the previous syno GPL releases and confirm it wasn't modified you can try. But again, this is a very risky endeavour.

Unfortunately, for LSI drivers, such as mptsas3, I have tried many times to use the 25426branch kernel code of 918 to compile, and errors will be reported when the driver is loaded (the specific error content I mentioned in the above post) and the system cannot be correct. Start, and jumkey uses the mpt3sas source code of 4.4.180 released by the Linux open source community to compile the mpt3sas driver, which can be loaded correctly and recognize the hard disk, so if for data security considerations, should we wait until Qunhui releases the 41890 kernel source code? 

Link to post
Share on other sites

@ThorGroup as always thank you so much for all your work and your time that you spent to reply to all those pages. I even went to sleep and woke up and you guys kept replying...amazing.

 

May I suggest the following if you do not mind... Can you guys share a roadmap? Where are we, whats needs to be done from your perspective before an official release. In that way it might help a little bit with all that hacking that is going on lately. ;)

  • Like 1
Link to post
Share on other sites
20 minutes ago, mcdull said:

A noob question.  For every new loader image to test with, is it mandatory to migrate the entire system again?

Look at the first batch of today's answers from ThorGroup. I asked the same question look at the answer...

Edited by gadreel
Link to post
Share on other sites
4 hours ago, ThorGroup said:

So what exactly fails with the upgrade? Are you able to recover the log?

 

Unfortunately i can`t (or don`t know how). I have not respond on ping, SSH&telnet are not available. So i don`t have access to console to retrieve logs.

 

Once again to be clear: start point 6.2.3 DS918+ (jun), put redpill loader 7.x, start PC and no ping, no telnet, no access to ip:5000.

Working way for me: 6.2.3 DS918+ (jun) -> 7.0.1 DS3615xs (redpill) -> 7.01 DS918+ (redpill).

I don`t have idea why i cannot upgrade directly from 6.2.3 DS918+ to 7.0x DS918+.

Link to post
Share on other sites
1 hour ago, maxhartung said:

I can see the error: We have detected errors on hdd 1,3

1st) connect your drives in the order starting from SATA0 port.

2nd) add SataPortMap=4 to your config grub.cfg file. Set this vaule to corresponding to your number of hard drives

Link to post
Share on other sites
3 hours ago, ThorGroup said:

 

Are there any other errors during the benchmark? In dmesg or /var/log/messages?

 

Only this error in /var/log/messages :

 

2021-09-13T09:34:08+03:00 Apollo kernel: [246979.674416] md_error: sdd2 is being to be set faulty
2021-09-13T09:34:08+03:00 Apollo kernel: [246979.674710] raid1: Disk failure on sdd2, disabling device.
        Operation continuing on 3 devices
2021-09-13T09:37:30+03:00 Apollo synodisklatencyd[10373]: raid_uuid_get.c:81 Disk [/dev/sdd3] in RAID [/dev/md3] has no RAID superblock, ret=0
2021-09-13T09:37:30+03:00 Apollo synodisklatencyd[10373]: raid_uuid_get.c:81 Disk [/dev/sdd3] in RAID [/dev/md3] has no RAID superblock, ret=0
2021-09-13T09:37:37+03:00 Apollo synodisklatencyd[5770]: SYSTEM:        Last message 'raid_uuid_get.c:81 D' repeated 1 times, suppressed by syslog-ng on Apollo

 

and in dmesg:

 

[245399.816502] RTC time set to 2021-09-13  6:07:48 (UTC)
[246979.674416] md_error: sdd2 is being to be set faulty
[246979.674710] raid1: Disk failure on sdd2, disabling device.
                        Operation continuing on 3 devices
[246979.874503] RAID1 conf printout:
[246979.874722]  --- wd:3 rd:4
[246979.874954]  disk 0, wo:0, o:1, dev:sda2
[246979.875301]  disk 1, wo:0, o:1, dev:sdb2
[246979.875625]  disk 2, wo:0, o:1, dev:sdc2
[246979.875956]  disk 3, wo:1, o:0, dev:sdd2
[246979.941246] RAID1 conf printout:
[246979.941435]  --- wd:3 rd:4
[246979.941597]  disk 0, wo:0, o:1, dev:sda2
[246979.941813]  disk 1, wo:0, o:1, dev:sdb2
[246979.942028]  disk 2, wo:0, o:1, dev:sdc2
[246980.681243] md: unbind<sdd2>
[246980.687229] md: export_rdev(sdd2)

 

Meanwhile...the benchmark is stuck in this state, as the image attached.

 

Thank you for all of your work !

 

hdd benchmark faulty.jpg

After I stop the test, the array is resyncing.

hdd benchmark faulty 2.jpg

Edited by ct85msi
Link to post
Share on other sites
3 hours ago, ThorGroup said:

If anyone knows the solution working on Linux we will be glad to hear it. Due to the closed-source nature of ESXi we have no idea why ESXi dislikes perfectly valid VMDKs here. Oddly enough they work perfectly in other VMWare products. Go figure.

@ThorGroup My understanding is that for ESXi the vmdk is split into 2 files the raw data (Diskname-flat.vmdk) and the raw disk descriptor (Diskname.vmdk), whereas other products like VMWare Workstation have the descriptor followed by the disk data in a single vmdk file.. hope that helps.

Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.