pocopico Posted September 28, 2021 Share #2051 Posted September 28, 2021 I ordered an old HP SMART ARRAY P410, i want this for ESXi (7.0) also Link to comment Share on other sites More sharing options...
Orphée Posted September 28, 2021 Share #2052 Posted September 28, 2021 (edited) @pocopico I'm just saying you made need to compile missing driver if you try to passthrough the PCI card. mpt2sas is native in DS3615xs, whereas P222 was not detected. I did not look further with P222 needed module. Edited September 28, 2021 by Orphée Link to comment Share on other sites More sharing options...
pocopico Posted September 28, 2021 Share #2053 Posted September 28, 2021 Just now, Orphée said: @pocopico I'm just saying you made need to compile missing driver mpt2sas is native in DS3615xs, whereas P222 was not detected. I did not look further with P222 needed module. It should work with hpsa.ko 1 Link to comment Share on other sites More sharing options...
flybird08 Posted September 28, 2021 Share #2054 Posted September 28, 2021 4 minutes ago, pocopico said: I know what you mean, problem is that mpt3sas module will name the devices in a way it cannot be interpreted correctly by DSM. Also not all synology models support SAS by default so here we are waiting for the RedPill magic I think many people are using the HBA sas to SATA control card. Even if it cannot be used directly, the data can be read out through manual mounting, which is acceptable in a short time. The next step is to wait quietly for redpill magic. I think I'm close to the truth that I can use it, but I still lack raid_class.ko, as shown in this picture. Thanks again. 😊 3 minutes ago, Orphée said: I know and I totally agree with you Yes, I'm using this cable. Link to comment Share on other sites More sharing options...
pocopico Posted September 28, 2021 Share #2055 Posted September 28, 2021 (edited) 3 minutes ago, flybird08 said: I think many people are using the HBA sas to SATA control card. Even if it cannot be used directly, the data can be read out through manual mounting, which is acceptable in a short time. The next step is to wait quietly for redpill magic. I think I'm close to the truth that I can use it, but I still lack raid_class.ko, as shown in this picture. Thanks again. 😊 Yes, I'm using this cable. I think, raid_class and libsas, scsi_transport_sas are included in the DSM kernel. Edited September 28, 2021 by pocopico Link to comment Share on other sites More sharing options...
flybird08 Posted September 28, 2021 Share #2056 Posted September 28, 2021 6 minutes ago, pocopico said: I think, raid_class and libsas are included in the DSM kernel. Bad news, neither seems to be included in the DS918+. Quote root@DS918:/# ls /lib/modules/r* /lib/modules/r8168.ko /lib/modules/raid0.ko /lib/modules/raid10.ko /lib/modules/raid456.ko /lib/modules/raid6_pq.ko /lib/modules/rodsp_ep.ko /lib/modules/rpcsec_gss_krb5.ko root@DS918:/# ls /lib/modules/lib* /lib/modules/libiscsi.ko /lib/modules/libiscsi_tcp.ko Link to comment Share on other sites More sharing options...
pocopico Posted September 28, 2021 Share #2057 Posted September 28, 2021 No you didn’t get me, they are not modules they are compiled in/with the kernel Link to comment Share on other sites More sharing options...
D.S Posted September 28, 2021 Share #2058 Posted September 28, 2021 1 hour ago, pocopico said: I also tried and it fails. I guess need firmware cml_guc_xxx.bin or cml_huc_xxx.bin to get it to work. https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/tree/i915 Link to comment Share on other sites More sharing options...
xavierxw1982 Posted September 28, 2021 Share #2059 Posted September 28, 2021 9 hours ago, Orphée said: You have to manually edit your global conf file. Jumkey's repository is not part of official redpill_tool_chain release. Here is mine (zoom on bromolow conf only) { "id": "bromolow-7.0.1-42218", "platform_version": "bromolow-7.0.1-42218", "user_config_json": "bromolow_user_config.json", "docker_base_image": "debian:8-slim", "compile_with": "toolkit_dev", "redpill_lkm_make_target": "dev-v7", "downloads": { "kernel": { "url": "https://sourceforge.net/projects/dsgpl/files/Synology%20NAS%20GPL%20Source/25426branch/bromolow-source/linux-3.10.x.txz/download", "sha256": "18aecead760526d652a731121d5b8eae5d6e45087efede0da057413af0b489ed" }, "toolkit_dev": { "url": "https://sourceforge.net/projects/dsgpl/files/toolkit/DSM7.0/ds.bromolow-7.0.dev.txz/download", "sha256": "a5fbc3019ae8787988c2e64191549bfc665a5a9a4cdddb5ee44c10a48ff96cdd" } }, "redpill_lkm": { "source_url": "https://github.com/RedPill-TTG/redpill-lkm.git", "branch": "master" }, "redpill_load": { "source_url": "https://github.com/jumkey/redpill-load.git", "branch": "develop" } }, @Amoureux develop branch can share the apollo conf file content for 7.0.1-42218? thank you Link to comment Share on other sites More sharing options...
svenger87 Posted September 28, 2021 Share #2060 Posted September 28, 2021 (edited) I´m struggling with installing the PAT file. The Webassistant shows the PAT is corrupted. System is a Intel NUC RTL 8111H card. Any hints whats wrong? Edit: Ah. I think i need to edit the SATA params in the grub.cfg. Edited September 28, 2021 by svenger87 Link to comment Share on other sites More sharing options...
Null Posted September 28, 2021 Share #2061 Posted September 28, 2021 I changed 3615 to 918 at night. There is no problem of automatic restart at present. However, although I can log in to synology account, as long as I set quickconnect, I will be forced to exit the account. I don't know whether it's synology's problem or a system bug. Being able to log in to the account should mean the calculated Sn and MAC, right? (My 3615 didn't have this problem before) Link to comment Share on other sites More sharing options...
Orphée Posted September 28, 2021 Share #2062 Posted September 28, 2021 2 minutes ago, Null said: I changed 3615 to 918 at night. There is no problem of automatic restart at present. However, although I can log in to synology account, as long as I set quickconnect, I will be forced to exit the account. I don't know whether it's synology's problem or a system bug. Being able to log in to the account should mean the calculated Sn and MAC, right? (My 3615 didn't have this problem before) You changed SN and MAC from real DS3615xs to real DS918+, did you ? Link to comment Share on other sites More sharing options...
Null Posted September 28, 2021 Share #2063 Posted September 28, 2021 Just now, Orphée said: 您将 SN 和 MAC 从真正的 DS3615xs 更改为真正的 DS918+,是吗? Yes, I think it is (I calculated the Sn and MAC code of 918, so I'm not sure it must be right. Does being able to log in to an account mean it's right?) Link to comment Share on other sites More sharing options...
svenger87 Posted September 28, 2021 Share #2064 Posted September 28, 2021 16 minutes ago, svenger87 said: I´m struggling with installing the PAT file. The Webassistant shows the PAT is corrupted. System is a Intel NUC RTL 8111H card. Any hints whats wrong? Edit: Ah. I think i need to edit the SATA params in the grub.cfg. Okay. It's the missing synoboot issue some members had earlier before. Link to comment Share on other sites More sharing options...
Orphée Posted September 28, 2021 Share #2065 Posted September 28, 2021 3 minutes ago, Null said: Yes, I think it is (I calculated the Sn and MAC code of 918, so I'm not sure it must be right. Does being able to log in to an account mean it's right?) I'm not talking about calculated SN an MAC. In order to use synology services you must have real genuine SN and MAC from a real DS918+ (or DS3615xs etc...) But still, using these services with Xpenology is strongly discouraged. 1 Link to comment Share on other sites More sharing options...
Null Posted September 28, 2021 Share #2066 Posted September 28, 2021 1 minute ago, Orphée said: 我不是在谈论计算的 SN 和 MAC。 要使用 Synology 服务,您必须拥有来自真正DS918+(或 DS3615xs 等...)的 真正正版SN 和 MAC 但是,强烈建议不要将这些服务与 Xpenology 一起使用。 ok,i will do it tomorrow ,thanks Link to comment Share on other sites More sharing options...
WiteWulf Posted September 28, 2021 Share #2067 Posted September 28, 2021 (edited) 12 minutes ago, svenger87 said: Okay. It's the missing synoboot issue some members had earlier before. I think this may be on the way to fix that: https://github.com/RedPill-TTG/redpill-boot-wait (Lots of interesting new bits appeared on GitHub this afternoon) Edited September 28, 2021 by WiteWulf Link to comment Share on other sites More sharing options...
Popular Post ThorGroup Posted September 28, 2021 Author Popular Post Share #2068 Posted September 28, 2021 (edited) Did anyone said drivers? You said you wanted drivers - we are delivering on the promise to find a way to get them to you. Today's release brings, what we believe is, a huge update to the RedPill as a project, and we hope to the community and the ecosystem around it as a whole. TL;DR We're introducing a decentralized extension manager into the RedPill project which is able to automate drivers installation. If you're a user, and we didn't screw up badly, just update to the newest release and nothing should change just yet. Soon you can expect your custom drivers to be added with a single command. Wait, did you just built... a package manager?! Yes, yes we did. In short after reviewing multiple solutions like ipkg nothing fit the bill with what we actually needed. The distribution of drivers and shell hacks is a pretty unique use-case. We needed to make it work in the minimal initramfs environment. It is not as crazy as it may sound, but the planning for that move was ongoing for quite some time. We mentioned it here and there in passing, that was part of the reason why we discouraged from just blindly stuffing drivers into the core ramdisk and hacking the scripts as we sort of knew what's coming soon However, as you probably realized by now we try to underpromise to avoid disappointment. Lessons learned from the history & other projects Previously used loader by the community (Jun's loader) used a system of a single ramdisk-like "extra.lzma" archive. This method, while worked, lacked flexibility and required hand-assembly by the packers. As we observed this lead to multiple versions of the whole loader image floating around - some with VirtIO, some with ethernet drivers, some with SAS HBA drivers, some with VirtIO **and** SAS HBA... Additionally, while reporting problems on the forum people usually reported whether they "used extra.lzma" which without the context was limited in information. In addition, to add extra functionality beyond kernel modules, a custom hacks had to be developed. The extension management shipping with the "redpill-load" hopes to alleviate the pain of packing and managing all extensions and mods. During the design process we took inspiration from both the Xpenology and Hackintosh communities, attempting to bring what's best. We set the following objectives while developing the current version: as simple and as automated as possible for users in practice users need only a single "add" command and a URL to add an extension updates are checked any time a user builds a new image updates are offered not only on per-platform basis but extension can be updated for the currently existing platform (however, this still requires image rebuilt) if no interdependent extensions are installed users don't need to do a thing as by default all added extensions are added to the image no more manually sharing zips over Mega or in attachments needed easy for developers/packers the most basic extension requires 3 files: index file, recipe file, and the file which is delivered JSON configs were organized such that it's easy to maintain a repo with multiple versions of them (more on that in the documentation for developers - see below for a separate thread) the structure is designed for maximum reusability and extension error-proof most errors are meant to be self-fixable (even if user caused them) when an error is displayed it tries to offer a reason and a possible way to fix it (like in `git`) all extensions offer multiple links pointing users to the correct place (do they have a problem with USING the extension? or maybe they have a problem with it being unstable? or maybe they want to see the forum thread where they can ask the author questions? => it's all supported) the code will prevent users from creating an image with a newer version of OS if extensions they rely on are not updated (as what's the point of updating to a new OS release if your ethernet driver doesn't work yet?) flexible in nature at first, we jotted the idea as delivering kernel extensions/drivers, but we quickly realized it's not enough the current (early) version allows for pretty much any customization of the OS before it's booted with custom scripts (e.g. changing CPU governor) current implementation is more a PoC (that's why it's a messy shell script monstrosity), but with feedback can be eventually built to a much higher standard; more on that below decentralized we don't want to exert any control or force anyone to do anything: this is our goal from the beginning the extension manager is designed to be not only open-source but open in nature. We deliberately don't offer any centralized repository and simply rely on community and the trust within it to offer a quality contribution the system is fully anonymous: we don't collect any statistics, nothing is sent to any of endpoints controlled by us We need the community support! This is a calling for developers & packers which normally provided drivers in the extra.lzma format to also consider providing drivers in a form of RedPill extensions. We hope we don't ask too much here but we know without your support and hard work plenty of things would never be possible. We certainly don't have the manpower to provide drivers for all network cards, HBAs, GPUs and other things people use. To filter-out the noise we created a separate thread to discuss the extensions manager and get suggestions from the developers itself: We created a separate thread to let people compiling drivers & preparing hacks get notifications of a lesser volume than here. Currently available packages As a proof-of-concept we didn't create just a throwaway example extension which isn't actually usable. Instead, we extracted the VirtIO driver into a package: https://github.com/RedPill-TTG/redpill-virtio In addition, we added a small new package which is shell-only (hint: someone can easily base the CPU governor package on it ;)) which blocks the boot process until /dev/synoboot is ready and is more useful in leaving a log as this seems to be one of the most popular problem users are facing. See https://github.com/RedPill-TTG/redpill-boot-wait for details. Both packages mentioned above are autoinstalled. In the future the VirtIO package will be optional (as it does nothing on e.g. ESXi or bare metal). The RedPill LKM will be an extension itself in some time as well (but for now we didn't want to break the @haydibe docker and forking scenarios until we have a good solution for that). The future The current extension manager is conceptually an MVP and can be used. It provides the basic functionality expected from a package manager and delivers the most important thing: drivers. However, this is definitely not its final form. Our goal now was to get it to the usable state and ensure the format of packages will not need to be changed (putting burden on people providing drivers) but rather enhanced as needed. The current biggest hurdle is the fact the package manager is written in bash, along with the loader generator. We're already prototyping a much better solution The end goal is to have a user simply start a docker image on any OS (whether it's Linux, macOS, Windows, or... DSM itself) with a single command and get a web interface where the loader can be generated, updated, and customized (similarly to how you do in the Hackintosh community). Packages discoverability While the packages are (and will be) decentralized we are already thinking about a better discoverability of packages Ideally we want to model a [much simpler yet similar] system to npmjs or packagist but server straight from a github repository anonymously and safely Discoverability will let us provide a small bootable image which can simply scan the hardware and tell you which packages you need and if maybe some hardware is not supported at all Currently we're simply collecting links to indexes - https://github.com/RedPill-TTG/redpill-extensions - some form of static generator with a list will come later So, what do you guys think? //parts of this post were incorporated into the documentation in the redpill-load repository - if you're reading this after some time it's better if you head to the repository directly and see the documentation there as things probably have changed: https://github.com/RedPill-TTG/redpill-load ========================= Yes, we promised to respond to all the posts - we will That last week was unusually busy - we will try to be better about that. ========================= Note to @haydibe: we tried not to break the docker toolchain - can you triple-test? We also left a separate env variable called MRP_SRC_NAME which is used instead of "./ext-manager.sh" when displaying any help/error messages in the extension manager. You may want to replace it with docker command so users aren't confused. Edited September 28, 2021 by ThorGroup 4 19 Link to comment Share on other sites More sharing options...
WiteWulf Posted September 28, 2021 Share #2069 Posted September 28, 2021 Fantastic work folks, can’t wait to see where this leads beyond simple driver handling 👍🏻 1 Link to comment Share on other sites More sharing options...
haydibe Posted September 28, 2021 Share #2070 Posted September 28, 2021 (edited) Well done @ThorGroup! Once again a marvelous update that brings a lot of excitement to all of us I did check the toolchain builder builds for the 6.2.4 and 7.0 configurations that are supported out of the box (=configurations from TTG's redpill-load repository). All of them use the new plugin manager and build like a charm. Though I haven't checked the generated image yet. I noticed the cached pat files fpr apollolake are re-downloading from redpill-load's build-loader.sh, but this time the `+` character is replaced with a `p` character in the output filename. Thus `ds918+_25556.pat` became `ds918p_25556.pat`. Who ever is affected can simply replace the + in the filenames with a p and prevent the files from beeing re-downloaded again. Additionaly I tested jumkey's apollolake-7.0.1-42218, which is not yet alligned with the recent changes in redpill-load. 1 hour ago, ThorGroup said: We also left a separate env variable called MRP_SRC_NAME which is used instead of "./ext-manager.sh" when displaying any help/error messages in the extension manager. You may want to replace it with docker command so users aren't confused. I must admit, this one went over my head I was going to check how `build-loader.sh` calls `ext-manager.sh` and then simply add an array to declare plugins in the toolchain builders global_config.json and execute the command for each element before executing the loader build. Update: the ext-manager.sh syntax is straight forward, no need to check how build-loader.sh uses it. Though, I have to find a clever way to pass in an array, even though docker doesn't support to use arrays as environment variables. Edited September 28, 2021 by haydibe Link to comment Share on other sites More sharing options...
svenger87 Posted September 28, 2021 Share #2071 Posted September 28, 2021 (edited) 1 hour ago, WiteWulf said: I think this may be on the way to fix that: https://github.com/RedPill-TTG/redpill-boot-wait (Lots of interesting new bits appeared on GitHub this afternoon) No luck. Also with the boot-wait plugin. Edited September 28, 2021 by svenger87 Link to comment Share on other sites More sharing options...
haydibe Posted September 28, 2021 Share #2072 Posted September 28, 2021 1 hour ago, haydibe said: Update: the ext-manager.sh syntax is straight forward, no need to check how build-loader.sh uses it. Though, I have to find a clever way to pass in an array, even though docker doesn't support to use arrays as environment variables. I think I found a clever way. I added "extensions": to each platform version with "id": and "url": fields, and pass in the whole extensions array as a single string and process it inside the container. Now I need some extensions to take this addition to a test drive. I also added support for a custom_config.json that allows to merge its values with the global_config.json. This would allow to keep individual platform version configurations in your own config file, which won't be replaced by a new release of the toolchain loader. 2 2 Link to comment Share on other sites More sharing options...
svenger87 Posted September 28, 2021 Share #2073 Posted September 28, 2021 1 hour ago, svenger87 said: No luck. Also with the boot-wait plugin. I think i found the issue. Missing USB controller drivers. How can i integrate the drivers from pocopicos repo? Link to comment Share on other sites More sharing options...
pocopico Posted September 28, 2021 Share #2074 Posted September 28, 2021 @ThorGroup A lot to diggest ! Looking forward for what the extension will bring ! Link to comment Share on other sites More sharing options...
ThorGroup Posted September 28, 2021 Author Share #2075 Posted September 28, 2021 (edited) 3 hours ago, haydibe said: I noticed the cached pat files fpr apollolake are re-downloading from redpill-load's build-loader.sh, but this time the `+` character is replaced with a `p` character in the output filename. Thus `ds918+_25556.pat` became `ds918p_25556.pat`. Who ever is affected can simply replace the + in the filenames with a p and prevent the files from beeing re-downloaded again. Additionaly I tested jumkey's apollolake-7.0.1-42218, which is not yet alligned with the recent changes in redpill-load. I must admit, this one went over my head I was going to check how `build-loader.sh` calls `ext-manager.sh` and then simply add an array to declare plugins in the toolchain builders global_config.json and execute the command for each element before executing the loader build. Update: the ext-manager.sh syntax is straight forward, no need to check how build-loader.sh uses it. Though, I have to find a clever way to pass in an array, even though docker doesn't support to use arrays as environment variables. About the change with + => p is essentially keeping with the same translation layer as in the kernel so the models are named in the exact same way (and that we don't have special characters in file paths ;)). As for the calling in docker we were thinking about something even simpler. Since we're playing with interactive interface it will be nice if users can just run the command in the container instead of editing a file. This is why ext-manager.sh script self-manages files based on commands. So that the author of an extension can just say "just call add [URL]" and call it a day. The "extensions" in the user config is something which we normally don't expect to be used as we wrote in the docs. This is only for a case where user has some interdependent extensions (e.g. you need to load thunderbolt driver first to then load RAID enclosure driver). The bundled-exts is even rarer of an occasion - this file defines extensions which are integral parts of the loader as we want to extract the LKM into an extension to make things simpler (similarly how we extracted VirtIO). So to give you an example how we think it can work: $ docker run -d --name rp -t ..... $ docker exec rp extension add 'https://.....' The output of ext-manager.sh add 'https://.....' $ docker exec rp build foo bar The output of build-loader.sh foo bar $ docker exec rp extension info test_dev.test_ext The output of ext-manager.sh info test_dev.test_ext It's actually simple - just symlink the .sh to /usr/local/bin/extension and /usr/local/bin/build. That way the docker just proxies the commands and we don't double the docs. It will also be simpler overall as we then have a unified interface. This also helps with one change which we have already planned: the add command for now accepts URLs but want it to accept URLs, names, and aliases. This way a user will be able to do "extension add trusted-dev.hp-222". As we mentioned in the previous post with time we want to also add (probably automatically) detection of VIDs & PIDs to the extensions so you will be able to essentially do "extension add fff1:fefe" and get the driver you need. When we get something sensible (i.e. not bash) we can also help users with settings manager where they can simply do "docker exec user-set vid 1234" ... "docker exec user-set mac1 00ff....". WDYT? 3 hours ago, svenger87 said: No luck. Also with the boot-wait plugin. Do you have file /.no_synoboot? If so then check dmesg to diagnose WHY the synoboot is not appearing. Edited September 28, 2021 by ThorGroup 1 1 Link to comment Share on other sites More sharing options...
Recommended Posts