Jump to content
XPEnology Community

RedPill - the new loader for 6.2.4 - Discussion


Recommended Posts

3 minutes ago, dolbycat said:

Hi everyone. What error is this? 

제목 없음.jpg

 

OrpheeGT repo do not have virtio for 7.0.1-42218.

 

you can use my repo if you want

 

        {
            "id": "apollolake-7.0.1-42218",
            "platform_version": "apollolake-7.0.1-42218",
            "user_config_json": "apollolake_user_config.json",
            "docker_base_image": "debian:10-slim",
            "compile_with": "toolkit_dev",
            "redpill_lkm_make_target": "prod-v7",
            "downloads": {
                "kernel": {
                    "url": "https://sourceforge.net/projects/dsgpl/files/Synology%20NAS%20GPL%20Source/25426branch/apollolake-source/linux-4.4.x.txz/download",
                    "sha256": "af815ee065775d2e569fd7176e25c8ba7ee17a03361557975c8e5a4b64230c5b"
                },
                "toolkit_dev": {
                    "url": "https://sourceforge.net/projects/dsgpl/files/toolkit/DSM7.0/ds.apollolake-7.0.dev.txz/download",
                    "sha256": "d349fa644392d4cfab8191243ee38aaa32bd517208c144678e0c855cb5a619ea"
                }
            },
            "redpill_lkm": {
                "source_url": "https://github.com/RedPill-TTG/redpill-lkm.git",
                "branch": "master"
            },
            "redpill_load": {
                "source_url": "https://github.com/chchia/redpill-load.git",
                "branch": "develop"
            }
        } 

 

 

  • Thanks 2
Link to comment
Share on other sites

5 hours ago, ThorGroup said:

Sorry to be blunt but what you're doing in this script is very stupid (in a sense of code, not you as the author :)) and dangerous.

You're adding compilation options which we're almost sure you have no idea about. You've got an error with the new GCC about the PIE relocation/ASLR so you just changed the makefile for the LKM to use -fno-pie... sure, go for it! That's definitely why that error was stopping the build process so that -fno-pie can be simply added to the build line (/s).

We're strongly recommending a read of a post we put on page 55 which discusses some of the GCC stuff (which was discussed many times in this thread before). GCC isn't like Python or V8 - the version REALLY matters. Linux v3 will not build with >=5 for example. There are good reasons why kernel developers are extremely careful and have patches applied per GCC version in the code. The fact some code compiles is just at best 50% of the success with GCC.

 

Additionally, you're going against everything we recommended here. Really, this script is a collection of all "what not to do" we warned against in this thread:

  • don't compile LKM or kernel bootstrap with GCC >4.9.2 (yet the script just YOLOs it)
  • don't install random drivers loading 101 unneeded ones (and that's why we developed the extensions at all, but the script just adds a collection of drivers even if most people don't need them)
  • configure user_config for yourself which you gonna keep for long time and which is semantically valid (yet the script just obliterates it with sed, not even a jq which was installed)
  • read the docs & configurable options in commands (yet the script just does a forceful "mv" instead just using an option already provided for that)
  • and as a desert: it will fail if your path has spaces in it.

To not be just grumpy and complain we will make a suggestion: utilize a docker image by @haydibe which works great and does things safely. It soon will be merged into the main repository as well.

Obviously, as the project is open-source and on top of that a full-blown GNU, we cannot stop you from using it. However trust us on that: we collectively wasted long weeks on dealing with pesky bugs where things weren't compiled exactly right and in a proper environment.

Yeap I understood what you said about GCC version and those used quick flags to bypass errors i have to improve it thanks to point out atleast you read my sript :) . By breaking norms and adding all drivers this is use for myself for my CI/CD as a makeshift update until you guys came up with better i will wait. to point of Docker i used it and found few issues to my use case else it is better toolchain but I can't integrate with my pipeline :( so I thought bash can do it so wrote it in 1 hr without caring much. Sorry I will remove that script. tune in background for my use case until you guys make something then my life will be easier :)

Link to comment
Share on other sites

On 10/1/2021 at 9:33 AM, ThorGroup said:

 

Finally we've got some time to respond to the questions. We will try to respond to as many as possible so stay tight :)
=======

 

Why there's a problem with compiling drivers when syno didn't publish kernel sources?

 

 

This is going to be a separate post addressing "why you cannot randomly grab drivers and compile them" so we can link to it. Let us explain where's the problem, this applies to many questions really.

There are few major issue with compiling drivers against a different sources. The three major ones are:
1) Versioning of syno kernel's is broken due to backporting
2) Drivers modification
3) Kernel <=> driver modification
4) Data structures missing in toolchains for in-tree drivers

 

Now let us explain each one of them separately and why this is an issue. There's a lot to unpack here. First lets address few terms which are used in kernel development:

  • "in-tree" = code which exists within the source of the kernel; in terms of drivers it means you will find it under "drivers/" (e.g. in drivers/scsi/mpt2sas)
  • "vanilla kernel" = source of the kernel which is the official version from kernel.org
  • "kernel headers" = .h files present in the toolchain. These are NOT all kernel's .h files but only a subset of public API kernel provides. You will find these under "include" or in the syno toolchain. What's important they DO NOT contain private interaction between modules. That's why if e.g. mpt3sas driver wants something from mpt2sas you cannot use toolchain to compile mpt3sas and you NEED full sources.
  • "upstream it" = make your custom patch for the kernel and get it accepted into the normal kernel sources

 

1) Versioning & backporting
Syno, like many other companies, take the kernel and modify it in house. This is something which kernel devs don't really like but since it's an open-source project everyone can. However, after some time these companies realize they want new features from new kernel but cannot get them as their modifications no longer apply to a new version. What they do is "backporting". They grab their e.g. 3.10.108 which was modified to oblivion, then download the e.g. 3.19.0 and check commits which add features they want. They get that commit and apply to their 3.10.108 making it 3.10.109. This is incorrect and broken. But that't not the biggest problem.
In case you don't see it: this poses a HUGE problem for using drivers from the vanilla kernel. Often times if you take a driver from vanilla 3.10 you will realize that suddenly doesn't work at all... as in fact they previously lifted a newer one from 3.20 and stuffed it in 3.10. Good luck loading such driver and using it.
For that reason alone you CANNOT use vanilla kernel code as you don't know which version of the code has to be used. Sure, there are ways to figure this out, but the only sane approach is to use the source provided by syno. In case of small-ish bumps in kernel it's usually safe to get e.g. driver from kernel from 3.10.105 and fake the version compiling for 3.10.108 hoping not much changed.

 

2) Drivers modification
Syno modifies in-tree drivers directly. To be honest they have no way around it as they apply vendor-specific hacks for e.g. timing bugs in certain chips. They should upstream it but... that's a different story and it's not as easy as it seems. Depending on WHAT was changed in the driver sometimes you can use kernel from a different kernel version. How do you know it's safe? Well... it's hard. If you don't have the sources for the new kernel (since if you did you wouldn't bother lifting the driver from an older version) you cannot know if the data structures changed. If they did you're pretty much toasted.
Languages like C or ASM (or hardware in general) doesn't have magical way of discovering sizes of things. Everything is based on static assignment (ok, not everything, but lets make a simplification here). This means if syno decided to add a field to a struct [it's like an object] in mpt3sas driver and that field was added to a struct used ONLY in mpt3sas driver you can probably load the old driver and be fine. However, if they added a new argument to a "public" function (function used outside of the mpt3sas driver) or a new field to a struct used externally from the mpt3sas driver things may APPEAR to work. Every time such function is called or struct is used more and more memory and/or stack gets corrupted. Eventually you will crash something very badly and you will be lucky if that was at least the driver and not something unrelated which was just in the same area. In kernel there's virtually no protections of one piece of code breaking memory of the other. There are small protection for the kernel CODE itself and some sensitive areas but it's not like in the user space where one process trying to write into memory of another process will simply be killed - in the kernel you just corrupt the memory silently.


3) Kernel <=> driver interactions
Scrolling through the questions we saw that some of you load drivers and see missing symbols. Well, this is the effect of 2). When they modify the drivers they often add their own hooks by just placing a f... gSynoFoo() call somewhere in the driver. Honestly we think it's very sloppy but it is how it is. The issue here is two-fold:

  • if you compile a driver from vanilla kernel you will not have these calls so things may look correctly... but these calls were placed there for a reason and other things exepect them to happen. Sometimes the issue may be really small like performance drop or hibernation not working correctly. Sometimes you may step into a data loss with a certain controller which syno fixed in their kernel after getting a possible fix from the manufacturer... you don't know.
  • if you try to add e.g. SAS to a non-SAS model things break badly. This is because it's not as simple as flipping the flag in synoinfo. By doing so you're telling the DSM "this model supports SAS". But the issue here is that models which REALLY support SAS have A LOT of code which is compiled only and only when the kernel config option for SAS was enabled. This config option being disabled literally REMOVED a bunch of code from many places when they compiled the kernel without SAS support. That's also the reason why getting SATA boon on 918+ was so challenging as the SATA-DOM code is literally removed on 918+. Bottom line is don't try to activate functions which depend on the kernel features being present. You can tweak number of disks, supported number of cameras and other user-land features but not re-enable features which are partially kernel and partially userland. It's like trying to pump diesel fuel into gasoline car after drilling a larger hole to fit the bigger diesel nozzle into the tank.

There's no straightforward way to KNOW if the feature you want to activate has a kernel component. However, going through ALL options in synoconfigs/KConfig.* files will give anybody an idea what sort of things can or cannot be activated. In general anything involving in-kernel drivers is a no-no (e.g. iGPU acceleration will not work on 3615xs most likely but NVidia GPU probably will).


4) Data structures missing in toolchains for in-tree drivers
You cannot get the code of a in-tree driver and compile it against the toolchain. This is because in-kernel drivers have access to many more things and link internal kernel files. Should they? Most likely no. However, the kernel is
30 YEARS OLD codebase and only recently they started to encapsulate drivers from using internal things. In most cases it's better to stick to the hardware which is officially supported.


We have v7 sources for geminilake which we took some hints from (e.g. about synobios structures). If there's a particular driver we can look and compare with v6. However, if there's anyone who has experience with drivers (@IG-88?) we can share the archive in a PM as syno pulled it from SF. Why they did pull it? It's possible it contains code from e.g. broadcomm which is under an NDA and should never be published as it's not GPL.

 

Many thanks to @ThorGroup. Really fantastic explanations for noobs like me.

Link to comment
Share on other sites

@ThorGroupBased on your information regarding SAS, there is one thing that I cannot understand right know. With the driver, I can see my 8 disks with fdisk (passthough on vm ESXi 6.5). But with the sda limitation (due to the SATA disk boot), Have only 7 disk seeing. Tried any many combination, and doesn't work. Maybe have to try with sata_remap, but the problem is that the disk is not viewing in the DSM GUI, only by the system (fdisk, can see 8 disk + synoboot). What do you think about baremetal on the same server, with USB boot? Is it solved my problem? With your incredible works, we have now extension, so baremetal is the best solution for my need with driver.  

 

  

On 10/7/2021 at 11:05 AM, RedwinX said:

@pocopicoSame error with SataPortMap=18. And as you can see, your driver load and see the 8 disks

image.thumb.png.ed3715574fd404204084b42743012dbd.png

 

Loading kmod #0 "libphy.ko" for pocopico.tg3 (args: )
[    1.980814] WARNING: module 'libphy' built without retpoline-enabled compiler, may affect Spectre v2 mitigation
Loading kmod #1 "tg3.ko" for pocopico.tg3 (args: )
Loading kmod #0 "mptbase.ko" for pocopico.mptsas (args: )
[    1.990015] Fusion MPT base driver 3.04.20
[    1.990390] Copyright (c) 1999-2008 LSI Corporation
Loading kmod #1 "mptscsih.ko" for pocopico.mptsas (args: )
Loading kmod #2 "mptsas.ko" for pocopico.mptsas (args: )
[    1.995030] Fusion MPT SAS Host driver 3.04.20
Loading kmod #3 "mptctl.ko" for pocopico.mptsas (args: )
[    1.997847] Fusion MPT misc device (ioctl) driver 3.04.20
[    1.998495] mptctl: Registered with Fusion MPT base driver
[    1.999221] mptctl: /dev/mptctl @ (major,minor=10,220)
Loading kmod #0 "mpt2sas.ko" for pocopico.mpt2sas (args: )
[    2.003700] mpt2sas version 14.100.00.00 loaded
[    2.004376] scsi1 : Fusion MPT SAS Host
[    2.006014] mpt2sas0: 64 BIT PCI BUS DMA ADDRESSING SUPPORTED, total mem (65983500 kB)
[    2.007517] mpt2sas 0000:03:00.0: irq 73 for MSI/MSI-X
[    2.008318] mpt2sas 0000:03:00.0: irq 74 for MSI/MSI-X
[    2.009533] mpt2sas 0000:03:00.0: irq 75 for MSI/MSI-X
[    2.010560] mpt2sas 0000:03:00.0: irq 76 for MSI/MSI-X
[    2.011535] mpt2sas 0000:03:00.0: irq 77 for MSI/MSI-X
[    2.012549] mpt2sas 0000:03:00.0: irq 78 for MSI/MSI-X
[    2.013544] mpt2sas 0000:03:00.0: irq 79 for MSI/MSI-X
[    2.014560] mpt2sas 0000:03:00.0: irq 80 for MSI/MSI-X
[    2.016542] mpt2sas0-msix0: PCI-MSI-X enabled: IRQ 73
[    2.017528] mpt2sas0-msix1: PCI-MSI-X enabled: IRQ 74
[    2.018549] mpt2sas0-msix2: PCI-MSI-X enabled: IRQ 75
[    2.019530] mpt2sas0-msix3: PCI-MSI-X enabled: IRQ 76
[    2.020531] mpt2sas0-msix4: PCI-MSI-X enabled: IRQ 77
[    2.021543] mpt2sas0-msix5: PCI-MSI-X enabled: IRQ 78
[    2.022534] mpt2sas0-msix6: PCI-MSI-X enabled: IRQ 79
[    2.023534] mpt2sas0-msix7: PCI-MSI-X enabled: IRQ 80
[    2.024551] mpt2sas0: iomem(0x00000000fd4f0000), mapped(0xffffc90008820000), size(65536)
[    2.026523] mpt2sas0: ioport(0x0000000000004000), size(256)
[    2.170709] mpt2sas0: Allocated physical memory: size(16690 kB)
[    2.171709] mpt2sas0: Current Controller Queue Depth(9979), Max Controller Queue Depth(10240)
[    2.172709] mpt2sas0: Scatter Gather Elements per IO(128)
[    2.202800] tsc: Refined TSC clocksource calibration: 2200.003 MHz
[    2.203758] Switching to clocksource tsc
[    2.234235] mpt2sas0: LSISAS2308: FWVersion(20.00.07.00), ChipRevision(0x05), BiosVersion(00.00.00.00)
[    2.234799] mpt2sas0: 0x1F38: Vendor(0x1000), Device(0x0087), SSVID(0x1028), SSDID(0x1F38)
[    2.235796] mpt2sas0: Protocol=(Initiator,Target), Capabilities=(TLR,EEDP,Snapshot Buffer,Diag Trace Buffer,Task Set Full,NCQ)
[    2.237883] mpt2sas0: sending port enable !!
[    2.242815] mpt2sas0: host_add: handle(0x0001), sas_addr(0x5b82a720d32ad200), phys(8)
[    2.255105] scsi 1:0:0:0: Direct-Access     ATA      ST8000NM0055-1RM112      SN05 PQ: 0 ANSI: 6
[    2.255830] scsi 1:0:0:0: SATA: handle(0x0009), sas_addr(0x4433221100000000), phy(0), device_name(0x5000c500c39d2d7a)
[    2.257836] scsi 1:0:0:0: SATA: enclosure_logical_id(0x5b82a720d32ad200), slot(3)
[    2.258979] scsi 1:0:0:0: atapi(n), ncq(y), asyn_notify(n), smart(y), fua(y), sw_preserve(y)
[    2.260847] scsi 1:0:0:0: qdepth(32), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1)
[    2.265442] sd 1:0:0:0: [sdb] 15628053168 512-byte logical blocks: (8.00 TB/7.27 TiB)
[    2.266804] sd 1:0:0:0: [sdb] 4096-byte physical blocks
[    2.268292] scsi 1:0:1:0: Direct-Access     ATA      ST8000VN0022-2EL112      SC61 PQ: 0 ANSI: 6
[    2.268843] scsi 1:0:1:0: SATA: handle(0x000a), sas_addr(0x4433221101000000), phy(1), device_name(0x5000c500b62bce05)
[    2.270865] scsi 1:0:1:0: SATA: enclosure_logical_id(0x5b82a720d32ad200), slot(2)
[    2.271930] scsi 1:0:1:0: atapi(n), ncq(y), asyn_notify(n), smart(y), fua(y), sw_preserve(y)
[    2.273908] scsi 1:0:1:0: qdepth(32), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1)
[    2.274900] sd 1:0:0:0: [sdb] Write Protect is off
[    2.276304] sd 1:0:0:0: [sdb] Mode Sense: 7f 00 10 08
[    2.276838] usb 2-1: new full-speed USB device number 2 using uhci_hcd
[    2.277377] sd 1:0:1:0: [sdc] 15628053168 512-byte logical blocks: (8.00 TB/7.27 TiB)
[    2.277380] sd 1:0:1:0: [sdc] 4096-byte physical blocks
[    2.280446] sd 1:0:0:0: [sdb] Write cache: enabled, read cache: enabled, supports DPO and FUA
[    2.286954] scsi 1:0:2:0: Direct-Access     ATA      ST8000VN0022-2EL112      SC61 PQ: 0 ANSI: 6
[    2.288913] scsi 1:0:2:0: SATA: handle(0x000b), sas_addr(0x4433221102000000), phy(2), device_name(0x5000c500b61f7abb)
[    2.290895] scsi 1:0:2:0: SATA: enclosure_logical_id(0x5b82a720d32ad200), slot(1)
[    2.291931] sd 1:0:1:0: [sdc] Write Protect is off
[    2.292002] scsi 1:0:2:0: atapi(n), ncq(y), asyn_notify(n), smart(y), fua(y), sw_preserve(y)
[    2.292006] scsi 1:0:2:0: qdepth(32), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1)
[    2.294354] sd 1:0:2:0: [sdd] 15628053168 512-byte logical blocks: (8.00 TB/7.27 TiB)
[    2.294356] sd 1:0:2:0: [sdd] 4096-byte physical blocks
[    2.297985] sd 1:0:1:0: [sdc] Mode Sense: 7f 00 10 08
[    2.298035] scsi 1:0:3:0: Direct-Access     ATA      ST8000VN004-2M2101       SC60 PQ: 0 ANSI: 6
[    2.298040] scsi 1:0:3:0: SATA: handle(0x000c), sas_addr(0x4433221103000000), phy(3), device_name(0x5000c500c02f0786)
[    2.298043] scsi 1:0:3:0: SATA: enclosure_logical_id(0x5b82a720d32ad200), slot(0)
[    2.298142] scsi 1:0:3:0: atapi(n), ncq(y), asyn_notify(n), smart(y), fua(y), sw_preserve(y)
[    2.298146] scsi 1:0:3:0: qdepth(32), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1)
[    2.300060] sd 1:0:3:0: [sde] 15628053168 512-byte logical blocks: (8.00 TB/7.27 TiB)
[    2.300062] sd 1:0:3:0: [sde] 4096-byte physical blocks
[    2.305022] sd 1:0:3:0: [sde] Write Protect is off
[    2.305026] sd 1:0:3:0: [sde] Mode Sense: 7f 00 10 08
[    2.306970] sd 1:0:3:0: [sde] Write cache: enabled, read cache: enabled, supports DPO and FUA
[    2.308019] scsi 1:0:4:0: Direct-Access     ATA      ST8000NM0055-1RM112      SN05 PQ: 0 ANSI: 6
[    2.308025] scsi 1:0:4:0: SATA: handle(0x000d), sas_addr(0x4433221104000000), phy(4), device_name(0x5000c500b6aa83ab)
[    2.308028] scsi 1:0:4:0: SATA: enclosure_logical_id(0x5b82a720d32ad200), slot(7)
[    2.308126] scsi 1:0:4:0: atapi(n), ncq(y), asyn_notify(n), smart(y), fua(y), sw_preserve(y)
[    2.308130] scsi 1:0:4:0: qdepth(32), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1)
[    2.310602] sd 1:0:4:0: [sdf] 15628053168 512-byte logical blocks: (8.00 TB/7.27 TiB)
[    2.310605] sd 1:0:4:0: [sdf] 4096-byte physical blocks
[    2.312559] sd 1:0:2:0: [sdd] Write Protect is off
[    2.312562] sd 1:0:2:0: [sdd] Mode Sense: 7f 00 10 08
[    2.314956] sd 1:0:2:0: [sdd] Write cache: enabled, read cache: enabled, supports DPO and FUA
[    2.321978] scsi 1:0:5:0: Direct-Access     ATA      ST8000NM0055-1RM112      SN05 PQ: 0 ANSI: 6
[    2.322925] scsi 1:0:5:0: SATA: handle(0x000e), sas_addr(0x4433221105000000), phy(5), device_name(0x5000c500c3a145a0)
[    2.323891] scsi 1:0:5:0: SATA: enclosure_logical_id(0x5b82a720d32ad200), slot(6)
[    2.324008] scsi 1:0:5:0: atapi(n), ncq(y), asyn_notify(n), smart(y), fua(y), sw_preserve(y)
[    2.324946] scsi 1:0:5:0: qdepth(32), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1)
[    2.327470] sd 1:0:4:0: [sdf] Write Protect is off
[    2.328732] sd 1:0:4:0: [sdf] Mode Sense: 7f 00 10 08
[    2.329951] sd 1:0:5:0: [sdg] 15628053168 512-byte logical blocks: (8.00 TB/7.27 TiB)
[    2.329954] sd 1:0:5:0: [sdg] 4096-byte physical blocks
[    2.329962] sd 1:0:1:0: [sdc] Write cache: enabled, read cache: enabled, supports DPO and FUA
[    2.333446]  sdb: sdb1 sdb2 sdb5
[    2.336891] sd 1:0:4:0: [sdf] Write cache: enabled, read cache: enabled, supports DPO and FUA
[    2.341825] scsi 1:0:6:0: Direct-Access     ATA      ST8000NM0055-1RM112      SN05 PQ: 0 ANSI: 6
[    2.343280] scsi 1:0:6:0: SATA: handle(0x000f), sas_addr(0x4433221106000000), phy(6), device_name(0x5000c500c3a38bc5)
[    2.345269] scsi 1:0:6:0: SATA: enclosure_logical_id(0x5b82a720d32ad200), slot(5)
[    2.346736] scsi 1:0:6:0: atapi(n), ncq(y), asyn_notify(n), smart(y), fua(y), sw_preserve(y)
[    2.348313] scsi 1:0:6:0: qdepth(32), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1)
[    2.349410]  sde: sde1 sde2 sde5
[    2.353124] sd 1:0:5:0: [sdg] Write Protect is off
[    2.353728] sd 1:0:6:0: [sdh] 15628053168 512-byte logical blocks: (8.00 TB/7.27 TiB)
[    2.353731] sd 1:0:6:0: [sdh] 4096-byte physical blocks
[    2.357445] sd 1:0:5:0: [sdg] Mode Sense: 7f 00 10 08
[    2.362258] sd 1:0:5:0: [sdg] Write cache: enabled, read cache: enabled, supports DPO and FUA
[    2.369020] sd 1:0:3:0: [sde] Attached SCSI disk
[    2.372026] sd 1:0:0:0: [sdb] Attached SCSI disk
[    2.372990] scsi 1:0:7:0: Direct-Access     ATA      ST8000VN0022-2EL112      SC61 PQ: 0 ANSI: 6
[    2.375011] scsi 1:0:7:0: SATA: handle(0x0010), sas_addr(0x4433221107000000), phy(7), device_name(0x5000c500b6225371)
[    2.376041] scsi 1:0:7:0: SATA: enclosure_logical_id(0x5b82a720d32ad200), slot(4)
[    2.376345] sd 1:0:6:0: [sdh] Write Protect is off
[    2.376349] sd 1:0:6:0: [sdh] Mode Sense: 7f 00 10 08
[    2.377878]  sdd: sdd1 sdd2 sdd5
[    2.379273] scsi 1:0:7:0: atapi(n), ncq(y), asyn_notify(n), smart(y), fua(y), sw_preserve(y)
[    2.379420] sd 1:0:6:0: [sdh] Write cache: enabled, read cache: enabled, supports DPO and FUA
[    2.383307] scsi 1:0:7:0: qdepth(32), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1)
[    2.387332] sd 1:0:7:0: [sdi] 15628053168 512-byte logical blocks: (8.00 TB/7.27 TiB)
[    2.388625] sd 1:0:7:0: [sdi] 4096-byte physical blocks
[    2.393969] mpt2sas0: port enable: SUCCESS


As you can see, the fdisk -l see the 8 disk : 

 

root@FRAHQFIL503:~# fdisk -l
Disk /dev/synoboot: 128 MiB, 134217728 bytes, 262144 sectors
Disk model: TYPE D 3SE
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0xf110ee87

Device         Boot  Start    End Sectors Size Id Type
/dev/synoboot1        2048 100351   98304  48M 83 Linux
/dev/synoboot2      100352 253951  153600  75M 83 Linux
/dev/synoboot3      253952 262143    8192   4M 83 Linux


Disk /dev/sdb: 7.3 TiB, 8001563222016 bytes, 15628053168 sectors
Disk model: ST8000NM0055-1RM112
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: 67FC142E-D83C-11EB-90D4-B496916F9E98

Device       Start         End     Sectors  Size Type
/dev/sdb1     2048     4982527     4980480  2.4G Linux RAID
/dev/sdb2  4982528     9176831     4194304    2G Linux RAID
/dev/sdb5  9453280 15627846239 15618392960  7.3T Linux RAID


Disk /dev/sde: 7.3 TiB, 8001563222016 bytes, 15628053168 sectors
Disk model: ST8000VN004-2M2101
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: 6C43C76F-D83C-11EB-90D4-B496916F9E98

Device       Start         End     Sectors  Size Type
/dev/sde1     2048     4982527     4980480  2.4G Linux RAID
/dev/sde2  4982528     9176831     4194304    2G Linux RAID
/dev/sde5  9453280 15627846239 15618392960  7.3T Linux RAID


Disk /dev/sdd: 7.3 TiB, 8001563222016 bytes, 15628053168 sectors
Disk model: ST8000VN0022-2EL112
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: 67126838-D83C-11EB-90D4-B496916F9E98

Device       Start         End     Sectors  Size Type
/dev/sdd1     2048     4982527     4980480  2.4G Linux RAID
/dev/sdd2  4982528     9176831     4194304    2G Linux RAID
/dev/sdd5  9453280 15627846239 15618392960  7.3T Linux RAID


Disk /dev/sdc: 7.3 TiB, 8001563222016 bytes, 15628053168 sectors
Disk model: ST8000VN0022-2EL112
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: 6B62881B-D83C-11EB-90D4-B496916F9E98

Device       Start         End     Sectors  Size Type
/dev/sdc1     2048     4982527     4980480  2.4G Linux RAID
/dev/sdc2  4982528     9176831     4194304    2G Linux RAID
/dev/sdc5  9453280 15627846239 15618392960  7.3T Linux RAID


Disk /dev/sdf: 7.3 TiB, 8001563222016 bytes, 15628053168 sectors
Disk model: ST8000NM0055-1RM112
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: 6BAD6B80-D83C-11EB-90D4-B496916F9E98

Device       Start         End     Sectors  Size Type
/dev/sdf1     2048     4982527     4980480  2.4G Linux RAID
/dev/sdf2  4982528     9176831     4194304    2G Linux RAID
/dev/sdf5  9453280 15627846239 15618392960  7.3T Linux RAID


Disk /dev/sdg: 7.3 TiB, 8001563222016 bytes, 15628053168 sectors
Disk model: ST8000NM0055-1RM112
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: 6BBA0433-D83C-11EB-90D4-B496916F9E98

Device       Start         End     Sectors  Size Type
/dev/sdg1     2048     4982527     4980480  2.4G Linux RAID
/dev/sdg2  4982528     9176831     4194304    2G Linux RAID
/dev/sdg5  9453280 15627846239 15618392960  7.3T Linux RAID


Disk /dev/sdh: 7.3 TiB, 8001563222016 bytes, 15628053168 sectors
Disk model: ST8000NM0055-1RM112
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: 69062E8B-D83C-11EB-90D4-B496916F9E98

Device       Start         End     Sectors  Size Type
/dev/sdh1     2048     4982527     4980480  2.4G Linux RAID
/dev/sdh2  4982528     9176831     4194304    2G Linux RAID
/dev/sdh5  9453280 15627846239 15618392960  7.3T Linux RAID


Disk /dev/sdi: 7.3 TiB, 8001563222016 bytes, 15628053168 sectors
Disk model: ST8000VN0022-2EL112
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: 69DAB975-D83C-11EB-90D4-B496916F9E98

Device       Start         End     Sectors  Size Type
/dev/sdi1     2048     4982527     4980480  2.4G Linux RAID
/dev/sdi2  4982528     9176831     4194304    2G Linux RAID
/dev/sdi5  9453280 15627846239 15618392960  7.3T Linux RAID

 

Edited by RedwinX
Link to comment
Share on other sites

8 minutes ago, RedwinX said:

@ThorGroupBased on your information regarding SAS, there is one thing that I cannot understand right know. With the driver, I can see my 8 disks with fdisk (passthough on vm ESXi 6.5). But with the sda limitation (due to the SATA disk boot), Have only 7 disk seeing. Tried any many combination, and doesn't work. Maybe have to try with sata_remap, but the problem is that the disk is not viewing in the DSM GUI, only by the system (fdisk, can see 8 disk + synoboot). What do you think about baremetal on the same server, with USB boot? Is it solved my problem? With your incredible works, we have now extension, so baremetal is the best solution for my need with driver.  

 

  

 

have you previously posted an lspci -k ? 

Link to comment
Share on other sites

2 hours ago, taiziccf said:

 

OrpheeGT repo do not have virtio for 7.0.1-42218.

 

you can use my repo if you want

 


        {
            "id": "apollolake-7.0.1-42218",
            "platform_version": "apollolake-7.0.1-42218",
            "user_config_json": "apollolake_user_config.json",
            "docker_base_image": "debian:10-slim",
            "compile_with": "toolkit_dev",
            "redpill_lkm_make_target": "prod-v7",
            "downloads": {
                "kernel": {
                    "url": "https://sourceforge.net/projects/dsgpl/files/Synology%20NAS%20GPL%20Source/25426branch/apollolake-source/linux-4.4.x.txz/download",
                    "sha256": "af815ee065775d2e569fd7176e25c8ba7ee17a03361557975c8e5a4b64230c5b"
                },
                "toolkit_dev": {
                    "url": "https://sourceforge.net/projects/dsgpl/files/toolkit/DSM7.0/ds.apollolake-7.0.dev.txz/download",
                    "sha256": "d349fa644392d4cfab8191243ee38aaa32bd517208c144678e0c855cb5a619ea"
                }
            },
            "redpill_lkm": {
                "source_url": "https://github.com/RedPill-TTG/redpill-lkm.git",
                "branch": "master"
            },
            "redpill_load": {
                "source_url": "https://github.com/chchia/redpill-load.git",
                "branch": "develop"
            }
        } 

 

 

It's the same as mine.
It was made well a week ago.

But there's an error right now.

Try making it now.

Link to comment
Share on other sites

7 hours ago, ThorGroup said:

So a quick up-not-to-date ;)

 

You guys are writing so many post and we're committed to reading all of them and responding where needed.

We promised to write something more last time but few of us had slightly unpredictable life events - given the fact we usually get together physically we encounter a small delay.

 

So, regarding the SAS issue: the sas-activator may be a flop. We tested it with SSDs and few spare HDDs which obviously weren't >2TB. It seems like the driver syno ships is ancient and while it does work it only runs using SAS2 (limit to 2TB is one of the symptoms). Don't we all love that ancient kernel? ;) Probably on systems with mpt3sas the situation isn't a problem.

 

As to the native SAS support and how it works: in short it's useless. The SAS support in syno is tied to just a few hardcoded models. Most of them support only external ports (via eboxes). Some, like in a few FS* products support internal ports. Unfortunately, this is completely useless as the OS checks for custom firmware, board names, responses etc. Is it possible to emulate that? Yes. Does it make sense? In our opinion not at all. With the community-provided drivers we can all be fully independent of the syno-provided .ko in the image. As the LKM supports SAS-to-SATA emulation for the purpose of syno APIs thinking it's a "SATA" device further investment into true SAS doesn't really make sense. To be clear here: there's no loss of functionality or anything. Many things in the kernel actually use the "SATA" type. It seems like syno intended it to be a true SATA but down the road changed it to really mean "a normal SCSI-based disk".

 

 

 

Presumably there's something in the external SAS connectivity spaghetti code that allows for more than 26 drives - maybe that's a reason to emulate?  In theory it'd allow people to use external SAS trays as well (although I'm unclear on whether emulation is actually needed to support that or not).

 

Curious if anyone actually has a functioning native synology system with more than 26 drives to donate to unwinding the mess.

 

Link to comment
Share on other sites

1 hour ago, pocopico said:

have you previously posted an lspci -k ? 

yep This is what I had : 

0000:02:00.0 Class 0c03: Device 15ad:0774
        DeviceName: usb
        Subsystem: Device 15ad:1976
        Kernel driver in use: uhci_hcd
0000:02:02.0 Class 0106: Device 15ad:07e0
        DeviceName: sata0
        Subsystem: Device 15ad:07e0
        Kernel driver in use: ahci
0000:02:03.0 Class 0c03: Device 15ad:0770
        DeviceName: ehci
        Subsystem: Device 15ad:0770
        Kernel driver in use: ehci-pci
0000:03:00.0 Class 0107: Device 1000:0087 (rev 05)
        DeviceName: pciPassthru0
        Subsystem: Device 1028:1f38
        Kernel driver in use: mpt2sas
0000:0b:00.0 Class 0200: Device 8086:10d3
        DeviceName: Ethernet1
        Subsystem: Device 15ad:07d0
        Kernel driver in use: e1000e
0000:13:00.0 Class 0200: Device 15ad:07b0 (rev 01)
        DeviceName: Ethernet0
        Subsystem: Device 15ad:07b0
0001:07:00.0 Class 0106: Device 1b4b:9235 (rev 11)
0001:08:00.0 Class 0106: Device 1b4b:9235 (rev 11)
0001:09:00.0 Class 0106: Device 1b4b:9235 (rev 11)
0001:0a:00.0 Class 0106: Device 1b4b:9235 (rev 11)

Link to comment
Share on other sites

So I just booted RedPill DS3615XS 7.0.1 42218 , as baremetal on my HP G8 MS , with onboard NIC , both ports ,and LSI card in IT mode

used @pocopico tg3 and mpt2sas . 

 

(It did a migration , spare drive I used must have already had a 6.x build installed) 

 

both network ports are good . 

HDD on LSI detected, smart working etc . 

 

like @RedwinX the first first drive take sdb rather than sda , not sure if this would be an issue other than cosmetic , I assume everything would just be moved up so 8th drive would become sdi rather than sdh ? 

 

I'm going to convert the image to VMDK now and try passthrough on the card on Esxi 

 

Heres current output of lspci -k and disk -l 

 

root@RedPillDS3615XS:~# fdisk -l
Disk /dev/sdb: 232.9 GiB, 250059350016 bytes, 488397168 sectors
Disk model: VB0250EAVER             
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0xd8b10283

Device     Boot   Start       End   Sectors   Size Id Type
/dev/sdb1          2048   4982527   4980480   2.4G fd Linux raid autodetect
/dev/sdb2       4982528   9176831   4194304     2G fd Linux raid autodetect
/dev/sdb3       9437184 488192351 478755168 228.3G fd Linux raid autodetect


Disk /dev/synoboot: 2 GiB, 2118647808 bytes, 4137984 sectors
Disk model: Flash Disk              
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0xf110ee87

Device         Boot  Start    End Sectors Size Id Type
/dev/synoboot1 *      2048 100351   98304  48M 83 Linux
/dev/synoboot2      100352 253951  153600  75M 83 Linux
/dev/synoboot3      253952 262143    8192   4M 83 Linux


Disk /dev/md0: 2.4 GiB, 2549940224 bytes, 4980352 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/zram0: 2.4 GiB, 2526019584 bytes, 616704 sectors
root@RedPillDS3615XS:~# lspci -k
0000:00:00.0 Class 0600: Device 8086:0158 (rev 09)
0000:00:01.0 Class 0604: Device 8086:0151 (rev 09)
	Kernel driver in use: pcieport
0000:00:06.0 Class 0604: Device 8086:015d (rev 09)
	Kernel driver in use: pcieport
0000:00:1a.0 Class 0c03: Device 8086:1c2d (rev 05)
	Subsystem: Device 103c:330d
	Kernel driver in use: ehci-pci
0000:00:1c.0 Class 0604: Device 8086:1c10 (rev b5)
	Kernel driver in use: pcieport
0000:00:1c.4 Class 0604: Device 8086:1c18 (rev b5)
	Kernel driver in use: pcieport
0000:00:1c.6 Class 0604: Device 8086:1c1c (rev b5)
	Kernel driver in use: pcieport
0000:00:1c.7 Class 0604: Device 8086:1c1e (rev b5)
	Kernel driver in use: pcieport
0000:00:1d.0 Class 0c03: Device 8086:1c26 (rev 05)
	Subsystem: Device 103c:330d
	Kernel driver in use: ehci-pci
0000:00:1e.0 Class 0604: Device 8086:244e (rev a5)
0000:00:1f.0 Class 0601: Device 8086:1c54 (rev 05)
	Kernel driver in use: lpc_ich
0000:01:00.0 Class 0880: Device 103c:3306 (rev 05)
	Subsystem: Device 103c:3381
0000:01:00.1 Class 0300: Device 102b:0533
	Subsystem: Device 103c:3381
0000:01:00.2 Class 0880: Device 103c:3307 (rev 05)
	Subsystem: Device 103c:3381
0000:01:00.4 Class 0c03: Device 103c:3300 (rev 02)
	Subsystem: Device 103c:3381
	Kernel driver in use: uhci_hcd
0000:03:00.0 Class 0200: Device 14e4:165f
	DeviceName: NIC Port 1
	Subsystem: Device 103c:2133
	Kernel driver in use: tg3
0000:03:00.1 Class 0200: Device 14e4:165f
	DeviceName: NIC Port 2
	Subsystem: Device 103c:2133
	Kernel driver in use: tg3
0000:04:00.0 Class 0c03: Device 1912:0014 (rev 03)
	Subsystem: Device 103c:1996
	Kernel driver in use: xhci_hcd
0000:07:00.0 Class 0107: Device 1000:0087 (rev 05)
	DeviceName: Storage Controller
	Subsystem: Device 1590:0041
	Kernel driver in use: mpt2sas
0001:07:00.0 Class 0106: Device 1b4b:9235 (rev 11)
	DeviceName: Storage Controller
0001:08:00.0 Class 0106: Device 1b4b:9235 (rev 11)
0001:09:00.0 Class 0106: Device 1b4b:9235 (rev 11)
0001:0a:00.0 Class 0106: Device 1b4b:9235 (rev 11)

 

1789352370_Untitled2.thumb.jpg.8dcea6cfc08dd5dda5a224b5566af9c2.jpg

Link to comment
Share on other sites

13 minutes ago, Mixpower said:

You are using a SAS 2008?

With my 9207-4i4e my disks are detect on boot but DSM only sees one disk in the end.

Is it flashed in IT mode?

 

Thanks!

yes i flashed in IT mode.

It never happened to me either no drive detected or all drives detected. Mine is dell h310 card

Link to comment
Share on other sites

4 minutes ago, psychoboi32 said:

yes i flashed in IT mode.

It never happened to me either no drive detected or all drives detected. Mine is dell h310 card

Okay great, weird mine does not work.

With the b120i in AHCI mode it works fine though, guess I'll be looking for a new HBA.

Link to comment
Share on other sites

1 minute ago, Mixpower said:

Okay great, weird mine does not work.

With the b120i in AHCI mode it works fine though, guess I'll be looking for a new HBA.

oh I didn't know about that, I bypassed internal SAS port by adding PCI-E card. and remember you need new cable for dl380g8p because there are angled one

Link to comment
Share on other sites

1 minute ago, psychoboi32 said:

@WiteWulf you own g8 hp server I want to ask can i get serial output on my ilo v4 web console that will be huge help for me thanks

No, you only get textcons on the web interface.

 

For serial output you need to ssh into the iLO and run 'vsp'

 

I can't remember if you need a full iLO license for this or not, but it's worth having anyway and you can get them for ~£10 on eBay.

Link to comment
Share on other sites

2 minutes ago, psychoboi32 said:

oh I didn't know about that, I bypassed internal SAS port by adding PCI-E card. and remember you need new cable for dl380g8p because there are angled one

Thanks, I'm using a Gen8 Microserver with a LSI 9207-4i4e card and all 4 disks are detected when the loader is booting but in DSM only one disk is available.

So for now I am using the B120i in AHCI and now I have 4 disks working.

Link to comment
Share on other sites

1 minute ago, WiteWulf said:

No, you only get textcons on the web interface.

 

For serial output you need to ssh into the iLO and run 'vsp'

 

I can't remember if you need a full iLO license for this or not, but it's worth having anyway and you can get them for ~£10 on eBay.

I have full lic. my mine bios is old not like G9 where RBSU come like laptop bios, so I have to find little deep thanks now I don't need serial cable

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...