DSM 6.1.x Loader

[jun]

Loader Download: here / (link updated due to directory structure change of mega share) -- by jun 2-28

I've uploaded a 6.1 alpha build to mega, for brave guys only . (Note: AMD needs extra work.)

scripts I use to generate bootable image

https://github.com/kref/scripts

 

 

---Beginning of addition by polanskiman---

Tutorials by polanskiman:

Install/Upgrade DSM 5.2 to DSM 6.0.2 here

Install/Upgrade DSM 5.2 to DSM 6.1 here

 

Loader download mirrors:

Managed by Polanskiman. includes .iso format images made by myself. here

Managed by Trantor. This mirror is currently offline. here

 

For info:

v1.01 (DS3615xs) is for DSM 6.0.2 - AMD compatible - Latest version for DSM 6.0.2

v1.02b (DS3615xs, DS3617xs and DS916+) is for DSM 6.1 - AMD loosely compatible and with Bios tweaks - Latest version for DSM 6.1

 

Spoiler

v1.02a (DS3615xs) is for DSM 6.1 - AMD not compatible - Deprecated
v1.02a (DS3617xs and DS916+) is for DSM 6.1 - AMD not compatible - Deprecated - ('time bomb' bug fixed in v1.02a2)
v1.02a2 (DS3617xs and DS916+) is for DSM 6.1 - AMD not compatible - Deprecated

 

DO NOT install DSM updates on your working machine BEFORE looking into the forum or testing the update on a test rig/vm.

DO NOT update to DSM 6.2 with loader v.1.02b. It will not work. I repeat. DO NOT update to DSM 6.2

 

Further explanation by jun on AMD compatibly for loader v1.02a, v1.02a2 and v1.02b:

On 7/6/2017 at 5:47 AM, jun said:

please see arch/x86/kernel/cpu/amd.c, it contains all kinds of errata workaround and feature detections. synology kernel does not have it compiled in. will amd cpu work without it? i dont know for sure, seems some works, some does not, and some requires workaround.

---End of addition by polanskiman---

 

 

Spoiler

I've recently annouce that my work on a kernel mode dynamic patcher in this post

https://xpenology.com/forum/topic/6213-xpenology-developers-contact-me-for-7393-kernel-src-code/?do=findComment&comment=54105

 

Now seems everything work out of box on VMware, so it's pretty close to an everybody useable loader,

But lots of hardwares drivers are missing, to support as much hardware as possible requires some extra work,

So I hope these is some help from the community to built & test drivers, make it usable for everyone.

 

AMD users have a look 3rd Post.

 

 

I've upload a tar archive of the boot partion, it should be untar in to a usb drive's EFI System Partition.

I prefers EFI bootloader, so I used grub2-efi here, but you can install your prefered ones, even none-EFI ones.

the zImage is a vanilla synology kernel, ramdisk.lzma is repacked to add some network drivers plus my patcher.

you need change sn/mac/vid/pid as usual before installation.

 

and to make installer happy, usb drive seem should have at least 2 partitions, one for boot, and one for recovery I guess.

 

I suggest to test it on VM first, then add hardware drivers for boot on bare metals.

 

Below is how my usb drive looks like

 

$ sudo gdisk -l /dev/sdc

GPT fdisk (gdisk) version 0.8.8

 

Partition table scan:

MBR: protective

BSD: not present

APM: not present

GPT: present

 

Found valid GPT with protective MBR; using GPT.

Disk /dev/sdc: 32784384 sectors, 15.6 GiB

Logical sector size: 512 bytes

Disk identifier (GUID): 4308A165-B0A6-4EC9-8B8E-9BEFB00BF33F

Partition table holds up to 128 entries

First usable sector is 34, last usable sector is 32784350

Partitions will be aligned on 2048-sector boundaries

Total free space is 4061 sectors (2.0 MiB)

 

Number Start (sector) End (sector) Size Code Name

1 2048 31457280 15.0 GiB 8300 Linux filesystem

2 31459328 31868927 200.0 MiB 8300 Linux filesystem

3 31868928 32784350 447.0 MiB EF00 EFI System

 

 

$ mount|grep tmp1

/dev/sdc3 on /mnt/tmp1 type vfat (rw)

 

 

$ ls -l /mnt/tmp1/

total 24

drwxr-xr-x 3 root root 8192 Sep 18 17:19 EFI

drwxr-xr-x 5 root root 8192 Sep 20 21:16 grub

drwxr-xr-x 3 root root 8192 Sep 20 21:15 image

 

$ lsusb |grep Flash

Bus 001 Device 010: ID 058f:6387 Alcor Micro Corp. Flash Drive

 

link

https://mega.nz/#!c8kn3JAL!fy4a0sXuYfcv1Sx7adGbr2OS-guqMS4GLRH3TOFkNfY

 

Edit:

I add a few popular network & disk drivers, and create a boot image for VM.

Just add some scsi or sata disk, then boot, and follow normal installation process.

https://mega.nz/#!llU23QgA!oGaZQuQLe0fsx1tuGiSBoZ19DSfEQH5Aq3GJXfEjpHE

 

for esxi, pls use this link

https://mega.nz/#!Agt00ZDT!lN5blyZc4gMaj-mKt-xCxoTtle-7cLF0ui9bF6ZBnY0

 

Plus a usb boot image for bare metal in UEFI mode. pls consider it unstable, test at your own risk.

https://mega.nz/#!Z4UzWZxZ!hu9SZXpzJBTakHk7lO8pJfyHYb-YgGatI-Em9E3JXWI

 

EditEdit:

 

Many people reported that DSM ask you to reinstall forever.

It turn out that lack of a serial port caused the failure. That is really a surprise .

Anyway, I upload a new ramdisk to workaround the issue.

You should replace the image/DS3615xs/ramdisk.lzma in the boot partiton with this new one.

Edited post

The esxi and bare metal image are updated, so manually replace the ramdisk is not required.

Alternatively, VM users may add a serial port as a workaround.

Bare metal users can enable the serial port in bios if your board has one.

 

remaining problem:

Login to your DSM via ssh/telnet, your will see these messages in kernel log,

this is another consequence of lack of a serial console.

[ 1360.575754] init: tty main process (13856) terminated with status 1
[ 1360.575823] init: tty main process ended, respawning
[ 1370.591635] init: tty main process (13881) terminated with status 1
[ 1370.591755] init: tty main process ended, respawning

 

 

To stop it from spam your log, you can stop/disable the tty serivce.

 

update:

above workaround is not enough for boards without serial port,

these unlucky guys should edit following serivce config manually, replace "console output" with "console none" after installation

root@test:~# grep -r "^console output$" /usr/ 2> /dev/null
/usr/share/init/syno_poweroff_task.conf:console output
/usr/share/init/burnin_loader.conf:console output
/usr/share/init/udevtrigger.conf:console output
/usr/share/init/bs-poweroff.conf:console output
/usr/share/init/udevd.conf:console output

 

 

Another issue:

I've found a hidden check related to console,

everyone should edit grub.cfg, replace console=uart8250,io,0x3f8,115200n8 with console=ttyS0,115200n8

You have to change sn/mac/vid/pid as usual before installation !

If you don't edit vid/pid in grub.conf you will get error 13 (file corruption) during setup.

 

How to see the value of my VID & PID stick

 

 

EDIT by Trantor (10-28-2016):

 

Updated loader by Arcao latest images (10-28-2016) : https://mega.nz/#F!Oc8TCLgD!IiullNuGs95RlelM9SKd5w

Original post : https://xpenology.com/forum/topic/6253-dsm-602-loader-was-a-new-loader-for-latest-dsm/?do=findComment&comment=55903

 

Sources for dev:

quicknick : https://github.com/quiknick/7274-kernel

setsunakawa : http://setsunakawa.ddns.net/synology/dsm6/

 

 

10/29:

It's time to release a loader with a proper version number now.

Here is my loader v1.0

https://mega.nz/#F!18kB1BTB!1ft3N5Hnrcnqsneu0aQUkA

 

New features & fix:

 

Support bios & uefi boot & VMware & ESXI

Fix cmos reset issue( at least on bios mode VMware workstation)

most configurations can be done via grub command line now.

Remember last boot entry.

Autodetect synoboot partitions, not extensively tested. It is still recommended to set correct vid/pid.

AMD CPU support, not extensively tested, there are a few positive report on the forum.

Add grub menu for reinstallation.

Add grub menu to force installation when interrupted due to "Disks Disabled" message, not extensively tested

Automatically apply workaround for boards without serial port.

Fix SATA disk order remap, and allow to set SAS disk starting no, but disk order are still not deterministic between SAS disks.

Fix SAS disk naming, not extensively tested, only with one SAS2008 based HBA and synology provided mpt2sas driver.

Add ata_piix support, tested on VMware and it does not crash, only complain about "Read current link speed fail", so I

assume it works , this driver requires a bit of patching to the kernel, so I include it in the package.

 

Integrate up-to-date network drivers compiled by Arcao and Trantor.

 

 

Known issue:

When runing on a slow single core machine, there is a race condition causes the patcher loading too late.

the most obvious sign is console not working properly. I don't expected this to happen normally,

the only example is a VM on my laptop when running on battery. If anyone believes have this issue,

please send me a message.

 

Some ethernet drivers crash when set MTU above about 4096(Jumbo frame), I've not identified the culprit yet.

 

 

How to install:

 

find the vid/pid of your usb drive

Burn the image to the drive, optionally edit grub.cfg to set vid/pid and boot entry, OR

connect monitor/keyboard/serial console to your Box.

Press C during Grub boot menu to enter command line

vid 0xXXXX
pid 0xYYYY

 

to configure pid and vid ("pid" and "vid" are command, and "XXXX", "YYYY" are you usb drive's vid,pid)

(There are a few other options can be config this way, you can read grub.cfg to know more detail.)

Press Esc to go back to boot menu

Select baremeta/baremetal AMD entry, press enter to boot.

If your encounter the "disk port disabled" error during installation, try the force install menu entry.

if both option does not work, you have to fallback to SataPortMap method,

type this line in grub command line, replace XX with appropriate value.

append SataPortMap=XX

 

 

11/2:

as I promised, here is v1.01, download link is the same as v1.0

This update contains nothing fancy, mainly for inner peace,

Plus, I revert megaraid_sas mpt3sas to old version.

 

It's not shining new now, the title is obsolete

 

Edited August 6, 2018 by Polanskiman
Updated spoiler plugin code

[Aigor]

Cool Job, if i wanted to try, is tere some possbilities to have your software?

[jun]

10/28:

 

Recently, I've implenmented an advanced patching method, its interesting in principle.

So I utilize it to add AMD cpu support, hope it works . This is the updated zImage, please test and send your feedback.

https://mega.nz/#!Vk0E2RrL!eFRPWcnaXn9b ... Sihf5BTRG0

Edited October 28, 2016 by Guest

[WizP]

An ISO of this would be perfect!

[Hostilian]

Nice work!

+1 for ISO. Would be handy, thanks!

[LuckyNumber77]

Hy, im new in this Topic..., so sorry for my question.

Is it now possible to get a running DSM 6.0 VM with this tgz file?

When yes, maybe some one can tell me how?

 

thanks

lucky

[quicknick]

compiling new kernel w/drivers now. Took snippets from Jun and will see how it runs in a few minutes.

[Aigor]

Maybe a dumb question, if i wanted compile my own set of drivers i need to have kernel source of kernel version running, so i need kernel version from synology.

Do i miss something?

[koalala]

Hi jun, could you provide me your loader and the kernel sources please ? I'll try on ESXI and on bare metal.

Do you have a github repo for this loader ?

 

Kind regards

[HellG]

Hi jun,

you seem to know what you are doing so maybe you could help me to understand the protection mechanisms of the Synology DSM 6.

I have a DS710+ (2 bay Intel Atom D410) which does not get any further updates beyond DSM 5.x (5592). I was able to create a pat file to install DSM 6.0 (7321) by just changing the update file for the DS712+ (2 bay Intel Atom D425). I just changed the synoinfo.conf, scemd.conf and manutil.conf according to what I found in the latest DS710+ update and after repacking I was able to update my box to DSM 6.0 version 7321 (and also the subsequent updates to patchlevel 7).

With the same method I was able to create an update file for DSM 6.0.2 version 8451, however now I have the same problems as others reportet here that have accidentially updated to 6.0.1 (7393) or 6.0.2 (8451): existing volumes are gone and cannot be mounted and I cannot create new volumes. I did a little research but I have no clue how I could get around the protection. For example the devices nodes for sda are not created under /dev so scemd complains about not able to access /dev/sda. If I login via ssh and create them with mknod they exist for a while but as soon as I log into the Web UI or only reload it they are gone again.

Do you know if there is documentation about how the protection mechanism works or could you maybe give me some hints how I could make it work? Maybe even your kernel mode dynamic patcher could help me with this? I would appreciate it if you could give me some help.

 

Best regards,

HellG

[Trantor]

This is very interesting

Unfortunately I'm very busy at work and have no time for XPEnology.

 

I'm glad that people continue to develop this project.

 

Don't forget that compiling kernel from Syno sources is very tricky.

Syno don't release all sources, some parts are missing like iscsi and synoacl.

 

Using original syno kernel + additional modules is very great but should prevent us to edit some part of the kernel.

You can use XPEnoboot 5.2 sources if you want to check how I did to add/update drivers and/or the HARDWARE SUPPORTED LIST to ckeck drivers names.

 

For exemple Broadcom tg3, bnx2 and bnx2x are higly patched in order to have successful compilation. Intel and realtek are very easy to update.

Keep in mind that including latest version of module/driver allow more hardware support that using sources from kernel tree.

 

Happy XPEnologing

[quicknick]

Trantor,

 

If you look at a directory diif with meld using the OEM rd.gz, xpenology rd.gz, and Jun's ramdisk.lzma, you can see some of the differences ande how he injected the code, but like you, still waiting on source to include in the kernel and hide the module. Very clever stuff Jun! I am pm'ing you with location to upload source.

 

I also agree that the source does not include all parts. That is why in very spare time I am working on complete patch from open source kernel and synologies open source kernel so that we can move to any version of kernel and make it easier to include the synology modded and missing stuff.

[umchem]

jun,

 

I have time and resources, what can I help with?

 

I am also interested in your project.

[jun]

Hi jun,

you seem to know what you are doing so maybe you could help me to understand the protection mechanisms of the Synology DSM 6.

I have a DS710+ (2 bay Intel Atom D410) which does not get any further updates beyond DSM 5.x (5592). I was able to create a pat file to install DSM 6.0 (7321) by just changing the update file for the DS712+ (2 bay Intel Atom D425). I just changed the synoinfo.conf, scemd.conf and manutil.conf according to what I found in the latest DS710+ update and after repacking I was able to update my box to DSM 6.0 version 7321 (and also the subsequent updates to patchlevel 7).

With the same method I was able to create an update file for DSM 6.0.2 version 8451, however now I have the same problems as others reportet here that have accidentially updated to 6.0.1 (7393) or 6.0.2 (8451): existing volumes are gone and cannot be mounted and I cannot create new volumes. I did a little research but I have no clue how I could get around the protection. For example the devices nodes for sda are not created under /dev so scemd complains about not able to access /dev/sda. If I login via ssh and create them with mknod they exist for a while but as soon as I log into the Web UI or only reload it they are gone again.

Do you know if there is documentation about how the protection mechanism works or could you maybe give me some hints how I could make it work? Maybe even your kernel mode dynamic patcher could help me with this? I would appreciate it if you could give me some help.

 

Best regards,

HellG

Hi,

The are various check around DSM programs, especially in Web Interface, maybe some are still not found out.

 

One kind is to make sure you are running on real DSM hardwares, specifically, it checks that you have pci devices the model should have. This check is a bit forgiving, i.e., if you only have 9 out of the 10 devices, you may still pass the check. Some models have mcu to control powers, leds, fans, DSM also checks that it can communicate with the controller properly.

 

Another kind of check is to make sure your system does not have any clue that you pretend to be a DSM model, for example has some file named xpenoboot somewhere,or has pid, vid in kernel cmdline, these should not exists in a real DSM devices. Sometimes, it also checksum each other to make sure system executables are not patched.

 

If DSM determines that is is not run on intended hardware, it will try to unmount all your volumes, shutdown the system, sometimes also corrupt some DSM system files etc. No sign of intentionally breaking user data is observed though.

 

Clearly, your DSM software thinks that it is not running on a real DS712+ device, so it unmounts all your volumes.

And what I did is to make sure DSM gets what it wants, so it will run happlily and ignorantly.

[jun]

This is very interesting

Unfortunately I'm very busy at work and have no time for XPEnology.

 

I'm glad that people continue to develop this project.

 

Don't forget that compiling kernel from Syno sources is very tricky.

Syno don't release all sources, some parts are missing like iscsi and synoacl.

 

Using original syno kernel + additional modules is very great but should prevent us to edit some part of the kernel.

You can use XPEnoboot 5.2 sources if you want to check how I did to add/update drivers and/or the HARDWARE SUPPORTED LIST to ckeck drivers names.

 

For exemple Broadcom tg3, bnx2 and bnx2x are higly patched in order to have successful compilation. Intel and realtek are very easy to update.

Keep in mind that including latest version of module/driver allow more hardware support that using sources from kernel tree.

 

Happy XPEnologing

Thanks for your advice. Yes, dynamic patching has its obvious limitations. So the proper source code is still very usefull.

 

I have to admit I do not have time to maintain a large list of supported drivers either.

[jun]

Trantor,

 

If you look at a directory diif with meld using the OEM rd.gz, xpenology rd.gz, and Jun's ramdisk.lzma, you can see some of the differences ande how he injected the code, but like you, still waiting on source to include in the kernel and hide the module. Very clever stuff Jun! I am pm'ing you with location to upload source.

 

I also agree that the source does not include all parts. That is why in very spare time I am working on complete patch from open source kernel and synologies open source kernel so that we can move to any version of kernel and make it easier to include the synology modded and missing stuff.

 

DSM mainly checks /proc/bus/pci/devices, /sys/bus/pci/devices, /proc/cmdline and /dev/ttyS1.

You should provide the list of devices to first 2 pseudo files/folder, and hide unwanted entries from /proc/cmdline

plus /dev/ttyS1 should looks as if a mcu is attached on the other side (I am not very clear about the detail).

A bit of reverse engineering skill and kernel programming kung fu is all you need.

[koalala]

Hi,

The are various check around DSM programs, especially in Web Interface, maybe some are still not found out.

 

One kind is to make sure you are run on real DSM hardwares, specifically, it checks that you have pci devices the model should have. this check is a bit forgiving, i.e., if you only has 9 of the 10 devices, you may still pass the check.Some models have mcu to control powers, leds, funs, DSM also checks that it can communicate with the controller properly.

 

Another kind check is to make sure you system does not has any clue that you pretend to be a DSM model, for example has some file named xpenoboot somewhere,or has pid, vid in kernel cmdline, these should not exists in a real DSM devices. sometimes, it also checksum each other to make sure system executables are not patched.

 

if DSM determine that is is not run on intended hardware, it will try to umount all you volumes, shutdown the system, sometimes also corrupt some DSM system files etc. No sign of intentionally breaking user data is observed though.

 

Clearly, your DSM software think that it is not run on a real DS712+ device, so it umount all you volumes.

And what I did is to make sure DSM get what it wants, so it will running happlily and ignorantly.

 

Then maybe we should take some inspiration from what was done for Hackintosh with OSX. The loaders are patching ACPI tables (DSDT & SSDT) to make the devices recognized properly. We can actually use GRUB to load modified ACPI tables. Using that approach we should be able to add virtual devices & rename existing ones. We would be able to use vanilla kernel without any modification (as long as the devices are supported).

[quicknick]

There is quite a bit Synology has modified in their kernel since I have been dissecting the code. It appears they make certain calls to hw as Jun has also pointed out. While ACPI tables could work, a plain vanilla kernel will still need many Synology patches.

 

Sent from my SM-N930T using Tapatalk

[dan1j3l]

Hi,

 

I tried to install on bare-metal MSI N3150I ECO ITX board (celeron N3150), but installer always fails at 58% with error "Failed to install the file. The file is probably corrupted. (13)". On same setup i have installed latest 5.2 XPEnoboot without any problems. Any insights on this problem ?

[koalala]

Jun, I tried your VM image on ESXi 6.0. However it is HW version 12 so it's not usable (max v11).

 

I converted it to V11 and for some reasons it does not work. There is exactly the same issue than the DSM6 from okstime, it installs fine but it wants to update every time you boot.

 

We have to use HW v10 but your image cannot be converted because you selected Debian + EFI and this is not supported by VMWare Converter. Could you please provide a VM Image with HW version 10 ?

 

Many thanks

[jun]

Hi,

 

I tried to install on bare-metal MSI N3150I ECO ITX board (celeron N3150), but installer always fails at 58% with error "Failed to install the file. The file is probably corrupted. (13)". On same setup i have installed latest 5.2 XPEnoboot without any problems. Any insights on this problem ?

 

maybe you did not change the vid/pid, so the installation process can not find the synoboot partitions.

 

I should make it clear that you need to change vid/pid BEFORE installation.

 

PS: DS716/916 should have better compatibility with your hardware, I'll release a DS916 version later when I can test it properly.

[dan1j3l]

Hi,

 

I tried to install on bare-metal MSI N3150I ECO ITX board (celeron N3150), but installer always fails at 58% with error "Failed to install the file. The file is probably corrupted. (13)". On same setup i have installed latest 5.2 XPEnoboot without any problems. Any insights on this problem ?

 

maybe you did not change the vid/pid, so the installation process can not find the synoboot partitions.

 

I should make it clear that you need to change vid/pid BEFORE installation.

 

PS: DS716/916 should have better compatibility with your hardware, I'll release a DS916 version later when I can test it properly.

 

You were right, somehow I really did miss to change vid/pid. Now installation goes smooth but after restart synology complains that my disks were moved from another DS3615xs and wants to reinstall/update os.

[trucidare]

Works out of the box on bare metal. Updates, raid group creation and volume creation with btfrs.

Virtual DSM, Docker DSM, both network adapters.

all seems working perfectly.

 

(Z77n-wifi - Xeon E3)

 

Thanks Jun for this release

[jun]

Hi,

 

I tried to install on bare-metal MSI N3150I ECO ITX board (celeron N3150), but installer always fails at 58% with error "Failed to install the file. The file is probably corrupted. (13)". On same setup i have installed latest 5.2 XPEnoboot without any problems. Any insights on this problem ?

 

maybe you did not change the vid/pid, so the installation process can not find the synoboot partitions.

 

I should make it clear that you need to change vid/pid BEFORE installation.

 

PS: DS716/916 should have better compatibility with your hardware, I'll release a DS916 version later when I can test it properly.

 

You were right, somehow I really did miss to change vid/pid. Now installation goes smooth but after restart synology complains that my disks were moved from another DS3615xs and wants to reinstall/update os.

 

Seem it can not mount /dev/md0 in the boot process, so falls back to installer, but in the installation process, disks are recognized correctly. So it looks a timing issue to me, will look into it.

 

EDIT:

I've found the root cause, and updated my thread, you may want to take another chance to see if it works.

Edited September 26, 2016 by Guest

[nghkpo]

 

maybe you did not change the vid/pid, so the installation process can not find the synoboot partitions.

 

I should make it clear that you need to change vid/pid BEFORE installation.

 

PS: DS716/916 should have better compatibility with your hardware, I'll release a DS916 version later when I can test it properly.

 

You were right, somehow I really did miss to change vid/pid. Now installation goes smooth but after restart synology complains that my disks were moved from another DS3615xs and wants to reinstall/update os.

 

Seem it can not mount /dev/md0 in the boot process, so falls back to installer, but in the installation process, disks are recognized correctly. So it looks a timing issue to me, will look into it.

 

I was wondering how to change vid/pid. I'm facing same issue.