Tutorial: Use Linux to Create Bootable Xpenology USB

[shrabok]

***For experienced individuals comfortable with synology and linux command line, I take no responsibility for any issues encountered***
***Read fully before considering***

 

Please read all necessary documentation for setting up Xpenology before reading this guide. Try the start-here guides below:
https://xpenology.com/forum/forum/83-faq-start-here/

This guide docuements the process of using Linux to modify the synoboot.img for your environment and writing to a usb hard drive.
Additonally, this guide shows the upgrade process from 6.1.7 to 6.2.2 with extra drivers (extra.lzma file), your use case my not have the same requirements, so please make you know the process for your environment regardless of the directions shown here.

Collect required files
***this guide is using synology model 3617, adjust accordingly for the model you want to use***

Recommended to save your files to a unique folder on your system, in this example we will use folder "xpenology-3716-20200215" (model and timestamp)

 

Download the 6.2 boot loader from this post:
https://xpenology.com/forum/topic/12952-dsm-62-loader/

Downloaded extra drivers files from this post (this post also includes links to the synology pat file as well):
***Extra Drivers are only needed if your devices are not supported by the current boot loader files***
https://xpenology.com/forum/topic/21663-driver-extension-jun-103b104b-for-dsm622-for-3615xs-3617xs-918/

 

Download the synology pat file (from the post above)

Extact files for modifications

Extract synoboot zip file

unzip synoboot_3617.zip -d synoboot

 

Extract extra drivers zip file

unzip extra3517_v0.5_test.zip -d extra_lzma

 

Extract Synology DSM pat file
***Requires 7zip***
Install 7zip for your linux distribution (example shown is Ubuntu)

sudo apt install p7zip-full p7zip-rar

 

Use 7zip to extract pat file

7z -odsm x DSM_DS3617xs_24922.pat

 

Your folder should now have 3 new folders within as shown

ls -1

Output:

dsm <--- new folder
DSM_DS3617xs_24922.pat
extra3617_v0.5_test.zip
extra_lzma <--- new folder
synoboot <--- new folder
synoboot_3617.zip

Create mount folders to modify synoboot.img files

make these folders to mount the partitions of the synoboot.img for editing

mkdir usb

mkdir usb/part1

mkdir usb/part2

mkdir usb/part3

Mount synoboot.img partitions

First view the partition layout of the synoboot.img for mounting information

fdisk -l synoboot/ds3617_6.2/synoboot.img

Output:

Disk synoboot/ds3617_6.2/synoboot.img: 50 MiB, 52428800 bytes, 102400 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: B3CAAA25-3CA1-48FA-A5B6-105ADDE4793F

Device                            Start    End Sectors Size Type
synoboot/ds3617_6.2/synoboot.img1  2048  32767   30720  15M EFI System
synoboot/ds3617_6.2/synoboot.img2 32768  94207   61440  30M Linux filesystem
synoboot/ds3617_6.2/synoboot.img3 94208 102366    8159   4M BIOS boot

Important mounting information
Units byte size: here its 512 bytes
Start byte for each partition: to mount these partitions you need to multiply the start byte by the unit byte size for the offset

 

Mount partition 1 (we use bash arithmetic expansion to multiple the offset)

sudo mount -o loop,offset=$((512*2048)) synoboot/ds3617_6.2/synoboot.img usb/part1

Mount partition 2

sudo mount -o loop,offset=$((512*32768)) synoboot/ds3617_6.2/synoboot.img usb/part2

Mount partition 3
***This partition does not get modified and does not need to be mounted, but shown for those interested in looking at the contents***

sudo mount -o loop,offset=$((512*94208)) synoboot/ds3617_6.2/synoboot.img usb/part3

Collect required hardware information for modifying grub.cfg in partition 1

Get USB vid/pid information
***Run this command before plugging in your usb drive***

lsusb

Output:

Bus 002 Device 002: ID 8087:0024 Intel Corp. Integrated Rate Matching Hub
Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 004 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 001 Device 003: ID 1bcf:2c07 Sunplus Innovation Technology Inc. 
Bus 001 Device 002: ID 8087:0024 Intel Corp. Integrated Rate Matching Hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

 

Plugin your USB drive and run lsusb again to see the new device (comparing the two outputs makes this easier to know which drive is the USB)

Output:

Bus 002 Device 009: ID 0781:5583 SanDisk Corp. <--- new drive
Bus 002 Device 002: ID 8087:0024 Intel Corp. Integrated Rate Matching Hub
Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 004 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 001 Device 003: ID 1bcf:2c07 Sunplus Innovation Technology Inc. 
Bus 001 Device 002: ID 8087:0024 Intel Corp. Integrated Rate Matching Hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

Our USB VID is 0781
Our USB PID is 5583

 

Get Network Interface Card (NIC) MAC address

 

Find MAC address of NIC on your synology device (can be found in bios, use a bootable usb linux distribution, or physical nic mac information on card label if available)

Our MAC was in the bios of our motherboard, it is 90:2B:34:AC:9F:C4

 

If you have access to your device through cli you can use ifconfig to find the MAC address as well

 

Get Synology Serial Number

 

Generate a serial number for the model being used or use existing if your device is being upgraded (https://xpenogen.github.io/serial_generator/index.html)

Our Serial Generated is 1130ODN024917 (Note these are not official synology serial numbers)

 

We will use these values for modifying grub.cfg on partition 1

Modify grub.cfg on partition 1
***VIM is the editor used to modify files, please ensure you are familiar with the editor you choose***

Edit grub.cfg

sudo vi usb/part1/grub/grub.cfg

Modify the following lines (Shift+i to go into insert mode in VIM)

set vid=0x0781 <--- Our VID found earlier
set pid=0x5583 <--- Our PID found earlier
set sn=1330NZN022235 <--- Our Serial Number generated earlier
set mac1=902B34AC9FC4 <--- Our NIC MAC address found earlier

***You may want to modify SataPortMap for your environment and timeout if you want a faster boot time when grub loads***

 

Save changes (Hit escape a few times)

:wq (write and quit)

Add extra driver files (optional) to partition 2
***Only needed if you require extra drivers***

List current files in partition 2 to view permissions and ownership

ls -la usb/part2/

Output:

total 11068
drwxr-xr-x 2 root   root     16384 Dec 31  1969 ./
drwxrwxr-x 3 user   user      4096 Feb 16 09:46 ../
-rwxr-xr-x 1 root   root   1860613 Aug  1  2018 extra.lzma*
-rwxr-xr-x 1 root   root   6441636 Aug  1  2018 rd.gz*
-rwxr-xr-x 1 root   root   3006864 Aug  1  2018 zImage*

Copy extra.lzma file to partition 2

sudo cp extra_lzma/extra.lzma usb/part2/

sudo cp dsm/rd.gz usb/part2/

sudo cp dsm/zImage usb/part2/

 

List files in partition 2 to view permissions and ownership after copy

ls -la usb/part2/

Output:

total 13748
drwxr-xr-x 2 root   root     16384 Dec 31  1969 ./
drwxrwxr-x 3 user   user      4096 Feb 16 09:46 ../
-rwxr-xr-x 1 root   root   4570532 Feb 16 09:55 extra.lzma*
-rwxr-xr-x 1 root   root   6465192 Feb 16 09:58 rd.gz*
-rwxr-xr-x 1 root   root   3019392 Feb 16 09:58 zImage*

Each file should retain root ownership and permissions, the files copied should have a different size and date stamp 

Unmount partitions

sudo umount usb/part1/

sudo umount usb/part2/

sudo umount usb/part3/

 

confirm each partition is unmounted

ls -la usb/part1/

ls -la usb/part2/

ls -la usb/part3/

 

Output:

total 8
drwxrwxr-x 2 user user 4096 Feb 16 09:46 ./
drwxrwxr-x 3 user user 4096 Feb 16 09:46 ../

Find USB device name

Ensure USB drive is unplugged

ls -1 /dev/sd*

Output:

/dev/sda
/dev/sda1

This shows the existing storage devices, sda is the drive (my laptop root hdd), sda1 is the first partition

Plug in your USB drive

ls -1 /dev/sd*

Output:

/dev/sda
/dev/sda1
/dev/sdb
/dev/sdb1
/dev/sdb2
/dev/sdb3

You may not see the same partitions, but you want to see the additional device (sdb) showing

Format USB drive
***Please backup any data you want to retain from this drive, as it will be lost***
***Commands shown here are potentially dangerous, make sure you know the correct drives to format and double check your syntax (or use an alternative tool within your comfort level)***

Unmount any partitions that may have auto mounted (example shown here attempts to unmount any drives listed for the USB drive device sdb)

sudo umount /dev/sdb1

sudo umount /dev/sdb2

sudo umount /dev/sdb3

Wipe all filesystems off existing drive

sudo wipefs --all /dev/sdb

Output:

/dev/sdb: 8 bytes were erased at offset 0x00000200 (gpt): 45 46 49 20 50 41 52 54
/dev/sdb: 8 bytes were erased at offset 0xe51fffe00 (gpt): 45 46 49 20 50 41 52 54
/dev/sdb: 2 bytes were erased at offset 0x000001fe (PMBR): 55 aa
/dev/sdb: calling ioctl to re-read partition table: Success

Check USB drive layout

sudo fdisk -l /dev/sdb

Output:

Disk /dev/sdb: 57.3 GiB, 61505273856 bytes, 120127488 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

Confirm clean output without any partitions remaining

Write Synoboot Image to USB drive
***Commands shown here are potentially dangerous, make sure you know the correct drives to write to and double check your syntax***

sudo dd if=synoboot/ds3617_6.2/synoboot.img of=/dev/sdb bs=512

Output:

102400+0 records in
102400+0 records out
52428800 bytes (52 MB, 50 MiB) copied, 9.14248 s, 5.7 MB/s

Check drive and partition information to ensure it was written correctly

sudo fdisk -l /dev/sdb

Output:

GPT PMBR size mismatch (102399 != 120127487) will be corrected by w(rite). <--- this can be ignored, it alerts because fdisk does not support GPT partitions
Disk /dev/sdb: 57.3 GiB, 61505273856 bytes, 120127488 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: B3CAAA25-3CA1-48FA-A5B6-105ADDE4793F

Device     Start    End Sectors Size Type
/dev/sdb1   2048  32767   30720  15M EFI System
/dev/sdb2  32768  94207   61440  30M Linux filesystem
/dev/sdb3  94208 102366    8159   4M BIOS boot

Unmount and plug into your Xpenology device

sudo umount /dev/sdb1

sudo umount /dev/sdb2

sudo umount /dev/sdb3

Last minute changes after write

You can still make changes to the paritions/files on the USB drive by mounting the partition on your system

sudo mount /dev/sdb1 usb/part1

sudo mount /dev/sdb2 usb/part2

sudo mount /dev/sdb3 usb/part3

This mounts the usb drive directly to the folder specified, you can edit files as per previous steps in the guide
 

Edited February 16, 2020 by shrabok
Fix formatting and title

[gatorsong]

Thanks for the instructions.  I have the same mother board but when I try 1.03b for both DS3615 and ds3617, find.synology.com doesn't see anything.  When I try 1.04b for ds918, i do see my device.  Since you have the same motherboard, trying to see how your bios is setup.  Or do you know why it is not seeing the device when i use 1.03b?

 

FYI: I use the same vid, pid, sn and mac settings for all 3 synoboot.img. 

FYI: My current setup is 1.02 with DSM 6.1.7 for ds3615.  

 

Thanks,  Any help is much appreciated.

Edited May 22, 2020 by gatorsong

[shrabok]

Have a look at this Platform chart, I believe the issue you might be encountering is having an efi bios instead of legacy when using 3615/3617

Make sure you're set to legacy for those models and you should see them.

 

Let me know how that goes @gatorsong.

Edited May 22, 2020 by shrabok

[gatorsong]

I have enable legacy support.  I change all UEFI to Legacy, but when I go look at the boot order, I only see UEFI:Sandisk listed.  I dont see anything legacy.  I google gigabyte motherboard to legacy boot settings and come up empty.  They said enable CSM, but i dont have that settings.  Is it the USB i bought, where it only allows UEFI? or maybe the way i burn the synoboot.img using win32 image?

[shrabok]

@gatorsong, are you following this guide using linux or windows?

You shouldn't need win32 image... Also did you add the extra drivers files provided in topic below as per this guide? It is possible you might not have the same motherboard or NIC as I do. Make sure to check the NIC hardware to ensure the drivers provide support for it.

 

[IG-88]

3 hours ago, gatorsong said:

  I dont see anything legacy.  I google gigabyte motherboard to legacy boot settings and come up empty.  They said enable CSM, but i dont have that settings. 

 

you need to reboot with csm active to see the plugged in usb as legacy (device without uefi in name)

[gatorsong]

• @shrabok only NIC info i can find is Intel GBe. 
• @IG-88  I activated CSM bute still see no device without UEFI in name.

[IG-88]

31 minutes ago, gatorsong said:

I activated CSM bute still see no device without UEFI in name.

i have gigabyte b365m hd3 and i expected it to be similar as you also seem to use a gigabyte board (what exactly?)

when a have the usb plugged in and save bios settings with csm aktive and on next boot i go into bios i see usb 3 times two with uefi in name and the 3rd without

if you would'nt be so secretive about your hardware i might have had a look into the manual of your board

[gatorsong]

oh so sorry I thought i mention it.  I have a Gigabyte H170N-WIFI.  I did see a third one.  two of the UEFI and the third one said sandisk.  I think I try to boot on the sandisk and it still doesn't work.  I will try again now.

[gatorsong]

Okay that works, but there is a gotcha.  So I had to go to the boot override options to select sandisk. The sandisk option did show up as one of my options when i choose boot order.

[gatorsong]

Nevermind i got the boot order working now.  Had to go to HDD priorities and move it to option 1 so that it can be listed in the boot options.

 

Thanks both of you guys.  @shrabok And @IG-88

Edited May 25, 2020 by gatorsong

[el_Salmon]

Mounting steps doesn't work for me

mount -o loop

 

I have replaced for these kpartx commands:

 

kpartx -v -a synoboot/synoboot.img 
mount /dev/mapper/loop13p1 usb/part1
mount /dev/mapper/loop13p2 usb/part2

and after umount commands:

sudo kpartx -d  -v synoboot/synoboot.img