Xpenology runing on docker


Recommended Posts

On 1/19/2021 at 10:45 AM, apriliars3 said:

 

I have this issue and no work docker xpenology

 

INFO: Start downloading bootloader from URL ...

ERROR: Bootloader cannot be downloaded from URL.

WARNING: /image/bootloader.raw file size does not seem correct for synoboot.

INFO: Bootloader has been converted to qcow2

INFO: No Initial Disk found, creating disk /image/vm-disk-1.qcow2

Formatting '/image/vm-disk-1.qcow2', fmt=qcow2 size=34359738368 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16

INFO: KVM acceleration enabled

INFO: Configuring network ...

add bridge failed: Operation not permitted

WARNING: /image/bootloader.raw file size does not seem correct for synoboot.

ERROR: No Initial Disk found, CANNOT create disk /image/vm-disk-1.qcow2

INFO: KVM acceleration enabled

 

Probably because your URL pointing to bootloader is not valid.
And you do not have user permission to modify network configuration (try to run it as root)
Check this article which may help you: https://www.uxora.com/other/virtualization/57-xpenology-on-docker

HTH

Link to post
Share on other sites

in uxora/xpenology image, when use  BOOTLOADER_AS_USB=Y in a part of .pat install to :5000

get error xpenology failed to install the file. the file is probably corrupted. (13)

if set BOOTLOADER_AS_USB=N = working

 

segator/xpenology:latest from dockerhub DSM 6.0.2 work, but don't auto_attach qcow2 created disk, Someone now "howto" i can attach it?

BOOTLOADER_AS_USB

Edited by ZuruiKitsune
Link to post
Share on other sites
11 hours ago, ZuruiKitsune said:

in uxora/xpenology image, when use  BOOTLOADER_AS_USB=Y in a part of .pat install to :5000

get error xpenology failed to install the file. the file is probably corrupted. (13)

if set BOOTLOADER_AS_USB=N = working

 

segator/xpenology:latest from dockerhub DSM 6.0.2 work, but don't auto_attach qcow2 created disk, Someone now "howto" i can attach it?

BOOTLOADER_AS_USB

 

I don't know which bootloader and dsm version you are using ...
But I just tried on a new docker with "synoboot_103b_ds3615xs_virtio_9p.img" bootloader and DSM_DS3615xs_25426.pat, and it still work for me by following this tutorial https://www.uxora.com/other/virtualization/57-xpenology-on-docker
 

Link to post
Share on other sites

Hi, everyone

 

Got following error when running docker/xpenology.  Any thoughts?  Any help will be much appreciated!

 

docker run --privileged --cap-add=NET_ADMIN \
>     --device=/dev/net/tun --device=/dev/kvm \
>     -e CPU="qemu64" \
>     -e THREADS=1 \
>     -e RAM=512 \
>     -e DISK_SIZE="8G 10G" \
>     -e DISK_PATH="/image" \
>     -e BOOTLOADER_URL="http://192.168.1.10/synoboot_104b_ds918plus_virtio_9p.img" \
>     -e BOOTLOADER_AS_USB="Y" \
>     -e VM_ENABLE_VIRTIO="Y" \
>     uxora/xpenology
INFO: Start downloading bootloader from URL ...

     0K ........ ........ ........ ........ 64%  117M 0s
 32768K ........ ........ ..               100% 82.7M=0.5s
INFO: Bootloader has been successfully downloaded from URL.
INFO: /image/bootloader.raw file size seems valid for synoboot.
INFO: Bootloader has been converted to qcow2
INFO: No Initial Disk found, creating disk /image/vm-disk-1.qcow2
Formatting '/image/vm-disk-1.qcow2', fmt=qcow2 size=8589934592 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16
INFO: No Initial Disk found, creating disk /image/vm-disk-2.qcow2
Formatting '/image/vm-disk-2.qcow2', fmt=qcow2 size=10737418240 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16
INFO: KVM acceleration enabled
INFO: Configuring network ...
net.ipv4.ip_forward = 1
INFO: DHCP configured to serve IP 20.20.20.21/24 via dockerbridge
iptables: No chain/target/match by that name.
 

Link to post
Share on other sites
1 hour ago, whuang said:

Hi, everyone

 

Got following error when running docker/xpenology.  Any thoughts?  Any help will be much appreciated!

 

docker run --privileged --cap-add=NET_ADMIN \
>     --device=/dev/net/tun --device=/dev/kvm \
>     -e CPU="qemu64" \
>     -e THREADS=1 \
>     -e RAM=512 \
>     -e DISK_SIZE="8G 10G" \
>     -e DISK_PATH="/image" \
>     -e BOOTLOADER_URL="http://192.168.1.10/synoboot_104b_ds918plus_virtio_9p.img" \
>     -e BOOTLOADER_AS_USB="Y" \
>     -e VM_ENABLE_VIRTIO="Y" \
>     uxora/xpenology
INFO: Start downloading bootloader from URL ...

     0K ........ ........ ........ ........ 64%  117M 0s
 32768K ........ ........ ..               100% 82.7M=0.5s
INFO: Bootloader has been successfully downloaded from URL.
INFO: /image/bootloader.raw file size seems valid for synoboot.
INFO: Bootloader has been converted to qcow2
INFO: No Initial Disk found, creating disk /image/vm-disk-1.qcow2
Formatting '/image/vm-disk-1.qcow2', fmt=qcow2 size=8589934592 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16
INFO: No Initial Disk found, creating disk /image/vm-disk-2.qcow2
Formatting '/image/vm-disk-2.qcow2', fmt=qcow2 size=10737418240 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16
INFO: KVM acceleration enabled
INFO: Configuring network ...
net.ipv4.ip_forward = 1
INFO: DHCP configured to serve IP 20.20.20.21/24 via dockerbridge
iptables: No chain/target/match by that name.
 


As I did reply to you already in the article, for me, it works on several machine without getting this issue.

This error may come from your Docker/Host machine configuration.

However, the following command seems to be the one throwing that error:

# Hack for guest VMs complaining about "bad udp checksums in 5 packets"
$ iptables -A POSTROUTING -t mangle -p udp --dport bootpc -j CHECKSUM --checksum-fill


You can try to test/debug by running a bash prompt with "-it --entrypoint /bin/bash"  as follow:

docker run --privileged --cap-add=NET_ADMIN \
     --device=/dev/net/tun --device=/dev/kvm \
     -e CPU="qemu64" \
     -e THREADS=1 \
     -e RAM=512 \
     -e DISK_SIZE="8G 10G" \
     -e DISK_PATH="/image" \
     -e BOOTLOADER_URL="http://192.168.1.10/synoboot_104b_ds918plus_virtio_9p.img" \
     -e BOOTLOADER_AS_USB="Y" \
     -e VM_ENABLE_VIRTIO="Y" \
     -it --entrypoint /bin/bash \
     uxora/xpenology

 

Try to test and see why the faulty "iptables" command does not work.

Or edit "/usr/bin/vm-startup" script as you please (maybe like removing faulty cmd)
Then run that script to launch xpenology kvm.

 

Let me know if you find something... like this I may change the script to be more compatible.

HTH

Link to post
Share on other sites
2 hours ago, uxora-com said:


As I did reply to you already in the article, for me, it works on several machine without getting this issue.

This error may come from your Docker/Host machine configuration.

However, the following command seems to be the one throwing that error:


# Hack for guest VMs complaining about "bad udp checksums in 5 packets"
$ iptables -A POSTROUTING -t mangle -p udp --dport bootpc -j CHECKSUM --checksum-fill


You can try to test/debug by running a bash prompt with "-it --entrypoint /bin/bash"  as follow:


docker run --privileged --cap-add=NET_ADMIN \
     --device=/dev/net/tun --device=/dev/kvm \
     -e CPU="qemu64" \
     -e THREADS=1 \
     -e RAM=512 \
     -e DISK_SIZE="8G 10G" \
     -e DISK_PATH="/image" \
     -e BOOTLOADER_URL="http://192.168.1.10/synoboot_104b_ds918plus_virtio_9p.img" \
     -e BOOTLOADER_AS_USB="Y" \
     -e VM_ENABLE_VIRTIO="Y" \
     -it --entrypoint /bin/bash \
     uxora/xpenology

 

Try to test and see why the faulty "iptables" command does not work.

Or edit "/usr/bin/vm-startup" script as you please (maybe like removing faulty cmd)
Then run that script to launch xpenology kvm.

 

Let me know if you find something... like this I may change the script to be more compatible.

HTH

 

Hi, HTH

 

Thank you for the quick reply.

 

After adding "-it --entrypoint /bin/bash" that you suggested, when running docker/xpenology, it does not show "iptables: No chain/target/match by that name." any more.

 

I don't find the  "/usr/bin/vm-startup" script in my system.  I wonder if you could kindly show me an example how to incorporate what you suggested in that script.

 

I really appreciate your advise/help.

 

Regards,

Wade

Link to post
Share on other sites
On 1/28/2021 at 6:01 AM, whuang said:

 

Hi, HTH

 

Thank you for the quick reply.

 

After adding "-it --entrypoint /bin/bash" that you suggested, when running docker/xpenology, it does not show "iptables: No chain/target/match by that name." any more.

 

I don't find the  "/usr/bin/vm-startup" script in my system.  I wonder if you could kindly show me an example how to incorporate what you suggested in that script.

 

I really appreciate your advise/help.

 

Regards,

Wade


Adding "-it --entrypoint /bin/bash" makes the containers run "/bin/bash" instead of default "/usr/bin/vm-startup" (define in Dockerfile).
This gives you access to the command line in the uxora/xpenology container and it's in there you can find "/usr/bin/vm-startup"!

Maybe once you get command line access to the containers, I dont know ... try to find out what the issue by running for exemple:
- $ iptables -A POSTROUTING -t mangle -p udp --dport bootpc -j CHECKSUM --checksum-fill
- $ /usr/bin/vm-startup

 

If you don't really make an effort to really find out, I cannot be much help. Sorry.

HTH == "Hope That helps"

  • Like 1
Link to post
Share on other sites
  • 3 weeks later...

Hi

 

After migrating from Xpenology to UNRAID, I'm trying to install Xpenology in a docker to fill some lacks as nice Users Homes and Surveillance Station.

 

I tried Uxora package, and I sincerely thank him for this, but I got some issues. https://github.com/uxora-com/xpenology-docker

 

 

First, I had the Corrupted file issue when I installed DSM. Changing USB from Y to N did the job, but as DSM format every drive at installation, I had to reinstall everything at each start ... Finally got it working, with USB on Y and a better connection. Fine.

 

Then I had some issues on mounting share drives.

I didn't have to call the drivers as this because they were already loaded and found in another directory of the OS (like /usr/module ...)

# Load 9p drivers, if not already loaded
$ sudo insmod /volume1/homes/admin/9pnet.ko
$ sudo insmod /volume1/homes/admin/9pnet_virtio.ko
$ sudo insmod /volume1/homes/admin/9p.ko

 

After creating a Shared Folder called "datashare" in DSM, I was able to mount the VM directory on it with the command :

$ sudo mount -t 9p -o trans=virtio,version=9p2000.L,msize=262144 hostdata /volume1/datashare

 

Fine, but when testing it, I wasn't able to write anything on it. Placing some files in it on the VM, make them readable from DSM, but the opposite way failed with a message like "this action is not managed" (I had the french message so it is a translation).

 

Checking chmod and chown on UNRAID VM didn't make the trick.

 

Reboot is OK (one time a CPU #0 was stuck and I had to force the reboot from docker).

 

After that, I tried to add some other shared directories on my Docker and ... after reboot I had to reinstall DSM !

I don't understand what happened.

 

Maybe someone could give me a hint ?

 

Here are the dockers parameters

run -d --name='xpenology' --net='swag-docker-network' --cpuset-cpus='0,1,2,3' --privileged=true -e TZ="Europe/Paris" -e HOST_OS="Unraid" -e 'CPU'='qemu64' -e 'THREADS'='4' -e 'RAM'='2048' -e 'DISK_SIZE'='50G' -e 'DISK_PATH'='/image' -e 'BOOTLOADER_URL'='http://littlelama.fr/ftp/dsmBootLoaders/synoboot_103b_ds3615xs_virtio_9p-vanilla.img' -e 'BOOTLOADER_AS_USB'='Y' -e 'VM_ENABLE_VIRTIO'='Y' -e 'CORES'='4' -p '5000:5000/tcp' -p '5001:5001/tcp' -p '2222:22/tcp' -p '8081:80/tcp' -v '/mnt/user/appdata/xpenology/datashare':'/datashare':'rw' -v '/mnt/user/music':'/music':'rw,slave' -v '/mnt/user/appdata/xpenology/surveillance':'/surveillance':'rw,slave' -v '/mnt/user/homes/admin/':'/homes':'rw,slave' --device='/dev/net/tun' --device='/dev/kvm' --cap-add=NET_ADMIN 'uxora/xpenology'

 

Hare are some logs

https://pastebin.com/H6gFvWtK

 

Many thanks

 

Link to post
Share on other sites

Hello, 

I have successfully started this container. But found issues with "Storage pool". I couldn't create storage pool. DSM show that "Storage pool required HDD". 

But I have already created 16 Gb disk.  Can someone tell me where I am wrong?

 

 

 

Screenshot_406.thumb.png.5cdaa70d15007feb705a9303a9d4cdad.png

 

 

Link to post
Share on other sites
Hello, 
I have successfully started this container. But found issues with "Storage pool". I couldn't create storage pool. DSM show that "Storage pool required HDD". 
But I have already created 16 Gb disk.  Can someone tell me where I am wrong?
 
 
 
Screenshot_406.thumb.png.5cdaa70d15007feb705a9303a9d4cdad.png
 
 
Did you well drag and drop the 16GB disk to the grey place at right of the screen ?

Envoyé de mon ONEPLUS A6013 en utilisant Tapatalk

  • Like 1
Link to post
Share on other sites
15 minutes ago, LittleLama said:

Did you well drag and drop the 16GB disk to the grey place at right of the screen ?

Envoyé de mon ONEPLUS A6013 en utilisant Tapatalk
 

 

OMG, shame on me. It was not obvious for me. Thank you!  

Link to post
Share on other sites

Just want to confirm that image work with 6.2.3-25426 and ds3615xs. After this we need to apply patch and after that could successfully update to 6.2.3-25426 Update 3. 

Tested on VmWare Workstation. 

 

  • Like 1
Link to post
Share on other sites
  • 2 weeks later...
On 2/19/2021 at 11:03 AM, LittleLama said:

After that, I tried to add some other shared directories on my Docker and ... after reboot I had to reinstall DSM !

I don't understand what happened.

 

Maybe someone could give me a hint ?

 

Hi,

How do you restart your docker container?
 

Link to post
Share on other sites
 
Hi,

How do you restart your docker container?
 
Hello
Docker is restarted:
-either by unraid when a setting is changed by me in the docker container
-either by unraid when restarting the whole server
-most of the time by DSM itself

Envoyé de mon ONEPLUS A6013 en utilisant Tapatalk

Link to post
Share on other sites
17 minutes ago, LittleLama said:

Docker is restarted:
-either by unraid when a setting is changed by me in the docker container
-either by unraid when restarting the whole server
-most of the time by DSM itself


I don't use UNRAID and don't really know how it manages docker.

- "most of the time by DSM itself" : DSM can restart (which do not close the container) or it can shutdown (which normally terminate the container at the end)
-  "either by unraid" : Not sure how it manages docker but if it runs the full docker parameters when it restart, then this will create a new container at every restart. A new container will create new virtual disks if not exists, and need to do dsm installation again. 

Normally you should run the full run parameter only once to create the container, then you can stop/start/restart the container with the containerID (ie. docker container start <containerID>).
Otherwise, if you really want/need to create new container at every reboot (and don't care about saving disk space 😛 ) then "maybe" you can try to change DISK_PATH to a external storage (outside the container)  like "/datashare" to store virtual disks.

HTH

Link to post
Share on other sites

Yes, storing vdisk outside is the solution, but I didn't succeed in making it working due to(i guess) permission restrictions.
Now that my docker has been fixed I should make a new try.

Envoyé de mon ONEPLUS A6013 en utilisant Tapatalk

Link to post
Share on other sites
44 minutes ago, LittleLama said:

Yes, storing vdisk outside is the solution, but I didn't succeed in making it working due to(i guess) permission restrictions.
Now that my docker has been fixed I should make a new try.

 

On my test machine, I run docker container as root on a Debian 10.
I just tried and it works for me ... I don't have any permission issue and 9p mount is working as well.


If it cannot create vdisk on /datashare then the permission issue is between the Host/User->Docker->Container ... and not the virtual machine running inside the container.


As I suggest to whuang, try to get terminal access to linux OS container with the following command (when container is running) :
 

docker exec -ti $( docker container ls -f 'ancestor=uxora/xpenology' -f "status=running" -q ) /bin/bash

 

Then try to test/debug/figure out why you have permission issue which can be due to :
- Docker configuration issues
- Host or User permissions issues


And let us know if you find out the issue.

HTH

Link to post
Share on other sites

Just to let me know.
Does 9P gives the possibility to mount a host drive at the place of a Dsm shared folder ?

If I want to mount the host directory as replacement of DSM volume1/music directory to let it serve Audio Station, would it be possible ?
NFS share can't do this, does 9P ?

Same question with encrypted shared folder, is it possible with 9P to store encrypted files on a host directory or is it mandatory to have a vdisk to store the files in it ?

Thanks



Link to post
Share on other sites
1 hour ago, LittleLama said:

Does 9P gives the possibility to mount a host drive at the place of a Dsm shared folder ?

If I want to mount the host directory as replacement of DSM volume1/music directory to let it serve Audio Station, would it be possible ?
NFS share can't do this, does 9P ?

 

I don't really use 9p in my main xpenology, but just play with it a bit on a test system.

9p allow you to passthrough a folder from host to VM, and yes, I was able to make it as a DSM shared folder.
For that, you just need to create the folder as root, then mount it as 9p, then create a shared folder on DSM with with same folder name.

 

For ex:

[root]$ mkdir /volume1/datashare
[root]$ mount -t 9p -o trans=virtio,version=9p2000.L,msize=262144 hostdata /volume1/datashare/
# And then create "datashare" DSM shared folder in volume1 

 

Maybe this can be done with NFS as well, I have never tried it yet but I don't see why this cannot be done.

 

For encrypted shared folder, I have never tried so I don't know ... let us know if to get this working.


HTH 

 

Edited by uxora-com
Link to post
Share on other sites

Hi UXORA-COM, This is exactly what I need! I have  a working Xpenology (which I am using for years now) on Baremetal (Gen8 16Gb Xeon e3-1230v2) and several test installs on ESXI as a VM pulling content from NFS shares on the other Xpenology and OMV NAS.

 

In the past I have tried virtualising Xpenology on ESXI with VMDK disks but that was not a succes, after a While disks got state unreadable etc. so I stoped with that approach.

Also I the good thing about virtualisation is if you want to do a upgrade you can snapshot etc. But wil a lot of data on the system that is not an option. Once the solution with a  NFS share etc.

 

Since I am moving to a OMV or TRUENAS installation  because of the robust ZFS options etc. with all apps on docker I am kind of leaving Xpenology behind, but I still want to be able to use Synology apps, the DSM GUI and a docker approach is exactly what I want. Ideal it would be even better if we could get it to run in a LXC or Freebsd jail as sharing the ifs volumes with the jail or lxc containers is even better than passing volumes to docker. But I guess that won't work as DSM won't let you share external content as volume1 root volume. It kind of wants to build a raid with BTRFS on actual drives (or virtual ones).

 

Anyway, I am very interested to get this running but am wondering what your advise it.

I am thinking of using TRUENAS 12 on my new HPE Micro Gen10 Plus with 64GB ECC ram and 4 10TB drives. I want to run must stuff in a Freebsd JAIL (because of the speed and good sharing of ifs volumes etc.) and other stuff in a UBUNTU or other Docker centric distro where I will run all other apps in.

I will export the ZFS volume through NFS and mount it in the UBUNTU (or other distro) VM and pass it to the docker containers.

For the Docker XPENOLOGY container I guess I can mount the NFS share directly from the TRUENAS instead of going through the Volume option of Docker.

 

What do you think? Will this work?

 

Oh, almost forgot, all links to the modified boot loader are unavailable, can you help me get one?

 

Kind regards and hank you for your great work!

 

 

Edited by ferno
Link to post
Share on other sites
23 hours ago, ferno said:

Oh, almost forgot, all links to the modified boot loader are unavailable, can you help me get one?

 

 

You got all links in this tutorial  : https://www.uxora.com/other/virtualization/57-xpenology-on-docker

 

I'm not sure what you really try to do so the only advise I think i can give you is the less layers you got the better it get:
1- Baremetal -> DSM
2- Baremetal -> OS -> Qemu-kvm -> DSM
3- Baremetal -> OS -> Lxc/Docker -> Qemu-kvm -> DSM
4- Baremetal -> OS -> Lxc -> Docker -> Qemu-kvm -> DSM

 

Qemu-kvm is a VM and it is isolated enough so it doesn't really need to add another jail stuff I think.
But it all depends on what you want to do and the capabilities of your first OS layer.

HTH 

Link to post
Share on other sites

Hi UXORA-COM, 

 

I am still fiddling and trying things out.

 

Am having some trouble starting the LXC but I probably did something wrong and have to go over it again.

There are some parts of the tutorial that I might've interpreted wrongly.

 

One question, does the boot loader have to be loaded from a URL?

 

Also, I have to read-up on the virtio 9P driver etc.

Link to post
Share on other sites
14 hours ago, ferno said:

One question, does the boot loader have to be loaded from a URL?

 

 

From this docker, yes because this is how it has been designed.
But it is just needed once at the first container's run.
If you don't have a webserver, you can use gofile.io as explained in the tutorial.

 

HTH

 

Link to post
Share on other sites
  • 1 month later...

Hi There,

 

I can't get the download to work so I never see "INFO: Bootloader has been successfully downloaded from URL." , if I try to put a wrong URL on purpose then I don't see any error message as well

 

I am trying this on a VPS I am renting , I've verified all the pre-reqs.

 

I have nginx installed on the VPS , not sure if I need to uninstall it.

 

Does anyone has an idea so I can pass the bootloader download step ?

 

Thank you & sorry for my english

 

XT

 

EDIT : in other words this is what I always get , i've verified with wget that the URL can be resolved

 

 docker run --privileged \
> -e BOOTLOADER_URL="http://localhost/xpen/s.img" \
> uxora/xpenology
INFO: Start downloading bootloader from URL ...

 

I've verified Docker is running fine

 

docker run hello-world

Hello from Docker!
This message shows that your installation appears to be working correctly.

 

 

 

Edited by extenue
Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.