Jump to content
XPEnology Community

Attempt at High Availability


Moogs
 Share

Recommended Posts

Specs:

 

2x

U-NAS 800 chassis

4 4TB WD REDs

9211 LSI flashed to IT

16GB Vengence RAM

ASRock Avoton MB

Vender file hex modified to use the correct MAC addresses for both NICs on both machines.

 

Both machines BIOS settings, drive orders and placements are all identical. Network all the same etc. etc..

 

I've tried both GnoBoot with DSM 5 Update 2 and Trantor's beta 4.3 v7 build with HBA support. 5.0 beta couldn't detect HBA.

 

The result was close but the logs clearly show otherwise:

 

May  8 00:27:16 BigWorm ha.cgi: net_wol_enable.c:110 Cannot get current wake-on-lan settings
May  8 00:27:16 BigWorm ha.cgi: net_wol_enable.c:225 Failed to set wake-on-lan eth2 g
May  8 00:27:16 BigWorm ha.cgi: ha.cc:1666 Failed to set Wake on Lan 2
May  8 00:27:16 BigWorm ha.cgi: net_wol_enable.c:110 Cannot get current wake-on-lan settings
May  8 00:27:16 BigWorm ha.cgi: net_wol_enable.c:225 Failed to set wake-on-lan eth3 g
May  8 00:27:16 BigWorm ha.cgi: ha.cc:1666 Failed to set Wake on Lan 3
May  8 00:27:18 BigWorm ha.cgi: net_set_card1.c(167) : HUP ddnsd result: no such service or not enabled
May  8 00:27:30 BigWorm ha.cgi: main.cc:3909 Start binding the high-availability cluster.
May  8 00:27:30 BigWorm ha.cgi: ha.cc:749 [HA-DBG] start DoBinding
May  8 00:27:30 BigWorm ha.cgi: ha.cc:777 [HA-DBG] stop service/mount/drbd
May  8 00:27:31 BigWorm synopkg: pkgstartstop.cpp:114 Failed to stop package HAManager
May  8 00:27:32 BigWorm synologset1: log_set1.c:97 Failed to send syslog to remote node.
May  8 00:27:34 BigWorm synologset1: log_set1.c:97 Failed to send syslog to remote node.
killall: synolunbkp: no process killed
May  8 00:27:39 BigWorm ha.cgi: ha.cc:787 [HA-DBG] rc.volume stop
May  8 00:27:40 BigWorm synoha: serv stop 3rd
May  8 00:27:40 BigWorm synopkg: pkgstartstop.cpp:114 Failed to stop package HAManager
May  8 00:27:40 BigWorm synoha: SxxPost stop
May  8 00:27:40 BigWorm synoha:  shutdown /usr/syno/etc/rc.d/S02synovpnclient.sh...
May  8 00:27:41 BigWorm synoha: serv stop /usr/syno/etc/rc.d/S99zupdateindexdb.sh
May  8 00:27:41 BigWorm synoha: serv stop /usr/syno/etc/rc.d/S99zbootok.sh
May  8 00:27:41 BigWorm synoha: serv stop /usr/syno/etc/rc.d/S99tftpd.sh
May  8 00:27:41 BigWorm synoha: serv stop /usr/syno/etc/rc.d/S99synostoraged.sh
cat: can't open '/var/run/synostoraged.pid': No such file or directory
kill: you need to specify whom to kill
May  8 00:27:41 BigWorm synoha: serv stop /usr/syno/etc/rc.d/S99synoscheduler.sh
May  8 00:27:41 BigWorm synoha: serv stop /usr/syno/etc/rc.d/S99synoreport.sh
May  8 00:27:41 BigWorm synoha: serv stop /usr/syno/etc/rc.d/S99synomount.sh
 % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                Dload  Upload   Total   Spent    Left  Speed
 0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0May  8 00:27:41 BigWorm synoha: serv stop /usr/syno/etc/rc.d/S99sftpd.sh
killall: sftpd: no process killed
May  8 00:27:41 BigWorm synoha: serv stop /usr/syno/etc/rc.d/S99ftpd.sh
May  8 00:27:41 BigWorm synoha: serv stop /usr/syno/etc/rc.d/S99avahi.sh
kill: you need to specify whom to kill
May  8 00:27:41 BigWorm synoha: serv stop /usr/syno/etc/rc.d/S99WDidle3Dis.sh
100     1  100     1    0     0      2      0 --:--:-- --:--:-- --:--:--     5
May  8 00:27:41 BigWorm synoha: serv stop /usr/syno/etc/rc.d/S99EAUpgrade.sh
May  8 00:27:41 BigWorm synoha: serv stop /usr/syno/etc/rc.d/S98upnpd.sh
May  8 00:27:41 BigWorm synoha: serv stop /usr/syno/etc/rc.d/S98ssdpd.sh
May  8 00:27:41 BigWorm synoha: serv stop /usr/syno/etc/rc.d/S98findhostd.sh
May  8 00:27:41 BigWorm findhostd: findhostd.c:119(FHOSTDProcessNameGet) open [/proc/17467/cmdline] fail:(2)[No such file or directory]
May  8 00:27:41 BigWorm findhostd: findhostd.c:69(FHOSTDExit) exit on signal[15]
May  8 00:27:41 BigWorm root: /usr/syno/etc/rc.d/S98findhostd.sh stop findhostd
May  8 00:27:42 BigWorm hotplugd: ##### ACTION:remove
May  8 00:27:42 BigWorm hotplugd: DEVICE:/proc/bus/usb/001/002
May  8 00:27:42 BigWorm hotplugd: DEVPATH:1-1
May  8 00:27:42 BigWorm hotplugd: SUBSYSTEM:usb
May  8 00:27:42 BigWorm hotplugd: ##### ACTION:remove
May  8 00:27:42 BigWorm hotplugd: DEVICE:/proc/bus/usb/001/002
May  8 00:27:42 BigWorm hotplugd: DEVPATH:1-1:1.0
May  8 00:27:42 BigWorm hotplugd: SUBSYSTEM:usb
May  8 00:27:42 BigWorm hotplugd: INTERFACE:9/0/0
May  8 00:27:42 BigWorm hotplugd: ##### ACTION:remove
May  8 00:27:42 BigWorm hotplugd: DEVICE:/proc/bus/usb/001/003
May  8 00:27:42 BigWorm hotplugd: DEVPATH:1-1.1
May  8 00:27:42 BigWorm hotplugd: SUBSYSTEM:usb
May  8 00:27:42 BigWorm hotplugd: ##### ACTION:remove
May  8 00:27:42 BigWorm hotplugd: DEVICE:/proc/bus/usb/001/003
May  8 00:27:42 BigWorm hotplugd: DEVPATH:1-1.1:1.0
May  8 00:27:42 BigWorm hotplugd: SUBSYSTEM:usb
May  8 00:27:42 BigWorm hotplugd: INTERFACE:8/6/80
May  8 00:27:42 BigWorm hotplugd: ##### ACTION:remove
May  8 00:27:42 BigWorm hotplugd: DEVNAME:sdu
May  8 00:27:42 BigWorm hotplugd: DEVICE:/proc/bus/usb/001/003
May  8 00:27:42 BigWorm hotplugd: DEVPATH:sdu
May  8 00:27:42 BigWorm hotplugd: SUBSYSTEM:block
May  8 00:27:42 BigWorm hotplugd: PHYSDEVPATH:/devices/pci0000:00/0000:00:16.0/usb1/1-1/1-1.1/1-1.1:1.0/host1/target1:0:0/1:0:0:0
May  8 00:27:42 BigWorm hotplugd: hotplugd.c (1559) ==== USB disk [sdu] plugged out====
May  8 00:27:42 BigWorm synoha: serv stop /usr/syno/etc/rc.d/S97apache-webdav.sh
May  8 00:27:42 BigWorm synoha: serv stop /usr/syno/etc/rc.d/S97apache-user.sh
May  8 00:27:43 BigWorm synoha: serv stop /usr/syno/etc/rc.d/S96synosnmpcd.sh
May  8 00:27:43 BigWorm hotplugd: log_set1.c:97 Failed to send syslog to remote node.
May  8 00:27:43 BigWorm hotplugd: ##### ACTION:remove
May  8 00:27:43 BigWorm hotplugd: DEVICE:/proc/bus/usb/001/004
May  8 00:27:43 BigWorm hotplugd: DEVPATH:1-1.4
May  8 00:27:43 BigWorm hotplugd: SUBSYSTEM:usb
May  8 00:27:43 BigWorm hotplugd: ##### ACTION:remove
May  8 00:27:43 BigWorm hotplugd: DEVICE:/proc/bus/usb/001/004
May  8 00:27:43 BigWorm hotplugd: DEVPATH:1-1.4:1.0
May  8 00:27:43 BigWorm hotplugd: SUBSYSTEM:usb
May  8 00:27:43 BigWorm hotplugd: INTERFACE:9/0/0
May  8 00:27:43 BigWorm hotplugd: ##### ACTION:remove
May  8 00:27:43 BigWorm hotplugd: DEVICE:/proc/bus/usb/001/005
May  8 00:27:43 BigWorm hotplugd: DEVPATH:1-1.4.4
May  8 00:27:43 BigWorm hotplugd: SUBSYSTEM:usb
May  8 00:27:43 BigWorm hotplugd: ##### ACTION:remove
May  8 00:27:43 BigWorm hotplugd: DEVICE:/proc/bus/usb/001/005
May  8 00:27:43 BigWorm hotplugd: DEVPATH:1-1.4.4:1.0
May  8 00:27:43 BigWorm hotplugd: SUBSYSTEM:usb
May  8 00:27:43 BigWorm hotplugd: INTERFACE:3/1/1
May  8 00:27:43 BigWorm hotplugd: ##### ACTION:remove
May  8 00:27:43 BigWorm hotplugd: DEVICE:/proc/bus/usb/001/005
May  8 00:27:43 BigWorm hotplugd: DEVPATH:046B
May  8 00:27:43 BigWorm hotplugd: SUBSYSTEM:usb
May  8 00:27:43 BigWorm hotplugd: PHYSDEVPATH:/devices/pci0000:00/0000:00:16.0/usb1/1-1/1-1.4/1-1.4.4/1-1.4.4:1.0
May  8 00:27:43 BigWorm hotplugd: ##### ACTION:remove
May  8 00:27:43 BigWorm hotplugd: DEVICE:/proc/bus/usb/001/005
May  8 00:27:43 BigWorm hotplugd: DEVPATH:1-1.4.4:1.1
May  8 00:27:43 BigWorm hotplugd: SUBSYSTEM:usb
May  8 00:27:43 BigWorm hotplugd: INTERFACE:3/1/2
May  8 00:27:43 BigWorm hotplugd: ##### ACTION:remove
May  8 00:27:43 BigWorm hotplugd: DEVICE:/proc/bus/usb/001/005
May  8 00:27:43 BigWorm hotplugd: DEVPATH:046B
May  8 00:27:43 BigWorm hotplugd: SUBSYSTEM:usb
May  8 00:27:43 BigWorm hotplugd: PHYSDEVPATH:/devices/pci0000:00/0000:00:16.0/usb1/1-1/1-1.4/1-1.4.4/1-1.4.4:1.1
May  8 00:27:44 BigWorm synoha: serv stop /usr/syno/etc/rc.d/S90usbip.sh
May  8 00:27:44 BigWorm synoha: serv stop /usr/syno/etc/rc.d/S88synomkflvd.sh
May  8 00:27:44 BigWorm synoha: serv stop /usr/syno/etc/rc.d/S85synonetbkpd.sh
May  8 00:27:44 BigWorm synoha: serv stop /usr/syno/etc/rc.d/S84rsyncd.sh
May  8 00:27:44 BigWorm synoha: serv stop /usr/syno/etc/rc.d/S83nfsd.sh
killall: statd: no process killed
killall: mountd: no process killed
killall: nfsd: no process killed
killall: idmapd: no process killed
killall: svcgssd: no process killed
killall: rpcbind: no process killed
umount: can't umount /proc/fs/nfsd: Invalid argument
rm: can't remove '/var/run/nfsd.pid': No such file or directory
May  8 00:27:44 BigWorm synoha: serv stop /usr/syno/etc/rc.d/S81atalk.sh
May  8 00:27:44 BigWorm synoha: serv stop /usr/syno/etc/rc.d/S80samba.sh
stop smbd ... not running
stop winbindd ... not running
stop nmbd ... not running
remove temp tdbs
May  8 00:27:44 BigWorm synoha: serv stop /usr/syno/etc/rc.d/S79RCPower.sh
May  8 00:27:44 BigWorm synoha: serv stop /usr/syno/etc/rc.d/S78iscsitrg.sh
killall: synolunbkp: no process killed
May  8 00:27:47 BigWorm synoha: serv stop /usr/syno/etc/rc.d/S77synomkthumbd.sh
May  8 00:27:47 BigWorm synoha: serv stop /usr/syno/etc/rc.d/S66synoindexd.sh
May  8 00:27:47 BigWorm synoha: serv stop /usr/syno/etc/rc.d/S66fileindexd.sh
May  8 00:27:47 BigWorm synoha: serv stop /usr/syno/etc/rc.d/S66S2S.sh
May  8 00:27:47 BigWorm synoha: serv stop /usr/syno/etc/rc.d/S56gcpd.sh
GCP: gcpd is not running. stop nothing.
May  8 00:27:47 BigWorm synoha: serv stop /usr/syno/etc/rc.d/S55cupsd.sh
CUPS: cupsd is not running. stop nothing.
May  8 00:27:47 BigWorm synoha: serv stop /usr/syno/etc/rc.d/S40irqbalance.sh
May  8 00:27:47 BigWorm root: /usr/syno/etc/rc.d/S40irqbalance.sh stopped irqbalance
May  8 00:27:47 BigWorm synoha: serv stop /usr/syno/etc/rc.d/S23synologd.sh
May  8 00:27:47 BigWorm synoha: serv stop /usr/syno/etc/rc.d/S23ntpd.sh
May  8 00:27:47 BigWorm synoha: serv stop /usr/syno/etc/rc.d/S22syslogng.sh
May  8 00:27:47 BigWorm synoha: serv stop /usr/syno/etc/rc.d/S21mysql.sh
May  8 00:27:47 BigWorm synoha: serv stop /usr/syno/etc/rc.d/S20pgsql.sh
May  8 00:27:47 BigWorm synoha: serv stop /usr/syno/etc/rc.d/S13synorelayd.sh
May  8 00:27:47 BigWorm synoha: serv stop /usr/syno/etc/rc.d/S12upsmon.sh
Can't open /var/state/ups/usbhid-ups-auto.pid: No such file or directory
May  8 00:27:47 BigWorm synoha: serv stop /usr/syno/etc/rc.d/S11nslcd.sh
May  8 00:27:47 BigWorm synoha: serv stop /usr/syno/etc/rc.d/S10slapd.sh
May  8 00:27:47 BigWorm synoha: serv stop /usr/syno/etc/rc.d/S10UPnPportmap.sh
killall: upnpd: no process killed
May  8 00:27:48 BigWorm synoha: serv stop /usr/syno/etc/rc.d/S10NatpmpPortmap.sh
killall: natpmpd: no process killed
May  8 00:27:48 BigWorm synoha: serv stop /usr/syno/etc/rc.d/S09DDNS.sh
May  8 00:27:48 BigWorm synoha: serv stop /usr/syno/etc/rc.d/S08snmpd.sh
May  8 00:27:49 BigWorm synoha: serv stop /usr/syno/etc/rc.d/S04crond.sh
May  8 00:27:49 BigWorm synoha: serv stop /usr/syno/etc/rc.d/S03hotplugd.sh
May  8 00:27:49 BigWorm synoha: serv stop /usr/syno/etc/rc.d/S02synovpnclient.sh
May  8 00:27:49 BigWorm synoha: serv stop /usr/syno/etc/rc.d/S02pppoerelay.sh
May  8 00:27:49 BigWorm synoha: serv stop /usr/syno/etc/rc.d/S02dbus.sh
May  8 00:27:49 BigWorm synoha: serv stop /usr/syno/etc/rc.d/S02bootup.sh
May  8 00:27:49 BigWorm synoha: serv stop /usr/syno/etc/rc.d/S01tc.sh
May  8 00:27:49 BigWorm synoha: serv stop /usr/syno/etc/rc.d/S01iptables.sh
May  8 00:27:49 BigWorm synoha: SxxPre stop
May  8 00:27:49 BigWorm ha.cgi: iscsi_vaai_journal_file_create.c:93 Fail to get enable status of journal file
May  8 00:27:49 BigWorm synoha: rc.volume stopall
May  8 00:27:49 BigWorm synoha: volume_parts_info_get_by_dev.c:92 Dev:sdu cannot find PostfixNumber.
May  8 00:27:50 BigWorm synoha: log_set1.c:97 Failed to send syslog to remote node.
May  8 00:27:50 BigWorm synoha: usbbkp_hot_pull_out_ex.c:108 Remove /tmp/usbtab [sdu] failed. (0x2000)
May  8 00:27:50 BigWorm synovspace: vspace_wrapper_unload_all.c:66 [iNFO] The mount device path of space [/volume1] is [/dev/md2]
May  8 00:27:50 BigWorm ha.cgi: ha_drbd.cc:942 execute drbd cmd failed: /usr/syno/synoha/sbin/drbdadm pause-sync all > /tmp/ha//drbdcmd_output 2>&1.
May  8 00:27:50 BigWorm root: no resources defined!
May  8 00:27:50 BigWorm ha.cgi: ha_drbd.cc:942 execute drbd cmd failed: /usr/syno/synoha/sbin/drbdadm secondary all  > /tmp/ha//drbdcmd_output 2>&1.
May  8 00:27:50 BigWorm root: no resources defined!
May  8 00:27:50 BigWorm ha.cgi: ha_drbd.cc:942 execute drbd cmd failed: /usr/syno/synoha/sbin/drbdadm disconnect all > /tmp/ha//drbdcmd_output 2>&1.
May  8 00:27:50 BigWorm root: no resources defined!
May  8 00:27:50 BigWorm ha.cgi: ha_drbd.cc:942 execute drbd cmd failed: /usr/syno/synoha/sbin/drbdadm down all > /tmp/ha//drbdcmd_output 2>&1.
May  8 00:27:50 BigWorm root: no resources defined!
May  8 00:27:50 BigWorm ha.cgi: ha_drbd.cc:1025 SYNODrbdRemoveSettingAll.
May  8 00:27:50 BigWorm ha.cgi: ha_drbd.cc:942 execute drbd cmd failed: /usr/syno/synoha/sbin/drbdadm pause-sync all > /tmp/ha//drbdcmd_output 2>&1.
May  8 00:27:50 BigWorm root: no resources defined!
May  8 00:27:50 BigWorm ha.cgi: ha_drbd.cc:942 execute drbd cmd failed: /usr/syno/synoha/sbin/drbdadm secondary all  > /tmp/ha//drbdcmd_output 2>&1.
May  8 00:27:50 BigWorm root: no resources defined!
May  8 00:27:50 BigWorm ha.cgi: ha_drbd.cc:942 execute drbd cmd failed: /usr/syno/synoha/sbin/drbdadm disconnect all > /tmp/ha//drbdcmd_output 2>&1.
May  8 00:27:50 BigWorm root: no resources defined!
May  8 00:27:50 BigWorm ha.cgi: ha_drbd.cc:942 execute drbd cmd failed: /usr/syno/synoha/sbin/drbdadm down all > /tmp/ha//drbdcmd_output 2>&1.
May  8 00:27:50 BigWorm root: no resources defined!
May  8 00:27:50 BigWorm ha.cgi: ha.cc:808 [HA-DBG] set up corosync/pacemaker
May  8 00:27:50 BigWorm rc.ha: rc.ha_stop begin
May  8 00:27:51 BigWorm rc.ha: local rule.Prepare
May  8 00:27:51 BigWorm rc.ha: local status.BigWorm: standalone
May  8 00:27:51 BigWorm rc.ha: set prev role passive
May  8 00:27:52 BigWorm rc.ha: standby local
May  8 00:27:52 BigWorm rc.ha: stop pacemakerd
May  8 00:27:53 BigWorm rc.ha: stop corosync
May  8 00:27:53 BigWorm rc.ha: stop drbd
May  8 00:27:53 BigWorm synodrbd: ha_drbd.cc:942 execute drbd cmd failed: /usr/syno/synoha/sbin/drbdadm pause-sync all > /tmp/ha//drbdcmd_output 2>&1.
May  8 00:27:53 BigWorm root: no resources defined!
May  8 00:27:53 BigWorm synodrbd: ha_drbd.cc:942 execute drbd cmd failed: /usr/syno/synoha/sbin/drbdadm secondary all  > /tmp/ha//drbdcmd_output 2>&1.
May  8 00:27:53 BigWorm root: no resources defined!
May  8 00:27:53 BigWorm synodrbd: ha_drbd.cc:942 execute drbd cmd failed: /usr/syno/synoha/sbin/drbdadm disconnect all > /tmp/ha//drbdcmd_output 2>&1.
May  8 00:27:53 BigWorm root: no resources defined!
May  8 00:27:53 BigWorm synodrbd: ha_drbd.cc:942 execute drbd cmd failed: /usr/syno/synoha/sbin/drbdadm down all > /tmp/ha//drbdcmd_output 2>&1.
May  8 00:27:53 BigWorm root: no resources defined!
May  8 00:28:03 BigWorm rc.ha: set host name from BigWorm
May  8 00:28:03 BigWorm rc.ha: orig hostname: BigWorm
May  8 00:28:03 BigWorm rc.ha: rc.ha_stop end
May  8 00:28:04 BigWorm rc.ha: rc.ha_start begin
May  8 00:28:04 BigWorm rc.ha: Run daemon synohamonmd0
May  8 00:28:04 BigWorm rc.ha: check conf file
May  8 00:28:04 BigWorm rc.ha: set hostname from BigWorm
May  8 00:28:04 BigWorm rc.ha: orig hostname: BigWorm
May  8 00:28:04 BigWorm rc.ha: check if in ha
May  8 00:28:04 BigWorm rc.ha: check if sync drbd
May  8 00:28:04 BigWorm rc.ha: standby local
May  8 00:28:04 BigWorm rc.ha: start drbd
May  8 00:28:04 BigWorm rc.ha: start corosync
May  8 00:28:05 BigWorm corosync[19506]:   [TOTEM ] totemsrp.c:1879 memb_state_gather_enter:15
May  8 00:28:05 BigWorm corosync[19506]:   [TOTEM ] totemsrp.c:1879 memb_state_gather_enter:11
May  8 00:28:14 BigWorm rc.ha: start pacemakerd
May  8 00:28:14 BigWorm cib: [20188]: WARN: retrieveCib: Cluster configuration not found: /usr/syno/synoha/var/lib/heartbeat/crm/cib.xml
May  8 00:28:14 BigWorm cib: [20188]: WARN: readCibXmlFile: Primary configuration corrupt or unusable, trying backup...
May  8 00:28:14 BigWorm cib: [20188]: WARN: readCibXmlFile: Continuing with an empty configuration.
May  8 00:28:15 BigWorm rc.ha: rc.ha_start end
May  8 00:28:36 BigWorm crmd: [20192]: WARN: do_log: FSA: Input I_DC_TIMEOUT from crm_timer_popped() received in state S_PENDING
May  8 00:29:16 BigWorm ha.cgi: ha.cc:834 [HA-DBG] waiting node ready... 4:4
May  8 00:29:17 BigWorm ha.cgi: ha.cc:851 [HA-DBG] add basic crm resource
May  8 00:29:27 BigWorm synoupnp: (upnpclient.c:1582) clean up finished
May  8 00:29:30 BigWorm ha.cgi: ha.cc:856 [HA-DBG] drbd start all
May  8 00:29:31 BigWorm ha.cgi: ha_drbd.cc:1863 SYNODrbdPrepareAll.
May  8 00:29:31 BigWorm ha.cgi: ha_drbd.cc:1025 SYNODrbdRemoveSettingAll.
May  8 00:29:31 BigWorm ha.cgi: ha_drbd.cc:942 execute drbd cmd failed: /usr/syno/synoha/sbin/drbdadm pause-sync all > /tmp/ha//drbdcmd_output 2>&1.
May  8 00:29:31 BigWorm root: no resources defined!
May  8 00:29:31 BigWorm ha.cgi: ha_drbd.cc:942 execute drbd cmd failed: /usr/syno/synoha/sbin/drbdadm secondary all  > /tmp/ha//drbdcmd_output 2>&1.
May  8 00:29:31 BigWorm root: no resources defined!
May  8 00:29:31 BigWorm ha.cgi: ha_drbd.cc:942 execute drbd cmd failed: /usr/syno/synoha/sbin/drbdadm disconnect all > /tmp/ha//drbdcmd_output 2>&1.
May  8 00:29:31 BigWorm root: no resources defined!
May  8 00:29:31 BigWorm ha.cgi: ha_drbd.cc:942 execute drbd cmd failed: /usr/syno/synoha/sbin/drbdadm down all > /tmp/ha//drbdcmd_output 2>&1.
May  8 00:29:31 BigWorm root: no resources defined!
May  8 00:29:31 BigWorm ha.cgi: ha_drbd.cc:942 execute drbd cmd failed: /usr/syno/synoha/sbin/drbdsetup down all > /tmp/ha//drbdcmd_output 2>&1.
May  8 00:29:31 BigWorm root: USAGE: drbdsetup command device arguments options
May  8 00:29:31 BigWorm root: Device is usually /dev/drbdX or /dev/drbd/X.
May  8 00:29:31 BigWorm root: Commands are:
May  8 00:29:31 BigWorm root:     primary             secondary           attach
May  8 00:29:31 BigWorm root:     disk-options        detach              connect
May  8 00:29:31 BigWorm root:     net-options         disconnect          resize
May  8 00:29:31 BigWorm root:     resource-options    new-current-uuid    invalidate
May  8 00:29:31 BigWorm root:     invalidate-remote   pause-sync          resume-sync
May  8 00:29:31 BigWorm root:     suspend-io          resume-io           outdate
May  8 00:29:31 BigWorm root:     verify              down                state
May  8 00:29:31 BigWorm root:     role                sh-status           cstate
May  8 00:29:31 BigWorm root:     dstate              show-gi             get-gi
May  8 00:29:31 BigWorm root:     show                check-resize        events
May  8 00:29:31 BigWorm root:     wait-connect        wait-sync           new-resource
May  8 00:29:31 BigWorm root:     new-minor           del-minor           del-resource
May  8 00:29:31 BigWorm root: To get more details about a command issue 'drbdsetup help cmd'.
May  8 00:29:31 BigWorm root: command does not accept argument 'all'
May  8 00:29:31 BigWorm crmd: [20192]: WARN: status_from_rc: Action 12 (DRBD_:0_monitor_0) on BigWorm failed (target: 7 vs. rc: 0): Error
May  8 00:29:31 BigWorm crmd: [20192]: WARN: status_from_rc: Action 22 (DRBD_:0_monitor_0) on BigWorm2 failed (target: 7 vs. rc: 0): Error
May  8 00:29:31 BigWorm synoha: vspace stage-unload equal ha
May  8 00:29:33 BigWorm ha.cgi: ha_drbd.cc:1118 CreateResource for /dev/md2.
May  8 00:29:33 BigWorm ha.cgi: ha.cc:866 [HA-DBG] drbd construct remote
May  8 00:29:36 BigWorm synodrbd: ha_space.cc:5169 SYNODrbdHandleMsg.CRM_CMD_ENUM_TYPE_RES.
May  8 00:29:36 BigWorm synodrbd: ha_space.cc:5184 SYNODrbdHandleMsg.CRM_CMD_ENUM_PART_CONF_RES.
May  8 00:29:37 BigWorm synodrbd: ha_space.cc:5207 SYNODrbdHandleMsg.CRM_CMD_ENUM_DRBD_CONF_RES.
May  8 00:29:58 BigWorm synodrbd: ha_space.cc:5361 SYNODrbdHandleMsg.CRM_CMD_DRBD_CONSTRUCT_RES.
May  8 00:29:58 BigWorm synodrbd: ha_space.cc:5364 construct. result: 22.
May  8 00:30:01 BigWorm synodrbd: ha_space.cc:5288 SYNODrbdHandleMsg.CRM_CMD_ENUM_SPACE_INTEGRITY_RES.
May  8 00:30:01 BigWorm synodrbd: ha_space.cc:5291 space integrity. result: 21.
May  8 00:30:04 BigWorm ha.cgi: ha.cc:886 [HA-DBG] drbd up all
May  8 00:30:28 BigWorm synoupnp: (upnpclient.c:1582) clean up finished
May  8 00:31:29 BigWorm synoupnp: (upnpclient.c:1582) clean up finished
May  8 00:32:05 BigWorm ha.cgi: ha_drbd.cc:942 execute drbd cmd failed: /usr/syno/synoha/sbin/drbdadm up all > /tmp/ha//drbdcmd_output 2>&1.
May  8 00:32:05 BigWorm root: Command '/usr/syno/synoha/sbin/drbdsetup attach 0 /dev/md2 /dev/loop.md2 flexible --c-max-rate=1000M --resync-rate=1000M --c-plan-ahead=20 --c-min-rate=30M --c-fill-target=500M --disk-flushes=no --md-flushes=no' did not terminate within 121 seconds
May  8 00:32:05 BigWorm ha.cgi: ha.cc:888 [HA-DBG] drbd force primary all
May  8 00:34:06 BigWorm ha.cgi: ha_drbd.cc:942 execute drbd cmd failed: /usr/syno/synoha/sbin/drbdadm primary all --force > /tmp/ha//drbdcmd_output 2>&1.
May  8 00:34:06 BigWorm root: Command '/usr/syno/synoha/sbin/drbdsetup primary 0 --force' did not terminate within 121 seconds
May  8 00:34:06 BigWorm ha.cgi: ha.cc:892 [HA-DBG] drbd secondary all
May  8 00:35:31 BigWorm synoha: synoha.cc:1197 wait-then-online triggered
May  8 00:35:31 BigWorm crmd: [20192]: WARN: match_down_event: No match for shutdown action on BigWorm
May  8 00:35:32 BigWorm crmd: [20192]: WARN: match_down_event: No match for shutdown action on BigWorm2
May  8 00:35:32 BigWorm cib: [20188]: WARN: cib_process_replace: Replacement 0.23.15 not applied to 0.23.16: current num_updates is greater than the replacement
May  8 00:35:32 BigWorm cib: [20188]: WARN: cib_diff_notify: Update (client: crmd, call:247): -1.-1.-1 -> 0.23.15 (Update was older than existing configuration)
May  8 00:35:32 BigWorm crmd: [20192]: WARN: finalize_sync_callback: Sync from BigWorm2 resulted in an error: Update was older than existing configuration
May  8 00:35:32 BigWorm crmd: [20192]: WARN: do_log: FSA: Input I_ELECTION_DC from finalize_sync_callback() received in state S_FINALIZE_JOIN
May  8 00:35:32 BigWorm synoha: confsync start.
May  8 00:35:32 BigWorm synoha: infoset start.
May  8 00:35:32 BigWorm synoha: checkpoint_last start.
May  8 00:35:32 BigWorm synoha: drbd start /usr/syno/synoha/etc/drbd.d/md2.res.
May  8 00:36:07 BigWorm ha.cgi: ha_drbd.cc:942 execute drbd cmd failed: /usr/syno/synoha/sbin/drbdadm secondary all  > /tmp/ha//drbdcmd_output 2>&1.
May  8 00:36:07 BigWorm root: Command '/usr/syno/synoha/sbin/drbdsetup secondary 0' did not terminate within 121 seconds
May  8 00:36:07 BigWorm ha.cgi: boot_lock.c(224): failed to mount boot device /dev/synoboot2 /tmp/bootmnt (errno:6)
May  8 00:36:07 BigWorm ha.cgi: manutil_vender_open.c:16 SYNOBootLockPartition() fail
May  8 00:36:07 BigWorm ha.cgi: ha.cc:902 Failed to get Mac.
May  8 00:36:07 BigWorm ha.cgi: ha.cc:936 [HA-DBG] online myself
May  8 00:36:37 BigWorm ha.cgi: ha.cc:948 [HA-DBG] waiting local online.
May  8 00:37:08 BigWorm ha.cgi: ha.cc:948 [HA-DBG] waiting local online.
May  8 00:37:38 BigWorm ha.cgi: ha.cc:948 [HA-DBG] waiting local online.
May  8 00:38:08 BigWorm ha.cgi: ha.cc:948 [HA-DBG] waiting local online.
May  8 00:38:38 BigWorm ha.cgi: ha.cc:948 [HA-DBG] waiting local online.
May  8 00:39:08 BigWorm ha.cgi: ha.cc:948 [HA-DBG] waiting local online.
May  8 00:39:38 BigWorm ha.cgi: ha.cc:948 [HA-DBG] waiting local online.
May  8 00:40:09 BigWorm ha.cgi: ha.cc:948 [HA-DBG] waiting local online.
May  8 00:40:32 BigWorm lrmd: [20189]: WARN: DRBD_:0:start process (PID 24486) timed out (try 1).  Killing with signal SIGTERM (15).
May  8 00:40:32 BigWorm lrmd: [20189]: WARN: operation start[16] on DRBD_:0 for client 20192: pid 24486 timed out
May  8 00:40:32 BigWorm crmd: [20192]: ERROR: process_lrm_event: LRM operation DRBD_:0_start_0 (16) Timed Out (timeout=300000ms)
May  8 00:40:32 BigWorm crmd: [20192]: WARN: status_from_rc: Action 84 (DRBD_:0_start_0) on BigWorm failed (target: 0 vs. rc: -2): Error
May  8 00:40:32 BigWorm crmd: [20192]: WARN: update_failcount: Updating failcount for DRBD_:0 on BigWorm after failed start: rc=-2 (update=INFINITY, time=1399534832)
May  8 00:40:39 BigWorm ha.cgi: ha.cc:948 [HA-DBG] waiting local online.
May  8 00:40:52 BigWorm lrmd: [20189]: WARN: DRBD_:0:notify process (PID 25190) timed out (try 1).  Killing with signal SIGTERM (15).
May  8 00:40:52 BigWorm lrmd: [20189]: WARN: operation notify[20] on DRBD_:0 for client 20192: pid 25190 timed out
May  8 00:40:52 BigWorm crmd: [20192]: ERROR: process_lrm_event: LRM operation DRBD_:0_notify_0 (20) Timed Out (timeout=20000ms)
May  8 00:40:52 BigWorm synoha: vspace stage-unload equal ha
May  8 00:40:52 BigWorm synoha: drbd stop /usr/syno/synoha/etc/drbd.d/md2.res.
May  8 00:41:09 BigWorm ha.cgi: ha.cc:948 [HA-DBG] waiting local online.
May  8 00:41:39 BigWorm ha.cgi: ha.cc:948 [HA-DBG] waiting local online.
May  8 00:42:09 BigWorm ha.cgi: ha.cc:948 [HA-DBG] waiting local online.
May  8 00:42:39 BigWorm ha.cgi: ha.cc:948 [HA-DBG] waiting local online.
May  8 00:43:09 BigWorm ha.cgi: ha.cc:948 [HA-DBG] waiting local online.
May  8 00:43:40 BigWorm ha.cgi: ha.cc:948 [HA-DBG] waiting local online.
May  8 00:44:10 BigWorm ha.cgi: ha.cc:948 [HA-DBG] waiting local online.
May  8 00:44:40 BigWorm ha.cgi: ha.cc:948 [HA-DBG] waiting local online.
May  8 00:45:10 BigWorm ha.cgi: ha.cc:948 [HA-DBG] waiting local online.
May  8 00:45:40 BigWorm ha.cgi: ha.cc:948 [HA-DBG] waiting local online.
May  8 00:45:52 BigWorm lrmd: [20189]: WARN: DRBD_:0:stop process (PID 25417) timed out (try 1).  Killing with signal SIGTERM (15).
May  8 00:45:52 BigWorm lrmd: [20189]: WARN: operation stop[25] on DRBD_:0 for client 20192: pid 25417 timed out
May  8 00:45:52 BigWorm crmd: [20192]: ERROR: process_lrm_event: LRM operation DRBD_:0_stop_0 (25) Timed Out (timeout=300000ms)
May  8 00:45:52 BigWorm crmd: [20192]: WARN: status_from_rc: Action 1 (DRBD_:0_stop_0) on BigWorm failed (target: 0 vs. rc: -2): Error
May  8 00:45:52 BigWorm crmd: [20192]: WARN: update_failcount: Updating failcount for DRBD_:0 on BigWorm after failed stop: rc=-2 (update=INFINITY, time=1399535152)
May  8 00:46:10 BigWorm ha.cgi: ha.cc:948 [HA-DBG] waiting local online.
May  8 00:46:41 BigWorm ha.cgi: ha.cc:948 [HA-DBG] waiting local online.
May  8 00:47:11 BigWorm ha.cgi: ha.cc:948 [HA-DBG] waiting local online.
May  8 00:47:41 BigWorm ha.cgi: ha.cc:948 [HA-DBG] waiting local online.
May  8 00:48:11 BigWorm ha.cgi: ha.cc:948 [HA-DBG] waiting local online.
May  8 00:48:41 BigWorm ha.cgi: ha.cc:948 [HA-DBG] waiting local online.
May  8 00:49:11 BigWorm ha.cgi: ha.cc:948 [HA-DBG] waiting local online.
May  8 00:49:42 BigWorm ha.cgi: ha.cc:948 [HA-DBG] waiting local online.
May  8 00:50:12 BigWorm ha.cgi: ha.cc:948 [HA-DBG] waiting local online.
May  8 00:50:42 BigWorm ha.cgi: ha.cc:948 [HA-DBG] waiting local online.
May  8 00:50:53 BigWorm crmd: [20192]: WARN: status_from_rc: Action 113 (DRBD_:1_start_0) on BigWorm2 failed (target: 0 vs. rc: -2): Error
May  8 00:50:53 BigWorm crmd: [20192]: WARN: update_failcount: Updating failcount for DRBD_:1 on BigWorm2 after failed start: rc=-2 (update=INFINITY, time=1399535453)
May  8 00:51:12 BigWorm ha.cgi: ha.cc:948 [HA-DBG] waiting local online.
May  8 00:51:42 BigWorm ha.cgi: ha.cc:948 [HA-DBG] waiting local online.
May  8 00:52:12 BigWorm ha.cgi: ha.cc:948 [HA-DBG] waiting local online.
May  8 00:52:42 BigWorm ha.cgi: ha.cc:948 [HA-DBG] waiting local online.
May  8 00:53:13 BigWorm ha.cgi: ha.cc:948 [HA-DBG] waiting local online.
May  8 00:53:43 BigWorm ha.cgi: ha.cc:948 [HA-DBG] waiting local online.
May  8 00:54:13 BigWorm ha.cgi: ha.cc:948 [HA-DBG] waiting local online.
May  8 00:54:43 BigWorm ha.cgi: ha.cc:948 [HA-DBG] waiting local online.
May  8 00:55:13 BigWorm ha.cgi: ha.cc:948 [HA-DBG] waiting local online.
May  8 00:55:43 BigWorm ha.cgi: ha.cc:948 [HA-DBG] waiting local online.
May  8 00:56:13 BigWorm crmd: [20192]: WARN: status_from_rc: Action 4 (DRBD_:1_stop_0) on BigWorm2 failed (target: 0 vs. rc: -2): Error
May  8 00:56:13 BigWorm crmd: [20192]: WARN: update_failcount: Updating failcount for DRBD_:1 on BigWorm2 after failed stop: rc=-2 (update=INFINITY, time=1399535773)
May  8 00:56:14 BigWorm ha.cgi: ha.cc:948 [HA-DBG] waiting local online.
May  8 00:56:44 BigWorm ha.cgi: ha.cc:948 [HA-DBG] waiting local online.
May  8 00:56:44 BigWorm ha.cgi: ha.cc:944 waiting online timeout.
May  8 00:56:44 BigWorm ha.cgi: ha.cc:983 Failed to do binding.
May  8 00:56:44 BigWorm ha.cgi: crm.cc:889 [HA-DBG] HAUnbindRemote
May  8 00:56:44 BigWorm ha.cgi: crm.cc:916 [HA-DBG] waiting remote offline:BigWorm2:3....
May  8 00:57:14 BigWorm ha.cgi: crm.cc:916 [HA-DBG] waiting remote offline:BigWorm2:4....
May  8 00:57:29 BigWorm corosync[19506]:   [TOTEM ] totemsrp.c:1879 memb_state_gather_enter:11
May  8 00:57:29 BigWorm corosync[19506]:   [CPG   ] cpg.c:560 cpg_sync_init: have left nodes
May  8 00:57:44 BigWorm ha.cgi: crm.cc:916 [HA-DBG] waiting remote offline:BigWorm2:5....
May  8 00:57:44 BigWorm ha.cgi: crm.cc:930 [HA-DBG] waiting remote lost:BigWorm2...
May  8 00:57:46 BigWorm ha.cgi: ha_status.cc:224 passive node shouldn't be in unknown status
May  8 00:57:51 BigWorm ha.cgi: ha_status.cc:224 passive node shouldn't be in unknown status
May  8 00:57:57 BigWorm ha.cgi: ha_status.cc:224 passive node shouldn't be in unknown status
May  8 00:58:03 BigWorm ha.cgi: ha_status.cc:224 passive node shouldn't be in unknown status
May  8 00:58:08 BigWorm ha.cgi: main.cc:4042 Passive server status unknown.
May  8 00:58:09 BigWorm ha.cgi: ha_status.cc:224 passive node shouldn't be in unknown status
May  8 00:58:14 BigWorm ha.cgi: ha_status.cc:224 passive node shouldn't be in unknown status
May  8 00:58:15 BigWorm ha.cgi: crm.cc:986 crm remove constraint: 2.
May  8 00:58:18 BigWorm synoha: confsync stop.
May  8 00:58:18 BigWorm synoha: infoset stop.
May  8 00:58:18 BigWorm synoha: checkpoint_last stop.
May  8 00:58:18 BigWorm rsyncd[24655]: rsync error: received SIGINT, SIGTERM, or SIGHUP (code 20) at rsync.c(615) [Receiver=3.0.9]
May  8 00:58:49 BigWorm pengine: [20191]: WARN: process_rsc_state: Cluster configured not to stop active orphans. DRBD_:0 must be stopped manually on BigWorm
May  8 00:58:49 BigWorm pengine: [20191]: WARN: process_rsc_state: Cluster configured not to stop active orphans. DRBD_:0 must be stopped manually on BigWorm
May  8 00:58:49 BigWorm ha.cgi: main.cc:4179 Start unbinding HA.
May  8 00:58:50 BigWorm ha.cgi: ha.cc:1152 [HA-DBG] HAUnbindLocal
May  8 00:58:50 BigWorm ha.cgi: log_set1.c:97 Failed to send syslog to remote node.
May  8 00:58:50 BigWorm ha.cgi: ha.cc:1088 HADeactivate.
May  8 00:58:50 BigWorm synopkg: pkgstartstop.cpp:89 Package target path broken, HAManager
May  8 00:58:50 BigWorm synopkg: pkgstartstop.cpp:89 Package target path broken, Python
May  8 00:58:50 BigWorm synopkg: pkgstartstop.cpp:89 Package target path broken, HAManager
May  8 00:58:50 BigWorm synopkg: pkgstartstop.cpp:89 Package target path broken, Python
May  8 00:58:50 BigWorm ha.cgi: ha.cc:1172 [HA-DBG] stop ha service
killall: synolunbkp: no process killed
May  8 00:58:51 BigWorm rc.ha: rc.ha_stop begin
May  8 00:58:51 BigWorm rc.ha: local rule.Prepare
May  8 00:58:51 BigWorm rc.ha: local status.BigWorm: warning_online
May  8 00:58:52 BigWorm rc.ha: set prev role passive
May  8 00:58:52 BigWorm rc.ha: standby local
May  8 00:58:52 BigWorm rc.ha: stop pacemakerd
May  8 00:58:53 BigWorm pengine: [20191]: WARN: process_rsc_state: Cluster configured not to stop active orphans. DRBD_:0 must be stopped manually on BigWorm
May  8 00:58:53 BigWorm cib: [20188]: WARN: send_via_callback_channel: Cannot find client for token f1a02d06-5d9e-4ecb-886f-af25ec179150
May  8 00:58:53 BigWorm cib: [20188]: WARN: do_local_notify: A-Sync reply to crmd failed: client left before we could send reply
May  8 00:58:53 BigWorm pacemakerd: [20182]: WARN: send_ipc_message: IPC Channel to 20192 is not connected
May  8 00:58:53 BigWorm pacemakerd: [20182]: WARN: send_ipc_message: IPC Channel to 20190 is not connected
May  8 00:58:53 BigWorm pacemakerd: [20182]: WARN: send_ipc_message: IPC Channel to 20188 is not connected
May  8 00:58:53 BigWorm pacemakerd: [20182]: WARN: send_ipc_message: IPC Channel to 20187 is not connected
May  8 00:58:54 BigWorm rc.ha: stop corosync
May  8 00:58:54 BigWorm corosync[19506]:   [MAIN  ] util.c:141 Corosync Cluster Engine exiting with status 0 at main.c:1869.
killall: synolunbkp: no process killed
May  8 00:58:55 BigWorm rc.ha: stop drbd
May  8 00:58:59 BigWorm ha.cgi: ha.cc:3845 [HA-DBG] rc.volume stop
May  8 00:58:59 BigWorm synoha: rc.volume stop
May  8 00:58:59 BigWorm synoha: volume_parts_info_get_by_dev.c:92 Dev:sdu cannot find PostfixNumber.
May  8 00:58:59 BigWorm synoha: log_set1.c:97 Failed to send syslog to remote node.
May  8 00:58:59 BigWorm synoha: usbbkp_hot_pull_out_ex.c:108 Remove /tmp/usbtab [sdu] failed. (0x2000)
May  8 00:58:59 BigWorm synoha: ha_space.cc:6548 Failed to get ssd info.
May  8 00:58:59 BigWorm ha.cgi: crm.cc:889 [HA-DBG] HAUnbindRemote
May  8 00:58:59 BigWorm ha.cgi: crm.cc:897 No remote already.
May  8 00:58:59 BigWorm ha.cgi: crm.cc:986 crm remove constraint: 2.
May  8 00:59:00 BigWorm ha.cgi: ha.cc:1256 Failed to crm remove remote.
May  8 00:59:01 BigWorm ha.cgi: ha.cc:1145 [HA-DBG] HAUnbindLocal is processing
May  8 01:01:01 BigWorm ha.cgi: ha.cc:1145 [HA-DBG] HAUnbindLocal is processing
May  8 01:02:05 BigWorm rc.ha: wait drbd stop timeout
May  8 01:02:06 BigWorm rc.ha: set host name from BigWorm
May  8 01:02:06 BigWorm rc.ha: orig hostname: BigWorm
May  8 01:02:06 BigWorm rc.ha: rc.ha_stop end
May  8 01:02:06 BigWorm ha.cgi: ha.cc:1174 [HA-DBG] remove corosync configs
May  8 01:02:06 BigWorm ha.cgi: ha.cc:1193 [HA-DBG] remove all drbd configs
May  8 01:02:06 BigWorm synopkg: pkgstartstop.cpp:89 Package target path broken, HAManager
May  8 01:02:06 BigWorm synopkg: pkgstartstop.cpp:89 Package target path broken, Python
May  8 01:02:06 BigWorm synopkg: pkgstartstop.cpp:89 Package target path broken, HAManager
May  8 01:02:06 BigWorm synopkg: pkgstartstop.cpp:89 Package target path broken, Python
killall: synolunbkp: no process killed
May  8 01:02:09 BigWorm ha.cgi: ha.cc:3845 [HA-DBG] rc.volume stop
May  8 01:02:09 BigWorm synoha: rc.volume stop
May  8 01:02:10 BigWorm synoha: volume_parts_info_get_by_dev.c:92 Dev:sdu cannot find PostfixNumber.
May  8 01:02:10 BigWorm synoha: log_set1.c:97 Failed to send syslog to remote node.
May  8 01:02:10 BigWorm synoha: usbbkp_hot_pull_out_ex.c:108 Remove /tmp/usbtab [sdu] failed. (0x2000)
May  8 01:02:10 BigWorm synoha: ha_space.cc:6548 Failed to get ssd info.
May  8 01:03:01 BigWorm ha.cgi: ha.cc:1145 [HA-DBG] HAUnbindLocal is processing
May  8 01:05:01 BigWorm ha.cgi: ha.cc:1104 Failed to unbind local.
May  8 01:05:01 BigWorm syno_poweroff_task: System is going to reboot.
May  8 01:05:01 BigWorm rc.ha: rc.ha_stop begin
May  8 01:05:01 BigWorm rc.ha: local rule.Prepare
May  8 01:05:01 BigWorm rc.ha: local status.BigWorm: standalone
May  8 01:05:02 BigWorm rc.ha: set prev role passive
May  8 01:05:02 BigWorm rc.ha: standby local
May  8 01:05:02 BigWorm rc.ha: stop pacemakerd
May  8 01:05:03 BigWorm rc.ha: stop corosync
May  8 01:05:03 BigWorm rc.ha: stop drbd
May  8 01:08:13 BigWorm rc.ha: wait drbd stop timeout
May  8 01:08:13 BigWorm rc.ha: set host name from BigWorm
May  8 01:08:13 BigWorm rc.ha: orig hostname: BigWorm
May  8 01:08:13 BigWorm rc.ha: rc.ha_stop end

Link to comment
Share on other sites

 Share

×
×
  • Create New...