WiteWulf Posted October 13, 2021 Author Share #26 Posted October 13, 2021 Regarding soft lockups on hypervisor guests, I just found this: https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/managing_monitoring_and_updating_the_kernel/keeping-kernel-panic-parameters-disabled-in-virtualized-environments_managing-monitoring-and-updating-the-kernel Redhat explicitly state that in a vm guest you should have: nmi_watchdog=0 and softlockup_panic=0 ...to avoid spurious soft lockups. I think everyone running on a hypervisor should be setting these parameters in their setups 🤔 Quote Link to comment Share on other sites More sharing options...
Orphée Posted October 13, 2021 Share #27 Posted October 13, 2021 Interesting ! Curious to know @ThorGroup's opinion on this ! Quote Link to comment Share on other sites More sharing options...
Kouill Posted October 13, 2021 Share #28 Posted October 13, 2021 il y a 27 minutes, WiteWulf a dit : @Kouill was kind enough to give me a copy of their boot img so I could attempt to replicate their influx-test without crashes. Unfortunately the influx-test kernel panic'd my system within a few seconds. NB. there were the following differences: - kouill boots their system from the internal microSD slot, whereas mine was on a USB stick on the internal port - in the grub.cfg I changed the serial number to match mine - in the grub.cfg I changed mac1 to match mine - in the grub.cfg I changed I added mac2 with my mac address Sorry for the kernel panic issues, we had the same hardware possible to have a difference of bios settings and cpu can explain the crashes ? Quote Link to comment Share on other sites More sharing options...
WiteWulf Posted October 13, 2021 Author Share #29 Posted October 13, 2021 1 hour ago, Kouill said: Sorry for the kernel panic issues, we had the same hardware possible to have a difference of bios settings and cpu can explain the crashes ? No need to apologise, thank you for helping out! We've got a slightly different CPU, but when I collated a list of people's CPUs earlier it included a broad range of models. I'll go through my BIOS and make a note of all settings, page by page, so that we can compare. Quote Link to comment Share on other sites More sharing options...
WiteWulf Posted October 13, 2021 Author Share #30 Posted October 13, 2021 I've drawn up a quick spreadsheet with as much information as I could get relating to hardware, firmware, BIOS settings and general configuration. These are spread across four different tabs within the spreadsheet. https://docs.google.com/spreadsheets/d/1aBrWkJHzwBl6HcYqpBZhRyeX-5CxT5yTmXZ9p3d8zAY/edit?usp=sharing Feel free to comment on any of the settings you think are right or wrong. @Kouill could you compare this to your setup and either fill the sheet in with your details or just reply back here with anything you think is relevant, please? 2 Quote Link to comment Share on other sites More sharing options...
Orphée Posted October 13, 2021 Share #31 Posted October 13, 2021 It may not be useful but I will look my settings too. Just in case. 2 Quote Link to comment Share on other sites More sharing options...
Kouill Posted October 13, 2021 Share #32 Posted October 13, 2021 Done with my settings too Quote Link to comment Share on other sites More sharing options...
Orphée Posted October 13, 2021 Share #33 Posted October 13, 2021 I did it too. Quote Link to comment Share on other sites More sharing options...
WiteWulf Posted October 13, 2021 Author Share #34 Posted October 13, 2021 Thanks so much for taking the time to do this you two, I know it’s time consuming. I’ll have a look at the data properly later on. Quote Link to comment Share on other sites More sharing options...
Orphée Posted October 13, 2021 Share #35 Posted October 13, 2021 I have a new line compared to yours, because I have a PCIe card plugged (IRQ and somewere else, did not save it) Quote Link to comment Share on other sites More sharing options...
Orphée Posted October 14, 2021 Share #36 Posted October 14, 2021 (edited) @WiteWulf where do you setup nmi_watchdog=0 and softlockup_panic=0 ? Edit : in command lines boot OK, thanks 😛 Edited October 14, 2021 by Orphée Quote Link to comment Share on other sites More sharing options...
WiteWulf Posted October 14, 2021 Author Share #37 Posted October 14, 2021 (edited) They're passed to the kernel at boot time, so technically they're grub.cfg parameters, but practically on redpill they should go in the "extra_cmdline" stanza of <platform>_user>config.json, eg. { "extra_cmdline": { "pid": "0xc75c", "vid": "0x05dc", "sn": "1230LWNXXXXXX", "mac1": "001132XXXXX", "mac2": "001132XXXXXX", "nmi_watchdog": "0" }, "synoinfo": {}, "ramdisk_copy": {} } They can also be set manually on a running system, eg: echo 0 > /proc/sys/kernel/nmi_watchdog But this won't persist a reboot, obviously. Edited October 14, 2021 by WiteWulf 1 Quote Link to comment Share on other sites More sharing options...
WiteWulf Posted October 14, 2021 Author Share #38 Posted October 14, 2021 I've gone through the spreadsheet and highlighted differences in configuration that I think may be relevant (I've ignored cosmetic stuff like power on boot logos and fn key prompts). What stands out to me is that @Kouill is running slower RAM, but that matches their CPU and is configured appropriately in the "Maximum Memory Bus Frequency" section of the BIOS. Also, I've got "Intel Virtualization Technology" disabled, as I'm not running a hypervisor and read that's best to have it turned off in that situation. But can this affect docker? It shouldn't, as docker is basically a glorified chroot 🤔 We've all got slightly different IRQ assignments, too. Quote Link to comment Share on other sites More sharing options...
Orphée Posted October 14, 2021 Share #39 Posted October 14, 2021 Well, about mem frequency : Their max frequency is 1866, but they are running at 1600 MHz. Quote Link to comment Share on other sites More sharing options...
WiteWulf Posted October 14, 2021 Author Share #40 Posted October 14, 2021 Hmm, yeah, that too 🤔 Why are you underclocking your RAM @Kouill? Quote Link to comment Share on other sites More sharing options...
Orphée Posted October 14, 2021 Share #41 Posted October 14, 2021 (edited) Maybe he doesn't ! Our rams are underclocked to match CPU/MB requirement Maybe he's RAMs are real 1600 MHz. Edited October 14, 2021 by Orphée Quote Link to comment Share on other sites More sharing options...
WiteWulf Posted October 14, 2021 Author Share #42 Posted October 14, 2021 On the hardware tab it shows: 2x 8GB 1600MHz DDR3 ECC But in the BIOS setting for "Maximum Memory Bus Frequency" it's explicitly set to 1300MHz. Ah, Intel says max memory speed for that CPU (E3-1260L) is 1300MHz: https://ark.intel.com/content/www/us/en/ark/products/52275/intel-xeon-processor-e3-1260l-8m-cache-2-40-ghz.html That's correct Quote Link to comment Share on other sites More sharing options...
Orphée Posted October 14, 2021 Share #43 Posted October 14, 2021 Ah ! My bad Quote Link to comment Share on other sites More sharing options...
Kouill Posted October 14, 2021 Share #44 Posted October 14, 2021 (edited) My choices for ram speed in bios are 1066 or 1333. I could let it in auto that's true but I've already had issues in the past with non hp ecc recommanded ram with the auto settings. For the Intel Virtualization Technology enabled, i have linux/windows VM with Virtual Machine Manager. Edited October 14, 2021 by Kouill 1 Quote Link to comment Share on other sites More sharing options...
haydibe Posted October 15, 2021 Share #45 Posted October 15, 2021 On 10/14/2021 at 12:30 PM, WiteWulf said: Also, I've got "Intel Virtualization Technology" disabled, as I'm not running a hypervisor and read that's best to have it turned off in that situation. But can this affect docker? It shouldn't, as docker is basically a glorified chroot 🤔 It does not. Docker just uses CGROUPS (=what ressources it is allowed to use), Kernel Namespaces (=comparable to fenced partition of kernel components) and capabilities (=what low level features is the processes allowed to use). From the host perspective, container are merly fenced processes that are restricted to see whatever part of the host it is allowed for them to see. The container itself has no idea that it is restricted from the outside (this is not 100% true because a container can access some stats from the outside like the host cpu count and the installed total memory... containerized java proccesses suffer from this if done wrong). 1 Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.