We want to test a fix for bz#2263643, a GRUB out of memory error that has been preventing fedora installation on new gaming hardware, such as ASUS machines, because GRUB was running out of RAM to load the installation initramfs. We particularly want older and diverse hardware to be tested because the fix appears to work on new hardware, but could fail on older hardware.
All that is needed is to boot into the installer from this iso https://people.redhat.com/lsandova/oom/boot-max.iso It's not necessary to actually perform the installation.
We need to know if it was possible to boot to the installer, and we'd like the output of sudo lshw -short so that we know what hardware it was run on.
sudo lshw -short
Oh, time frame... sometime next week would be good, if possible. Otherwise we can wait until the new year.
In case you have a system that you encounter the OOM but you workaround it somehow, e.g. disabling TPM or other firmware features, one can also try installing rpms with fix on it [1], reboot, enable all disable firmware and continue. This check complements the ISO tests, either way is good feedback for us.
[1] https://src.fedoraproject.org/rpms/grub2/pull-request/198
Metadata Update from @kparal: - Custom field story_points adjusted to 2 - Issue set to the milestone: Fedora 44 - Issue tagged with: test days
This is a good case for organizing a test day.
We can let it run the whole next week, is that OK, or do you want just certain days?
Because this is a somewhat non-standard test day aimed at verifying a single regression, I wouldn't create a special test case page for it, it would have no future use. Instead, we can simply provide the instructions in the test day wiki page, and link to the right section in it. I'll create the wiki skeleton to show what I mean.
Since I won't be around next week, I'd need someone else to handle this event. @psklenar @lruzicka @jgroman any volunteers?
I've created: https://fedoraproject.org/wiki/Test_Day:2025-12-15_GRUB_out_of_memory_verification https://testdays.fedoraproject.org/testday/11
On the wiki, please edit it and update everything important. Don't forget about:
When announcing this test day, we should be explicit that we're looking for as large coverage as we can. While the bug was about ASUS gaming systems, we want to verify everything, all brands, old and new. This is extremely easy to participate, just boot from a usb drive. Help us spread the word. We'll cover our mailing list and Discussion, spread the word in your channels, internally, etc.
we'd like the output of sudo lshw -short
Oh, this one is a bit tricky, where to store the output? My best idea is to ask people to upload it to https://pastebin.com with Expiration=Never (other pastebins I know of have expiration in days/months) and include a link in the comment when submitting results. We can include this in wiki test instructions. Thoughts?
I've updated the wiki with pastebin instructions. Feel free to tweak/change.
This is a good case for organizing a test day. We can let it run the whole next week, is that OK, or do you want just certain days?
All next week is great. Hopefully the longer it runs, the more responses we get :)
we can simply provide the instructions in the test day wiki page, and link to the right section in > it. I'll create the wiki skeleton to show what I mean.
Sure, that makes sense. We will fill it in in the meantime. thanks!
So after trying the ISO myself, I have two comments: 1. Please inform people that SecureBoot needs to be disabled in order to boot it. If they see an Access Violation (or similar) message while trying to boot, it's SecureBoot blocking it. 2. It boots into anaconda environment and not Live environment. Which is fine, but it means that people can't use it to collect lshw output and upload it. That's a bit unfortunate, it means we'll need to tell them to boot their existing Linux system (if they have it installed), or boot a regular F43 Workstation Live image, to collect the logs and upload them. Slightly inconvenient, but I don't see an easier, unless you want to build a Live image instead.
Metadata Update from @kparal: - Issue assigned to jgroman
Will do about Secure Boot! good catch; thanks.
About the live image, I defer to Leo and what he wants to do... whether he wants to build the live image or not.
Good inputs @kparal !
I believe having a live image would be better so people can query the HW easily. Regarding Secure Boot off/on, lets just explicitly indicate users to turn off SB for the moment, this is not relevant for the moment.
I did some tweaks to https://fedoraproject.org/wiki/Test_Day:2025-12-15_GRUB_out_of_memory_verification , see History if interested.
Is the second test case (update RPMs on an existing install) happening?
Also, do you have your own Matrix channel to use?
great, thanks
I would consider it as optional... I will add instructions
let me create one because I have not used Matrix before
Something important to indicate (which I have done it https://fedoraproject.org/w/index.php?title=Test_Day:2025-12-15_GRUB_out_of_memory_verification&diff=prev&oldid=759375) is that ISO is signed, so there is NO NEED to disable SB.
I created this room for testing #fedora_GRUB_OOM:matrix.org is this ok, @kparal? I'm having issues using fedora.im, not sure why.
Thank you for reformatting the directions and everything. it looks much better than what I was able to do :)
Oh, I didn't mean you should create a special Matrix room just for this. We do have a special room already, #test-day:fedoraproject.org :-) So let's use that one (I edited the wiki). The idea was that if you have some permanent channel that you actively use and you have lots of members and contributors already present in it (for example, Anaconda folks have #anaconda:fp.o), it's better to use it, because testers can get better and faster responses there, and devs some visible feedback. But if you don't use Matrix regularly, no worry! Let's use our default channel instead.
#test-day:fedoraproject.org
#anaconda:fp.o
Folks, please publicize this test day in any channel that you find useful (even RH internal ones). Link to https://fedoraproject.org/wiki/Test_Day:2025-12-15_GRUB_out_of_memory_verification as the main entrance point. If you have a blog syndicated to https://fedoraplanet.org , that would be useful, or you can submit an article to https://communityblog.fedoraproject.org/ (ask them to publish it on Monday) and it will appear there as well. Some other ideas here: https://fedoraproject.org/wiki/QA/SOP_Test_Day_management#Promote_the_Test_Day
QA will email test/test-announce list and post an announcement to Discussion. @jgroman I usually schedule an email for morning of the first day. The Discussion post can also be scheduled, but it's complicated, better to just post it on Monday morning manually, in our #quality-team tag.
@jgroman Please also create the calendar entry, ideally today. It has a delay until it appears in external sources like Google Calendar.
I'll remove it from the testdays app for the moment, until the test case exists. We can then easily add it, just ping us here.
It seems this went well. @jgroman Please export the results to the wiki and close this ticket, thanks!
Metadata Update from @jgroman: - Issue close_status updated to: Fixed - Issue status updated to: Closed (was: Open)
@kparal @adamwill @mlewando
We have two regression [1,2] with the proposed fix [3] so unfortunately this is not yet over. Not sure who to proceed so asking here before asking devs for more info.
This is the status: During the test day, most testers boot fine so we merged [3]. During the previous weekend (10-11 Jan), people find boot issues [1] so [3] was untagged [4]. We are suspecting that this is something related to the filesystem, specially btrfs filesystems and this is why during the test day using a Live OS did not catch it.
Can we somehow re-open the test-day and ask people for further testing? OR would it be worth having a separate test-day, this time testing dnf upgrading instead of Live OS booting? Or do nothing on this area and ask directly on tickets [1,2] and [5] (this is the original bz)?
this is the extra steps I want testers to do it
Instructions:
Assuming you are on a running rawhide systems, enable grub debug and upgrade to 2.12-50 with the following
$ sudo dnf install -y koji $ grub2-editenv - set debug=all,-lexer,-scripting $ grub2-editenv - set pager=1 $ mkdir grub2-2.12-50 $ cd grub2-2.12-50 $ koji download-build --arch=x86_64 --arch=noarch grub2-2.12-50.fc44 $ sudo dnf install *.rpm $ lsblk -f # place it on a pastebin, never expire $ sudo reboot # in the same pastebin as above, place any boot error seing $ sudo dnf downgrade grub2*
[1] https://bugzilla.redhat.com/show_bug.cgi?id=2427945 [2] https://bugzilla.redhat.com/show_bug.cgi?id=2422881 [3] https://src.fedoraproject.org/rpms/grub2/pull-request/198 [4] https://forge.fedoraproject.org/releng/tickets/issues/13157 [5] https://bugzilla.redhat.com/show_bug.cgi?id=2263643
Hi Leo. Here's what we can do: 1. Ask people in those bugzilla reports that the fix works OK for them 2. (less work) Ask people in test list and in Discussion (under those original test day requests - discussion, test-announce but reply to test) to follow up with this new build and report whether it works fine for them on installed systems 3. (more work) Create a new test day, call it e.g. "Grub OOM round 2", and prepare two test cases this time - a Live image and an installed system - because clearly both are important. Or at least the second one. This is not a problem, we can set it up. I would avoid re-using the past test day, honestly, that might be confusing.
We can of course do both 2+3, if you wish.
thanks @kparal , let start with 2 and start planning 3 if required.
Hi Leo. Here's what we can do: 1. Ask people in those bugzilla reports that the fix works OK for them
what do you mean ?
(less work) Ask people in test list and in Discussion (under those original test day requests - discussion, test-announce but reply to test) to follow up with this new build and report whether it works fine for them on installed systems
great, I do that. I am not yet subscribed to the test-announce list, so let me do that first.
(more work) Create a new test day, call it e.g. "Grub OOM round 2", and prepare two test cases this time - a Live image and an installed system - because clearly both are important. Or at least the second one. This is not a problem, we can set it up. I would avoid re-using the past test day, honestly, that might be confusing.
ok, let me plan on this.
yes. This time I hope we get more testers, so 2+3 makes sense.
Simply to make sure the original reporters of the problem confirm that the fix works for them :-) I did not have time to review the bugs, you probably did that already, I just mentioned this for completeness-sake.
This issue has been migrated to Fedora Forge: https://forge.fedoraproject.org/quality/tickets/issues/853
Please continue any further discussion there.