releng / failed-composes

#5548 Fedora-Rawhide-20231021.n.0 DOOMED
Closed by jnsamyak. Opened by releng.

Closed
Reopen Issue

pungi.global.log

[OSTREE_INSTALLER] [ERROR   ] [FAIL] Ostree installer (variant Kinoite, arch aarch64) failed, but going on anyway.
[OSTREE_INSTALLER] [ERROR   ] Runroot task failed: 107863434. See /mnt/koji/compose/rawhide/Fedora-Rawhide-20231021.n.0/logs/aarch64/Kinoite/ostree_installer-1/runroot.log for more details.

[OSTREE_INSTALLER] [ERROR   ] [FAIL] Ostree installer (variant Kinoite, arch x86_64) failed, but going on anyway.
[OSTREE_INSTALLER] [ERROR   ] Runroot task failed: 107863605. See /mnt/koji/compose/rawhide/Fedora-Rawhide-20231021.n.0/logs/x86_64/Kinoite/ostree_installer-3/runroot.log for more details.

[OSTREE_INSTALLER] [ERROR   ] [FAIL] Ostree installer (variant Kinoite, arch ppc64le) failed, but going on anyway.
[OSTREE_INSTALLER] [ERROR   ] Runroot task failed: 107863685. See /mnt/koji/compose/rawhide/Fedora-Rawhide-20231021.n.0/logs/ppc64le/Kinoite/ostree_installer-2/runroot.log for more details.

[OSTREE_INSTALLER] [ERROR   ] [FAIL] Ostree installer (variant Silverblue, arch ppc64le) failed, but going on anyway.
[OSTREE_INSTALLER] [ERROR   ] Runroot task failed: 107863973. See /mnt/koji/compose/rawhide/Fedora-Rawhide-20231021.n.0/logs/ppc64le/Silverblue/ostree_installer-7/runroot.log for more details.

[LIVE_MEDIA      ] [ERROR   ] [FAIL] Live media (variant Labs, arch *, subvariant Games) failed, but going on anyway.
[LIVE_MEDIA      ] [ERROR   ] Live media task failed: 107865090. See /mnt/koji/compose/rawhide/Fedora-Rawhide-20231021.n.0/logs/x86_64/livemedia-Labs-Games.x86_64.log for more details.

Variant: Labs, subvariant: Games task failed. Pinging maintainers: @richardanaya @dagostinelli 

[LIVE_MEDIA      ] [ERROR   ] [FAIL] Live media (variant Labs, arch *, subvariant Scientific_KDE) failed, but going on anyway.
[LIVE_MEDIA      ] [ERROR   ] Live media task failed: 107865089. See /mnt/koji/compose/rawhide/Fedora-Rawhide-20231021.n.0/logs/x86_64/livemedia-Labs-Scientific_KDE.x86_64.log for more details.

Variant: Labs, subvariant: Scientific_KDE task failed. Pinging maintainers: @scitech 

[ERROR   ] [FAIL] Live media (variant Spins, arch aarch64, subvariant i3) failed, but going on anyway.
[LIVE_MEDIA      ] [INFO    ] [DONE ] Live media: Fedora-i3-Live (arches: aarch64 x86_64, variant: Spins, subvariant: i3) (task id: 107865125)

[ERROR   ] [FAIL] Live media (variant Spins, arch aarch64, subvariant KDE) failed, but going on anyway.
[LIVE_MEDIA      ] [INFO    ] [DONE ] Live media: Fedora-KDE-Live (arches: aarch64 x86_64, variant: Spins, subvariant: KDE) (task id: 107865100)

[ERROR   ] [FAIL] Live media (variant Spins, arch aarch64, subvariant LXQt) failed, but going on anyway.
[LIVE_MEDIA      ] [INFO    ] [DONE ] Live media: Fedora-LXQt-Live (arches: aarch64 x86_64, variant: Spins, subvariant: LXQt) (task id: 107865116)

[ERROR   ] Compose run failed: Live media task failed: 107865130. See /mnt/koji/compose/rawhide/Fedora-Rawhide-20231021.n.0/logs/aarch64-ppc64le-x86_64/livemedia-Workstation-Workstation.aarch64-ppc64le-x86_64.log for more details.

Compose Total time: 4:03:09
Compose phase INIT time: 0:00:22.
Compose phase PKGSET time: 0:27:33.
Compose phase WEAVER: FAILED.
Compose phase BUILDINSTALL time: 1:32:16.
Compose phase GATHER time: 1:34:49.
Compose phase OSTREE time: 0:46:02.
Compose phase OSTREE_INSTALLER time: 1:04:26.
Compose phase CREATEREPO time: 0:08:19.
Compose phase CREATEISO time: 0:09:13.
Compose phase REPOCLOSURE time: 0:01:31.
Compose phase IMAGE_BUILD time: 1:43:29.
Compose phase LIVE_MEDIA: FAILED.
Compose phase OSBUILD time: 0:55:56.

So this is kinda weird. Ever since Fedora-Rawhide-20231019.n.2 , the Workstation live build task - only that one - has failed in every attempt. It's the top-level task that fails, not the actual per-arch image build tests. Here's the one from this compose. It fails with just a Python exception - <class 'TypeError'>: %d format: a real number is required, not NoneType - shown in the Koji "Result" field. No log links or anything.

I don't actually know how to debug this further. I've logged into the builders of the last two failed tasks but can't find anything related to them still present on them. There's been no change to fedora-kickstarts or fedora-comps in the last few days that I can see, nor to pungi-fedora.

There must be some difference between Rawhide and F39 here, as the F39 composes aren't failing on this.

@kevin when you're back we may need a lil help here :)

Took a while, but I think I have figured this out.

The problem was the osbuild workstation builds started working, and thats done before the livemedia phase, so when it gets there it tries to build the workstation live and... there's already a build with that exact same name/version/release. Because of that koji doesn't increment the release right and the task fails.

I am trying to just disable the osbuild workstation and refiring a compose. If that works, then I was right and we need to get the osbuild version using a different name or something...

Metadata Update from @jnsamyak:
- Issue status updated to: Closed (was: Open)
Metadata
None

DOOMED Fedora Rawhide

None
None
None
Static archive. pagure.io was sunset in 2026; this is a read-only snapshot.
Red Hat Logo

Fedora is sponsored by Red Hat. Learn more about the relationship between Red Hat and Fedora »

© Red Hat, Inc. and others.