As discussed in today's weekly meeting, there is some work required to be done prior to creating the release.

We are waiting on this as we dont want to create the release until we are ready and we are currently blocked by upgrade of our z machines to z15

What is the definition of "we are ready" ?
Before, that definition was we needed epel-release.
We now have fedpkg-minimal, epel-release, and epel-rpm-macros all built, and ready to go through bodhi.

It is true that we cannot build packages that need s390x.
But I don't see why we cannot get everything ready and pushed through so that when the s390x machines come online, we are able to use them. Otherwise, we will have yet more delays once they come online.

I have scripts I am trying to write, but I'm blocked on there being no packages in the repos. In my opinion the best way to get these packages into the repo is via the proper path, which is through bodhi.

What is the definition of "we are ready" ?

When we are able to build archful packages, even the noarch packages will have problems when they hit any s390x builder, until the builders are updated to z15.

Before, that definition was we needed epel-release.
We now have fedpkg-minimal, epel-release, and epel-rpm-macros all built, and ready to go through bodhi.

It is true that we cannot build packages that need s390x.

Yes

But I don't see why we cannot get everything ready and pushed through so that when the s390x machines come online, we are able to use them. Otherwise, we will have yet more delays once they come online.

The update to z15 should be completed by the end of this, but haven't heard any update on it. Kevin is checking around.

I have scripts I am trying to write, but I'm blocked on there being no packages in the repos. In my opinion the best way to get these packages into the repo is via the proper path, which is through bodhi.

I understand, but having builder problems is not fun especially when we cant control which builders gets picked. Hopefully it will be resolved soon.

Can we just have qemu-user emulate z15 for Mock for s390x? That would let us get going now.

What is the definition of "we are ready" ?

When we are able to build archful packages, even the noarch packages will have problems when they hit any s390x builder, until the builders are updated to z15.

I don't understand that correlation. Are we concerned that someone will look at Bodhi and say "I must be able to build now because there is a epel9-next repository in bodhi"?

Whatever. Just know that I am blocked because of bodhi, not because of the builders.

Personally I'm not concerned about the release being added to bodhi early to allow us to get epel-release, epel-rpm-macros, and fedpkg-minimal into the repo properly. There are a roadblocks in place that will prevent packagers from submitting updates for this release.

1. The released version of fedpkg can't request epel9-next git branches yet. That has been added upstream but it isn't in a tag yet, much less in the fedpkg rpm packagers have installed.
2. You need a successful koji build for a release in order to submit a bodhi update for that release. Any archful koji builds that are submitted will fail because of the s390x problem. Noarch builds would pass, but I can't imagine packagers trying that until they have the git branch.

Once these roadblocks are resolved, we'll be ready to launch epel9-next. Just for clarity we could also send an email to epel-devel to let people know that they will see the bodhi release but it isn't ready quite yet, and another email will be coming when it is ready.

...snip...

1. You need a successful koji build for a release in order to submit a bodhi update for that release. Any archful koji builds that are submitted will fail because of the s390x problem. Noarch builds would pass, but I can't imagine packagers trying that until they have the git branch.

Does it fail? Or just build and then become unusable on a z14 based centos-9-stream host?

The builds fail. Here is an example of a failed build
https://koji.fedoraproject.org/koji/taskinfo?taskID=77799032
BuildrootError: could not init mock buildroot, mock exited with status 30; see root.log for more information

https://kojipkgs.fedoraproject.org//work/tasks/9233/77799233/mock_output.log

While installing the packages mock gives the error
"Fatal glibc error: CPU lacks VXE support (z14 or later required)"
Thus the packages never get installed, and there is no mock chroot to build upon.

Closing this ticket as the work is done, while the s390x upgrade is being tracked internally.