As kojid is concerned I see two ways to fix the issue:

• Either have a boolean that have use_createrepo_c_compat=True (or else)
  Then add this --compatibility field to createrepo_c and mergerepos_c commands assuming the compatibility works in all cases (past and future - yet to be defined properly).
  This still requires an updated createrepo_c to handle "properly" the --compatibiity flag.

• Or pass the appropriate options in the kojid.conf, to be tuned as for any koji builders instances. like --general-compress-type=xz --uncompress-comps-as-group (fictional option, doesn't exist yet), etc.

Also to be noted that I'm searching for a way to have a koji-hub policy to redirect some regen-repo task to certain builders. But it doesn't seem to work yet (task are still redirected to the default channel).

Example:
https://koji.rpmfusion.org/koji/taskinfo?taskID=618912
Instead of:
https://koji.rpmfusion.org/koji/taskinfo?taskID=618894

--
channel =
method createrepo && buildtag el :: use createrepo
method createrepo && buildtag f3 :: use createrepo
method createrepo && buildtag f4* :: use powerbuilder
[...]
all :: use default

Createrepo uses the regular createrepo_c

But to me this is still a hack and not a long term fix because I should use only one createrepo_c version to support all cases (like we have today).

Also createrepo_c from the builder version is assumed, so there is a need to have a recent builder to read the zck metadata. It's not possible to use createrepo_c from the target buildroot because then it will lead to a chicken and egg problem. (createrepo_c and mergerepos_c are used to populate the buildroot)

1) Making it more configurable makes sense to me. Having possibility to use improved repo format for some tags is valuable. Not sure if want to allow arbitrary options as they could interfere with what koji needs to set.
2) Routing newRepo/createrepo tasks anywhere else is not possible. Policy data for this type of task are very limited. What is maybe reasonable is to extend policy_data_from_task_args to expose also tag?

@mikem ?

Routing newRepo/createrepo tasks anywhere else is not possible. Policy data for this type of task are very limited. What is maybe reasonable is to extend policy_data_from_task_args to expose also
tag?

In principle, sure. Though might be a little messy.

The createrepo task in particular doesn't have a tag arg. It takes (repo_id, arch, oldrepo). Though I guess we could fetch the tag from the repo_id.

OTOH, newRepo does have a tag arg and you could just let the subtasks inherit the channel.

Several methods have a build_tag arg. Just a few have tag arg (newRepo, tagNotification, waitrepo, tagBuild)

At any rate, this is worth a separate rfe to hash it out.

Certainly it seems that koji needs to be more flexible in how it generates repos. Does it need to be per-tag though?

Historically, the repos have been things that the builders consume directly. That is, it is the dnf/yum on the builder that reads the repo, not the one in the chroot. So, the mismatch is not so much different buildroot content as it is different builder content.

If anything, koji should err on the side of highest compatibility with the repos it generates. I'm quite surprised (ok, maybe not so much surprised as disappointed) that createrepo_c defaults to generating repos that cannot be read by a still-supported RHEL version.

In the Open Build Service, there's two types of rpmmd publishing modes: regular and -legacy (ie. rpmmd-legacy). Maybe we could do something similar here?

So... where are we here?

Basically this is causing issues for fedora because if we update builders to createrepo_c 1.0+ we will break all epel7 buildroots.
But we want to upgrade so things like updates and rawhide composes use the newer one with specific settings.

Granted the clock is winding down on epel7, but it would be nice to not have to hold everything until that goes away. ;)

@kevin

/me is back from vacation today. and our infra is failing wrt to having zstd in rawhide that mandates createrepo_c 1.0.x

Basically, nothing has changed since the initial report at https://pagure.io/fedora-infrastructure/issue/11637

Still trying to find workarounds...

Tormorrows rawhide will again revert back to gz... and we will leave it alone until 2024-06-30, unless some solution shows up here before then.

@kevin

Thanks for the revert. This will solve the problem. When el7 will be EOL, it will be easier to deal with.

Still the problem is concerning. We were very careful to isolate environments when building packages, so features that land in development will not affects older releases.

But here, at least when koji is used, software editors cannot use the same instance to build for el7 and development fedora versions.
Of course the root cause is probably landing support of zstd in createrepo_c 1.0.x and in the same time, default to zstd in the same version. Comparing to the change from groupinstall to install @group given the syntax is supported back to RHEL5, but there must be something in-between as a sane change government.

PR #4121 will allow channel policy to redirect newRepo tasks based on tag

Commit a17395a4 relates to this ticket

This issue has been migrated to Fedora Forge:
https://forge.fedoraproject.org/koji/koji/issues/3945

Please continue any further discussion there.