https://kojipkgs.fedoraproject.org/repos-dist/f39-coreos-continuous/

I just regenerated this one so it now has the latest link again and is no longer a good example :)

This is not related to #1581, it's a side effect of the kojira work for on-demand repos.

With the current code, dist repos remain in the ready state for the $dist_repo_lifetime and are then expired. Once expired they deleted after $expired_repo_lifetime. It's about the same for non-dist, but there are some extra checks before expiration.

Cleanup for dist-repos is kind of an oddity. It's never been clear what the need here is. Koji only knows that the dist repo was created. It doesn't have much context as to why they might be needed. Koji itself doesn't need them at all.

Prior to 1.35, yes, the latest dist repo was kept alive (because it would only be marked expired when a newer dist repo took the mantle of latest). This lifespan was more of an accidental inheritance of behavior from normal repos than any explicit design. Many parts of the repo lifecycle changed in 1.35.

It's worth noting that normal repos are also removed similarly in the 1.35 code. We have a pending change to alter that. See #4270.

I could add some code to #4270 to extend the "keep latest" behavior for dist-repos, but the situation is different. Again, it's really hard for Koji to know whether a dist repo is needed. There are many knobs to twiddle for dist repos and latest doesn't tell the whole story.

If I'm not mistaken, the original intent with dist repos was for the content to be copied elsewhere if it was going to be used for important things. This is why dist repos have all the rpms hardlinked or copied into the tree rather than simply accessed via the toplink symlink like with normal koji repos.

I'm curious about the use cases here? What is driving you to use dist-repos instead of the normal ones? The coreos ones appear to not be using any of the dist-repo options?

So our use case for the -infra and -infra-stg ones is that we want signed rpms that we can then just use on our machines to provided fast trask updates or rpms that are not in fedora/epel/rhel.

coreos uses that one I think to just build other stuff, but I am unsure why they don't use buildroot repos.

I'm fine if the expectation is that you should sync the repos off to another location, but it would be good to document that if thats the case?

But keeping latest would save a bunch of complexity in syncing and other storage, etc. That would of course add a storage burden to the koji storage...

Oh, I guess coreos uses those because they still sometimes need to build things from tags that no longer have targets (and thus no buildroot)

Sometimes we fast track newer versions (not yet in bohdi stable) or freeze on older versions of packages for COSA (our build environment for CoreOS) so we use the coreos continuous repos for that purpose. Of course, after EOL, it's primarily for the "freeze on older version" part and not the fast-tracking, not unless we convinced a package maintainer to build and RPM that didn't get shipped in bodhi for whatever reason.

So our use case for the -infra and -infra-stg ones is that we want signed rpms

I figured. Yours are the only recent dist repos that I could find that actually seem to use a dist-repo feature.

Sometimes we fast track newer versions (not yet in bohdi stable) or freeze on older versions of packages for COSA (our build environment for CoreOS) so we use the coreos continuous repos for that purpose.

Otoh, all of this sounds like it can be solved with normal repos

So our use case for the -infra and -infra-stg ones is that we want signed rpms

I figured. Your's are the only recent dist repos that I could find that actually seem to use a dist-repo feature.

well, CentOS too, for the https://cbs.centos.org instance : once tagged to specific tags, message is emitted on message bus and signing process kicks in : packages are downloaded from koji, gpg signed, imported back into koji, dist-repo task launched and composed tree is then shipped out to mirrors (https://docs.centos.org/centos-sig-guide/delivery/) ... hopefully something that, as a CentOS Board member, you were aware of :D

dist-repo use for composes, like the CentOS case above, are why dist-repos were created. I don't think cases like that are going to be affected by the more aggressive cleaning noted here (because composes tend to generate fresh dist-repos often and end up copying the data).

Regardless, #4270 should now address the situation that prompted this.

FWIW, none of our dist-repos are multilib, but we have been talking about using dist-repos to make openh264 repos... and there's a request to make those multilib.

However, I see that that is broken currently (and also it's unclear what it expects you to pass to --multilib and tag2distrepo does not support that. ;)

Thanks for that info. There's some discussion of multilib in #4293

adding note here for my future self as I have to run this periodically for the time being:

for num in {39..41}; do koji dist-repo --non-latest "f${num}-coreos-continuous" --allow-missing-signatures; done

Commit 28a2da91 fixes this issue

Commit c0690c82 fixes this issue

This issue has been migrated to Fedora Forge:
https://forge.fedoraproject.org/koji/koji/issues/4295

Please continue any further discussion there.