#302 llhttp rebase
Closed: Deferred by carlwgeorge. Opened by carlwgeorge.

There is an incoming rebase of llhttp in CentOS 10. It has already been built in the CentOS koji, but hasn't yet been approved. This update changes the library soname from 9.1 to 9.2. Once this change lands on the mirrors it will be synced into the EPEL 10 external repo. I believe once this happens it will break EPEL 10 builds, because it will make rpmautospec uninstallable (via transitive dependencies).

The last time we needed to resolve a problem like this, we were able to override tag Fedora 40 builds to get a working buildroot to build the necessary packages. I don't think we'll have that luxury this time around, as I don't see any Fedora or ELN libgit2 builds in our koji that meet our needs (libgit2 1.7.2 linked against llhttp 9.2). CentOS also has to address this in their buildroot, but whatever solution they use won't be directly consumable by EPEL 10 since they are on a different koji instance.

My proposed solution to this is an llhttp9.1 compat package. Currently this package is not allowed by EPEL policy since it has a file conflict with llhttp 9.1 (/usr/lib64/libllhttp.so.9.1). Once llhttp 9.2 lands, this compat package would be allowed by policy. However, at that point the buildroot will be broken and it wouldn't be able to be built. To work around this conundrum, I went ahead and built it in a side tag. If we don't agree on this path as a solution, it's no trouble to delete the side tag and archive the dist-git repo. I just knew we needed to have it built before the buildroot was broken so it would be an option. If we do agree this is the desired solution, there are two ways we can go about rolling it out.

My preferred solution would be to merge this side tag now, and allow llhttp9.1 and llhttp 9.1 to coexist temporarily, which would be an exception to the policy of not conflicting with packages in the target base. I think this would avoid the buildroot breakage when llhttp 9.2 lands, because rpmautospec and its dependencies will stay installable both before and after. Afterwards, we can rebuild libgit2 against llhttp 9.2 and retire llhttp9.1.

The alternative would be to wait for llhttp 9.2 to land, and then rebuild libgit2 against llhttp 9.2 in the side tag. That build would be instantiated with rpmautospec using the llhttp9.1 package, but the resulting package would link against llhttp 9.2. After the libgit2 build is done, we can untag llhttp9.1 from the side tag and merge it. llhttp9.1 would never be published, and could be retired. While this approach strictly follows EPEL policy, it would result in the buildroot being broken for everything outside the side tag for a period of time.

There are probably other ways to go about solving this, but the compat package seems like the best approach to me.


We're investigating two other options in CS 10.

  1. The specific CVE that triggered the llhttp maintainer to rebase doesn't actually need to do a rebase. We could backport just the fix and maintain the API/ABI. This would work for now and kick the can down the road.
  2. We could modify libgit2 so that it carries a bundled version of llhttp internally so that we don't have a bootstrapping problem when llhttp rebases to an ABI-incompatible version. It would need to be rebuilt any time llhttp is updated.

The specific CVE that triggered the llhttp maintainer to rebase doesn't actually need to do a rebase. We could backport just the fix and maintain the API/ABI. This would work for now and kick the can down the road.

This one worries me long term, but it would indeed avoid the the problem for now. If we do think we will need to do this eventually anyways, it would be much better to do it now before the CentOS and EPEL 10 launch announcements (which are both tentatively planned for next month).

We could modify libgit2 so that it carries a bundled version of llhttp internally so that we don't have a bootstrapping problem when llhttp rebases to an ABI-incompatible version. It would need to be rebuilt any time llhttp is updated.

For clarity, this is in regards to the libgit2 package that is in the CentOS 10 buildroot. It is not shipped in the distro, so we also have a separate libgit2 in EPEL 10. I can see the value in having these packages take the same approach to stay as close as possible to reduce disruption on the off chance that the CentOS libgit2 moves from the buildroot to CRB in the future. I will point out that the bundled llhttp wasn't added until libgit2 version 1.8.1, while CentOS and EPEL both have 1.7.2. Such an approach would involve manually adding the bundled source or rebasing the libgit2 version. Both are doable, but IMO neither are appealing.

We have libgit 1.7.2 in CS 10 / RHEL, but it's carrying the patches I wrote to replace http-parser with llhttp.

We could switch that to static rather than dynamic linking.

We discussed this at today's EPEL Steering Committee meeting. We agreed the best approach was to grant a temporary exception to policy to allow llhttp9.1 and llhttp (9.1) to coexist temporarily. I've submitted an update from that side tag. Once llhttp is rebased, I'll circle back and rebuild libgit2 (and any other relevant EPEL 10 packages) against the new soname, and then retire llhttp9.1.

llhttp had it's CVE's backported, so there was no soname bump. We can see the update here.
https://gitlab.com/redhat/centos-stream/rpms/llhttp/-/merge_requests/4
https://kojihub.stream.centos.org/koji/buildinfo?buildID=70042

Since the rebase isn't happening, llhttp9.1 is no longer needed. I've retired llhttp9.1 on the epel10 branch and untagged the build.

Metadata Update from @carlwgeorge:
- Issue close_status updated to: Deferred
- Issue status updated to: Closed (was: Open)

Metadata Update from @carlwgeorge:
- Issue untagged with: meeting

This issue has been migrated to Fedora Forge:
https://forge.fedoraproject.org/epel/steering/issues/302

Please continue any further discussion there.

Metadata