pungi

#862 Handling traditional packages and modular ones with the same name
Closed: Fixed Opened by lsedlar.

Closed: Fixed
Reopen Issue

There's a deep baked assumption in Pungi that there will ever be one package set for a single compose.

This is causing problems in current Rawhide compose (and probably Branched):

  • Modular variant contains only the modular builds as expected, this is fine.
  • Everything variant is configured to include everything, so it really pulls in all packages from the set, including modules.

What happens in building the package set: pungi creates one package set for each tag it works with (one for traditional content, plus one for each module). Since the code expects to have a single package set, they need to be merged. Additionally it only includes each package once, with preference for builds from modules. This merge is what is currently slowing the compose down (by about 2 hours for Rawhide).

See the Not merging in RPM messages in the log – that means that a package with identical name was already included from some module.

Later when packages for Everything variant are getting gathered, the modular builds get in.

If we just remove the filtering during merging, the latest version (by just comparing versions) will end up everywhere.

Specific example: Fedora-Rawhide-20180303.n.0 used python2-markdown-2.4.1-11.module_0b083f9b.noarch.rpm in Everything, excluding python2-markdown-2.6.11-1.fc28.noarch.rpm.

@sgallagh Is the correct behaviour for Modular to contain the modular build, but Everything get the pure fc28 build?

@lsedlar Thats how I would expect it to be. Everything should be what it was before adding modular.

@lsedlar: Yes, with one caveat. We want to keep the door open in the future for being able to (somehow) tag module output to be put into Everything. However, in that case the non-modular RPM should be dropped from Everything, so it wouldn't conflict/merge.

For a more concrete example: We might choose in Fedora 29 to drop the nodejs package from the traditional Everything repo in favor of only carrying module streams, with one of them (likely Node.js 10.x) being the default stream. However, assuming we continue into a world where we want to also support disabling the modular repo, we may need to be able to tell it to copy the nodejs-10 stream into the Everything repo as well (so even with Modular disabled it can satisfy dependencies in the Everything repo).

Of course, this is aspirational; if it turns out that this won't be feasible, we may just make the decision that the Modular repo is not optional in F29, thereby eliminating the need for copying it into Everything.

This is still happening with https://kojipkgs.fedoraproject.org/compose/branched/Fedora-28-20180315.n.0/compose/Everything/x86_64/os/Packages/n/ nodejs rpm which got pulled from modules

CC @sgallagh

Also
* http://dl.fedoraproject.org/pub/fedora/linux/development/28/Everything/x86_64/os/Packages/p/python2-django-1.6.11.7-1.module_1560+089ce146.noarch.rpm
* http://dl.fedoraproject.org/pub/fedora/linux/development/28/Everything/x86_64/os/Packages/r/ReviewBoard-2.5.17-17.module_d032b812.noarch.rpm

and basically all of the contents of the Modular variant have made their way in here.

There should be a list of allowed packages for Everything in https://kojipkgs.fedoraproject.org/compose/branched/Fedora-28-20180315.n.0/work/x86_64/pungi/Everything.x86_64.comps.conf but it's missing.

I'm pretty sure I broke it when allowing the empty <modules> list in variants. I have a test running to make sure though.

Commit 6bff4bd1 relates to this ticket

This should be fixed in 4.1.22-10: https://koji.fedoraproject.org/koji/buildinfo?buildID=1059055

Metadata Update from @onosek:
- Issue close_status updated to: Fixed
- Issue status updated to: Closed (was: Open)
- Issue tagged with: 4.1.23

This issue has been migrated to Fedora Forge:
https://forge.fedoraproject.org/pungi/pungi/issues/862

Please continue any further discussion there.
Metadata
None

4.1.23

None
None
None
Static archive. pagure.io was sunset in 2026; this is a read-only snapshot.
Red Hat Logo

Fedora is sponsored by Red Hat. Learn more about the relationship between Red Hat and Fedora »

© Red Hat, Inc. and others.