In devel@lists.fedoraproject.org "Tagging large packages (texlive) takes a very long time", Kevin Fenzi said on 11/17/2017:

The bottleneck is the part where it's finished all the builds and needs
to check the noarch packages built on each arch to make sure they are
the same.

If so, then would a sha256sum digest of the .noarch rpms be a speed improvement? The checksum could be computed on each builder as each rpm is built, and accumulated into a file. Then the files from all builders could be sorted and analyzed by diff or comm, etc.

[Where is the code that supervises this part of the process?]

@jreiser Well, the noarch RPMs are never the same exactly. The koji checks are for the actual files inside it, skipping some, because the RPM contains timestamps and inode numbers.
The actual code for that function is at https://pagure.io/koji/blob/master/f/hub/kojihub.py#_8417

@puiterwijk

Well, the noarch RPMs are never the same exactly.

OK. If an RPM is a sequence of pieces, then the digest would be a group of lines, one line for each piece. The kinds of pieces might be 1) random [and usually short or omitted]; 2) same except for $ARCH, $DATE, $FILE_SIZE; 3) same [and possibly long]. Checksumming would be applied to each non-random piece (unified by substituting $ARCH, etc., if appropriate). The digest would still be produced as each [sub-]RPM is built. After finishing, then each builder could check its overall digest file (of multiple sub-RPMs) against the digest file for any builder that finished previously. Digest files still would be fewer and shorter; and processing would be parallelized, or at least pipelined and mostly hidden (by unfinished builders) or overlapped (among multiple builders that finish near to each other in time).

Thank you for the code pointer.

I wonder that rpmdiff is slow in this case. It is not computing hash values, just comparing rpm header information. I've created simple PR #774 which will run rpmdiff inside python interpreter instead of spawning process which save some time. I'll try to do some profiling on texlive builds.

Ok, seen how texlive looks nowadays. So problem is not speed of check itself, but amount of cross-comparisons (cca 5867 noarch rpms * 7 archs). I'll try to put some hashes to one file per architecture instead of opening 40k files.

Added commit for storing hash of rpmdiff values for quick comparison. See/comment #774

Commit 7b559a0f fixes this issue

Commit fb0ba44a relates to this ticket

This issue has been migrated to Fedora Forge:
https://forge.fedoraproject.org/koji/koji/issues/715

Please continue any further discussion there.