#25 Automated dependency testing
Closed: Fixed by carlwgeorge. Opened by avij.

It has happened way too often that packages are pushed to testing, then they spend the next 14 days in testing but nobody bothers testing them, and then finally the package owner pushes them to stable. At this point people notice that the package would need some dependant packages and chaos ensues.

To combat this, I have written a script that does nightly dependency testing for packages in epel and epel-testing, and mails me the diff from previous night's run. The script is basically just a "repoclosure -r base -r updates -r epel -r epel-testing" for each of EPEL 5,6,7. My plan is to give bad karma to those packages in epel-testing that have dependency problems. This should alert the package owner to not push the package to stable before the dep problems have been resolved. When the dependency problem has been resolved (one way or another), this will show up on the next list and I can give good karma to that package to clear the previous bad karma. Eventually giving this good/bad karma would need to be fully automated, but this is a start.


Testting this is a good idead. If you would like to pursue this, I recommend looking at https://fedoraproject.org/wiki/Taskotron which is an framework for doing these kind of automatic tests. I is already used for Fedora, maybe it can be activated for EPEL as well.

Btw. what is it you want to be handled/discussed in this ticket?

bstinson asked me to open a ticket for this task, so here it is. I suppose this ticket can be used for determining what would be the best solution for automatically detecting broken dependencies. Today's case of munin and perl-Net-CIDR showed that we'd need to find some sort of a solution to this problem.

I'm sure Taskotron would fulfill this need much more elegantly than my 15 lines of shell script. Taskotron was discussed in some earlier meeting, but it wasn't quite ready yet at that time. Perhaps it can be enabled for EPEL now, or when the orphan packages have been purged in December. There are folks who know the Taskotron situation much better than I do.

Until the switch to Taskotron happens, I'll be happy to take care of notifying packagers of broken EPEL dependencies using my own script.

I don't believe anything was ever implemented for EPEL in Taskotron. Taskotron has since been retired.

Bodhi now has an "Automated Tests" section, which is probably the right way to go about implementing this. Some Fedora updates are getting an "installability" check run here. I'm investigating why it's not being run consistently in Fedora, and what we need to do to have it run for EPEL updates too.

Metadata Update from @carlwgeorge:
- Issue close_status updated to: None

That investigation I mentioned previously involved opening this issue with the OSCI team. From my understanding, the blocker is that Testing Farm has issues providing RHEL to Fedora CI. This also affects the CI run on EPEL pull requests. About a year ago, PR CI for epel8 branches was set up to do an installability check using CentOS Stream 8, even though this will give both false positive and false negatives. Now that CI is failing because CentOS Stream 8 has reached EOL. There is also an open issue to set up the installability check for epel10 branches.

While the underlying problem of inadequate pre-publish checks remains unsolved, we have no choice but to keep being reactive to this problem post-publishing. I periodically run dnf -q repoclosure --newest --check epel --check epel-testing in various containers, inspect the output, and manually file bugs similar to Fedora's automated FTI bugs. We also have some tracking of this with @tdawson's willit tool. Eventually we should try to update Fedora's FTI automation to include EPEL branches.

We now have bodhi installability checks enabled for the leading branch of EPEL 10, which are able to utilize CentOS Stream 10. There is also an open issue to enable them for EPEL 9, but that will require public Testing Farm getting access to RHEL 9.

Since this is at least enabled for EPEL 10 going forward, I'm going to close out this issue.

Metadata Update from @carlwgeorge:
- Issue close_status updated to: Fixed
- Issue status updated to: Closed (was: Open)

This issue has been migrated to Fedora Forge:
https://forge.fedoraproject.org/epel/steering/issues/25

Please continue any further discussion there.

Metadata