This is a wishlist thing we've had for a while, part of the overall 'let's do CI' stuff. Right now we schedule openQA jobs via openqa_trigger.py either manually or with cron jobs. We should have a scheduler which works by listening for 'compose complete' fedmsg signals.

Right now we only get signals for the nightly composes - there are no signals for TCs or RCs. But I believe that's expected to change with the new Pungi when it's put into production.

I've been trying to write this for the last day or two, and it's one of those things that seems really simple and you can do some version of it really simply, but there are various annoying little quirks with trying to do it cleanly.

We've kind of built openQA around the idea of a release, and more specifically the release milestone compose versioning convention I've wired into fedfind and wikitcms. To avoid reinventing too many wheels ideally we'd want this scheduler and the existing one to submit jobs in basically the same way, but that doesn't really work, because the existing scheduler works off fedfind releases and there's no particularly sane way to get from a 'rawhide nightly compose completed!' signal to a fedfind release, at least as it works at present. The releng message I think we want to listen for really means 'we finished running pungi on the rawhide or branched tree and rsynced the results so it's now available from dl.fedoraproject.org'. I think at that point we can also rely on the datestamped directory in https://kojipkgs.fedoraproject.org/mash/ being available, which is one part of fedfind's concept of a 'nightly release', but it doesn't tell us anything about the other part of fedfind's concept of a 'nightly release', which is the images that are built outside of pungi - which includes the live images. When you tell openqa_trigger to "test nightly release YYYYMMDD", it's expecting both boot.iso and lives. I think in future the story is supposed to change a bit and we should actually get a fedmsg which means 'all the nightly image composes are now done and you can find metadata for them', which will be lovely, but right now we don't.

If you just look at it as 'what does it make sense to do when we get a compose complete message' the answer is 'get the URL from the fedmsg, grab the boot.iso (for right now we'd hardcode the URL suffix, when we have the nice new Pungi image list json thing we can parse that to find the image(s) we want), and schedule some tests for it'. Then the questions are:

1) What do you use for $BUILD?
2) Which tests do you schedule?
3) How do you get a fedfind.image.Image instance for download_image(), if you don't have a fedfind release?

I have slightly hacky answers to all three which should be workable as a medium-term thing until we have a better story on the releng side, I guess. We basically just reinvent the $BUILD value, and I think we should follow the fedfind versioning convention, so the tests run by the fedmsg listener would like exactly like tests run on the 'nightly release' by openqa_trigger (I don't think it'd make any sense to use different $BUILD values so we see those two things differently). So basically when we get a 'rawhide nightly complete!' message we declare that we're running tests for release Rawhide, milestone (empty string), compose YYYYMMDD; when we get a 'branched nightly complete!' message we get release from the message (it's in there), declare milestone to be Branched, and compose again YYYYMMDD. We can either just take datetime.datetime.today() for the date, or take the timestamp of the fedmsg message and turn that into a date; either way there's a question of exactly how the releng scripts decide what the date is (I think it may be tied to the timezone of the box running the script, so either way we could possibly wind up being out by a day if things work out unfortunately).

Then we take the release/milestone/compose we decided on, turn it into the $BUILD string, and also use it to get a fedfind.image.Image instance: happily I happen to have set up Image to be nicely separable from the rest of fedfind, so you can just feed it a URL and optionally feed in any of the attributes you want the instance to have, so we can give it the boot.iso URL and our version identifiers. (download_image() uses the image version info to rename the image file, so basically what I want to make sure we do here is ensure that both the fedmsg scheduler and openqa_trigger would use the same name for what is ultimately the same image, though they might get it from different places.)

As for what tests to schedule, we can basically just cheat: as we know at present we're only dealing with nightlies and we know exactly what image we're getting (a generic boot.iso), we schedule the generic_boot and universal tests, because we know that's what we want. It's kind of a hack, but hey.

Sooo....yeah. That's kinda where I'm at at present. This should all get much nicer with the new compose process, I'm hoping, because then the concept of the 'nightly compose' that releng is using will be rather closer to the concept that fedfind uses and everything will just kinda line up better (that's the short, hand-wavy version). But for now I think we can hack together something OK. I'll also look into whether we can schedule tests for the nightly Workstation live by listening out for a Koji fedmsg (don't yet know if that's going to be reasonable).