I'm not as familiar with how the standard interface tests are supposed to work with containers - why do the rpms under test need to be installed prior to running the test? As this is run after the non-container tests, how can a stable environment be guaranteed if the packages do need to be installed prior to running the container tests?

This function is missing the output return that was moved to the run_docker_interface_test. Without this, no results will be able to be sent to resultsdb

Sorry for the delay on this review - the notification from pagure must have gotten lost in my inbox :(

For folks who may not be able to read the linked ticket, it states:

As a QE engineer, as a workaround until Taskotron in running in production, I would like Taskotron shim to be updated to run inside a container. Currently, the shim only supports classic mode, we need to scan for tests in upstream staging instance with the container tag, and then run the command invoing the standard test interface using the --tags container option. The prereqs needed for docker should already be enabled in Vagrant.

e.g.

sudo ANSIBLE_INVENTORY=$(test -e inventory && echo inventory || echo /usr/share/ansible/inventory) TEST_SUBJECTS=docker:docker.io/library/fedora:rawhide TEST_ARTIFACTS=$PWD/artifacts ansible-playbook --tags container tests.yml

I assume that this is meant to at least mostly follow the docs on testing against containers with the standard interface? https://fedoraproject.org/wiki/Changes/InvokingTests

This function is missing the output return that was moved to the run_docker_interface_test. Without this, no results will be able to be sent to resultsdb

Not sure what you mean, result is processed later and returned.

I'm not as familiar with how the standard interface tests are supposed to work with containers - why do the rpms under test need to be installed prior to running the test? As this is run after the non-container tests, how can a stable environment be guaranteed if the packages do need to be installed prior to running the container tests?

why do the rpms under test need to be installed prior to running the test?

I need to install that rpm to container and then I run test on that container.

As this is run after the non-container tests, how can a stable environment be guaranteed if the packages do need to be installed prior to running the container tests?

Not sure what you mean, of course stable enviroment can't be guaranteed same way as stable enviroment can't be guaranteed without this commit, you could run shim on f24 with f26 package or f24 package on rawhide enviroment.

I'm sorry I feel like I don't understand your concerns. maybe can you schedule meeting on IRC or other platform?

I assume that this is meant to at least mostly follow the docs on testing against containers with the standard interface? https://fedoraproject.org/wiki/Changes/InvokingTests

I'm sorry no, this is first time I'm seeing this document. I briefly reviewed that document and I don't see any inconsistencies between that document and patch.

I'm also not sure how should reporting to resultdb should be done in this case when there are 2 cases run (classic and container), can you please take look to runtask.yml if way how I made reporting is acceptable ?