Linked issue #2391

The containers make it easier to run the Koji tests on macos :thumbsup:

I've tested this on my Fedora system and the py3 tests work. However the py2 tests fail with the centos:6 container. Here's the output: https://gist.github.com/ktdreyer/bd6f591ee923550fbf2c38a181f3a86f

I have some changes at https://pagure.io/fork/ktdreyer/koji/c/3d72decf2c03597a9a207e0cf575dbe5b087a558?branch=tox-tests-more-docs and I found out in https://pagure.io/pagure/issue/4929 that you have to explicitly enable "Pull Requests" on your fork so I can send a pull request: https://pagure.io/fork/zmc/koji/settings#projectoptions-tab

@ktdreyer would you mind testing on py2 again with the commit that I just pushed?

Thanks for adding more docs! Adding those changes via a PR to a repo/branch which currently has a PR filed on another repo/branch feels like new territory to me - have you done that before? I could also just cherry-pick your commit.

Thanks @zmc. I tested with the latest requests-mock pinning for py2, and that removes the test error. Now we just have this test failure: https://gist.github.com/ktdreyer/bd6f591ee923550fbf2c38a181f3a86f . I'm still trying to work out what's wrong there.

Re: the pull request in Pagure, sure, you can just cherry-pick my commit there. Whatever's easiest.

@ktdreyer I've seen that failure too, and I don't see it in the Jenkins output. Confusing. It's worth noting that the CI invokes the test in a way I don't see represented in the repo itself - meaning it's not using make test and contains its own invocations of nosetests directly.

hmm, I wonder what we're doing differently than Jenkins is doing

Cool! I finally got to this PR. I've used tox locally, but never got it to state which would be PRable.

Jenkins setup is little bit hackish as it that instance is not so much supported. So, I'm constantly fighting with what can be installed via rpms what must be updated via pip, etc. See current config:
https://docs.pagure.org/koji/configuring_jenkins/#configuration

Maybe different version of libcomps is used in that env?

We expect that py2 tests will pass on centos6 if epel is enabled and everything is installed from rpms. So, using virtualenv could be the test, but makes sense only fo cli/lib. builder tests should be run in pure centos/epel env.

Similar situation is for f(32/rawhide). Tests should pass on rpm-based installation. Does it make sense to create some "empty" env, which will use local resources without setting up virtualenv? Not sure what is the best practice for tox here.

Anyway, running all tests on centos6 is no more supported (only builder + lib should be run there).

@tkopecek Regarding the EL6 test failure, unless I'm mistaken there's no libcomps available, so the tests would use yumcomps (and the failing test is test_import_comps_sample_yumcomps). The the docker image I'm using has yum-3.2.29-81.el6.centos.0.1.noarch, which is the latest. I'm unable to see what yum version the Jenkins builds are using, so I'm not sure how to diagnose the failure.

I'm readying a commit that will cause the EL6 tests to avoid installing requirements.txt and thus require RPMs of all dependencies to be installed. It'll still install test-requirements.txt, unless we think that's a bad idea. I'd like to figure out the test failure before adding it to the PR, though.

It's now possible to test with RPM-only dependencies via the py2-rpmdeps testenv.

I'd love some feedback on how to figure out what's causing the comps test to fail.

Difference is that newer tox set PYTHONHASHSEED to random value (same behaviour as python 3.3+). Yum is affected by this (which I see as a yum bug) but this is probably fixed in newer versions as I don't see that behaviour elsewhere. As we're slowly leaving py2 I would suggest to set PYTHONHASHSEED=0 for py2 tests which works for me.

@tkopecek Looks like you were right! Both py2 and py2-rpmdeps pass now.

@tkopecek any more feedback?

It (make test) fails for me in f32 now. I've not time to look at it properly but it looks like a problem with rpm-py-installer py2/py3 versions.

writing manifest file '/tmp/pip-install-IaVpvr/rpm-py-installer/pip-egg-info/rpm_py_installer.egg-info/SOURCES.txt'
    [INFO] Installing...
    [INFO] Created working directory '/tmp/tmpKSGi8D-rpm-py-installer'
    Traceback (most recent call last):
      File "install.py", line 2202, in <module>
        main()
      File "install.py", line 2197, in main
        app.run()
      File "install.py", line 40, in run
        self.rpm_py.download_and_install()
      File "install.py", line 157, in download_and_install
        self.installer.install_from_rpm_py_package()
      File "install.py", line 948, in install_from_rpm_py_package
        self._download_and_extract_rpm_py_package()
      File "install.py", line 1098, in _download_and_extract_rpm_py_package
        self.rpm.download_and_extract(package_name)
      File "install.py", line 1684, in download_and_extract
        self.download(package_name)
      File "install.py", line 1764, in download
        Cmd.sh_e(cmd, stdout=subprocess.PIPE)
      File "install.py", line 1985, in sh_e
        raise exc
    UnicodeEncodeError: 'ascii' codec can't encode character u'\xed' in position 3: ordinal not in range(128)

Odd, make test succeeds for me inside the F32 container I made to run these, as does pip install rpm-py-installer inside a py2 virtualenv.

Would you mind putting the entire output including the invocation into a paste somewhere for me to look at?

Same container (from devtools/containers/fedora): https://pastebin.com/TQiMRBR9

Thanks, @tkopecek - I see it's an issue with getting python-rpm available for python2 on F32. I've just added rpm-devel to the container, so if you rebuild it and then try the same thing again, the bindings should be compiled. Your same workflow passed both py2 and py3 for me just now, with this change.

Coverage and flake8 excludes needs to be updated - they now parse also .tox directory.

Fixed

Commit 9557d1b2 fixes this pull-request