koji

#417 Python3 support for CLI + XMLRPC client
Merged by mikem. Opened by tkopecek.
tkopecek/koji python3-cli2-basic  into  master

Download 417.patch

So, here we come with another python2 revamp. Now I don't need to retain 2.4 compatibility, so six library was used. Client is working in many cases, though I've not tested all of them yet. If you can look to it and try your usecases, I would be glad.

  • I've all python3 branch https://pagure.io/fork/tkopecek/koji/branch/python3-cli2 from which I've branched and picked only CLI+client relevant pieces to not have such enormous PR at once. As result of removing, there is one not nice commit removing these changes. If you want to have most of tests passing, you can use that branch. Otherwise run tests selectively, as here most builder/hub/web tests can't pass yet.
  • krbV support is not there - anyway gssapi should solve all the problems - works for me with Fedora
  • urlgrabber not avaiable in python3 variant - substituted with pycurl and uglier progress bar

4 new commits added

  • fix python2 test
  • str/bytes in rpm headers
  • bytes in rpm headers
  • fix adler32 iterations

All CLI commands tested in their basic usecases. All unittests passing under python2, test_cli under python3. What is broken in both versions is nose --coverage (thus failing jenkins) which raises exception in py2 and coredumps under py3. Not sure what is the reason there, but it is not a blocker, as tests are finishing correctly without --with-coverage.
Coverage problem is caused by collision of six's reimplementation of module loader. I'll appreciate some experience with this.

@mikem Can you look to this PR, if we can test/proceed further?
Edited by tkopecek

The use of double underscore in the function __ord breaks inside the RawHeader class

>>> rh = koji.RawHeader(hdr)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "koji/__init__.py", line 596, in __init__
    self._index()
  File "koji/__init__.py", line 606, in _index
    data = [__ord(x) for x in self.header[8:12]]
NameError: global name '_RawHeader__ord' is not defined

the RawHeader class needed some more work. Now able to dump headers from an rpm on both 2 and 3.

https://github.com/mikem23/koji-playground/tree/python3-cli2-basic

bisect shows that the coverage breakage starts with this commit:

582ec29fcc812c99ec4f81bd64a78bdf7e2985f2 is the first bad commit
commit 582ec29fcc812c99ec4f81bd64a78bdf7e2985f2
Author: Tomas Kopecek <tkopecek@redhat.com>
Date:   Wed May 3 14:03:46 2017 +0200
    fix compatrequests

Ok, the coverage issue was due some test code that was manually patching the open builtin instead of letting mock.patch do it. Because of the way it was done, the patched open call persisted past the unit test it was patched for. I've added a fix on my playground branch.

https://github.com/mikem23/koji-playground/commits/python3-cli2-basic
Edited by mikem

Side note: Since we've added unit tests, I've only seen errors happen in coverage generation a few times, and each time it has been due to a significant problem in a unit test. Therefore I would view them with extreme suspicion

I don't think we should support compatrequests for py3. I'm pretty sure that code is not going to work there, and I don't see much value in porting it forward

Some more changes on my playground branch

I added a test3 target to the Makefile and moved some tests around to make it easier to run just the py3-applicable ones

The coverage core dump in py3 appears to be a bug in coverage. I've got a stripped down replicator for it here:

https://github.com/mikem23/replicate-coverage-bug

And filed a bug here:

https://bitbucket.org/ned/coveragepy/issues/583/fatal-python-error-with-threaded-unit-test

Bug appears to be fixed upstream. I've filed a Fedora bug for it

https://bugzilla.redhat.com/show_bug.cgi?id=1452339

Temporarily disabled the triggering test on my branch

rebased

1 new commit added

  • fixEncoding tests

I've included your changes. However, coverage will not work with make test3 as it needs to parse all code. That was the reason I've incorporated basic python3 compatibility with previous version.

2 new commits added

  • use coverage directly. adjust coverage config for py3 tests
  • Work around coverage bug in py3 on Fedora

1 new commit added

  • spec update

rebased

Commit 4f4639ec fixes this pull-request

Pull-Request has been merged by mikem@redhat.com
Metadata
None
No Tags
Static archive. pagure.io was sunset in 2026; this is a read-only snapshot.
Red Hat Logo

Fedora is sponsored by Red Hat. Learn more about the relationship between Red Hat and Fedora »

© Red Hat, Inc. and others.