Issue #253: koji tasks sometimes fail to start - pungi

pungi

#253 koji tasks sometimes fail to start

Closed: Fixed Opened Apr 5, 2016 by ausil.

Sometimes koji tasks fail to start like the armhfp Everything repo in https://kojipkgs.fedoraproject.org/compose/rawhide/Fedora-Rawhide-20160405.n.2/ we should never fail to create a task.


2016-04-05 18:59:07 [INFO    ] [FAIL] Buildinstall (variant Everything, arch armhfp) failed, but going on anyway.
2016-04-05 18:59:07 [INFO    ] super(type, obj): obj must be an instance or subtype of type


2016-04-05 18:59:07 [DEBUG   ] Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/pungi/util.py", line 466, in failable
    yield
  File "/usr/lib/python2.7/site-packages/pungi/phases/buildinstall.py", line 386, in process
    self.worker(compose, arch, variant, cmd, num)
  File "/usr/lib/python2.7/site-packages/pungi/phases/buildinstall.py", line 420, in worker
    koji_wrapper = KojiWrapper(compose.conf["koji_profile"])
  File "/usr/lib/python2.7/site-packages/pungi/wrappers/kojiwrapper.py", line 34, in __init__
    self.koji_module = koji.get_profile_module(profile)
  File "/usr/lib/python2.7/site-packages/koji/__init__.py", line 1602, in get_profile_module
    koji_module_loc[2])
  File "/usr/lib/python2.7/site-packages/koji/__init__.py", line 202, in 
    'PROBLEM',
  File "/usr/lib/python2.7/site-packages/koji/__init__.py", line 103, in __init__
    super(Enum,self).__init__([(value,n) for n,value in enumerate(self._order)])
TypeError: super(type, obj): obj must be an instance or subtype of type

lsedlar commented Apr 6, 2016

The error is raised inside koji client module. I think the problem is that the get_profile_module function is not thread safe. The buildinstall spawns a bunch of threads, each trying to get the profile.

This test script can replicate the problem most of the times without even connecting to Koji.

def stress():
    koji.get_profile_module('koji')
threads = [threading.Thread(target=stress) for _ in xrange(100)]
for t in threads:
    t.start()
for t in threads:
    t.join()j

ausil commented Apr 6, 2016

filed https://pagure.io/koji/issue/58

lsedlar commented Apr 6, 2016