fm-orchestrator

#1359 Module builds stall when resumed when a Koji error occurs in components._finalize
Opened by mprahl. Modified

Open
Close Issue

The module build 5153 failed when the following Koji communication error occurred in components._finalize at the end of a batch.

Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/module_build_service/scheduler/consumer.py", line 256, in process_message
    further_work = handler(conf, session, msg) or []
  File "/usr/lib/python2.7/site-packages/module_build_service/scheduler/handlers/components.py", line 167, in complete
    return _finalize(config, session, msg, state=koji.BUILD_STATES["COMPLETE"])
  File "/usr/lib/python2.7/site-packages/module_build_service/scheduler/handlers/components.py", line 99, in _finalize
    session, parent, config
  File "/usr/lib/python2.7/site-packages/module_build_service/builder/base.py", line 147, in create_from_module
    components=components,
  File "/usr/lib/python2.7/site-packages/module_build_service/builder/base.py", line 124, in create
    owner=owner, module=module, config=config, **extra)
  File "/usr/lib/python2.7/site-packages/module_build_service/utils/general.py", line 282, in wrapper
    return function(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/module_build_service/builder/KojiModuleBuilder.py", line 181, in __init__
    self.koji_session = self.get_session(config)
  File "/usr/lib/python2.7/site-packages/module_build_service/utils/general.py", line 151, in inner
    return function(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/module_build_service/builder/KojiModuleBuilder.py", line 510, in get_session
    koji_session.krb_login(principal=principal, keytab=keytab, ctx=ctx, ccache=ccache)
  File "/usr/lib/python2.7/site-packages/module_build_service/builder/koji_backports.py", line 76, in krb_login
    (rep_enc, sinfo_enc, addrinfo) = self.callMethod('krbLogin', req_enc, proxyuser)
  File "/usr/lib/python2.7/site-packages/koji/__init__.py", line 2410, in callMethod
    return self._callMethod(name, args, opts)
  File "/usr/lib/python2.7/site-packages/koji/__init__.py", line 2527, in _callMethod
    return self._sendCall(handler, headers, request)
  File "/usr/lib/python2.7/site-packages/koji/__init__.py", line 2449, in _sendCall
    return self._sendOneCall(handler, headers, request)
  File "/usr/lib/python2.7/site-packages/koji/__init__.py", line 2489, in _sendOneCall
    r.raise_for_status()
  File "/usr/lib/python2.7/site-packages/requests/models.py", line 834, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
HTTPError: 502 Server Error: Proxy Error

After the failure, the user resumed the module build, but it just stalled since there was no event to trigger MBS to tag all the components.

Retries could be added to that part of the code, the scheduler could handle this issue when the build is resumed, or code could be added in the poller to find modules stuck in this situation.

This issue has been migrated to Fedora Forge:
https://forge.fedoraproject.org/koji/mbs/issues/1359

Please continue any further discussion there.
Metadata
None
None
None
None
None
Static archive. pagure.io was sunset in 2026; this is a read-only snapshot.
Red Hat Logo

Fedora is sponsored by Red Hat. Learn more about the relationship between Red Hat and Fedora »

© Red Hat, Inc. and others.