In the #koji channel in Freenode, @lrossett saw the following error from koji-web (https://koji-mbbox.apps.ocp.stg.ci.centos.org/koji/):

requests.exceptions.SSLError: HTTPSConnectionPool(host='koji-mbbox.apps.ocp.stg.ci.centos.org', port=443): Max retries exceeded with url: /kojihub (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1076)')))

Leonardo was able to use python-requests directly, like so:

from requests import Request, Session
s = Session()
headers = {
  'Content-Type': 'text/xml'
}
data = '''<?xml version="1.0" encoding="utf-8"?>
<methodCall>
  <methodName>sslLogin</methodName>
  <params></params>
</methodCall>'''
url1 = 'https://koji-hub:8443/kojihub/ssllogin'
url2 = 'https://koji-mbbox.apps.ocp.stg.ci.centos.org/kojihub/ssllogin'
res = s.post(url2, data=data, headers=headers, cert='/etc/webcert/client.pem', verify='/etc/cacert/cert')
print(res.status_code)
print(res.text)

Here is Leonardo's koji-web config:

[web]
SiteName = MBOX Koji
KojiHubURL = https://koji-hub:8443/kojihub
KojiFilesURL = https://koji-mbbox.apps.ocp.stg.ci.centos.org/pkgs
Secret = FooBar
LibPath = /usr/share/koji-web/lib
LoginDisabled = True
KojiHubCA = /etc/cacert/cert
InsecureNoSSLVerify = True

A couple of things occurred to me:

1) koji-web ought to use the KojiHubCA parameter for 100% of its connections to the hub. However, when I read www/kojiweb/index.py, the only code that uses KojiHubCA is in _sslLogin(). If I'm reading this right, _getServer() sets up a ClientSession without setting serverca, so koji-web will not trust KojiHubCA for all https connections (only the one time where we call sslLogin). Probably need to set up a small reproduction case here: Set up koji-hub and koji-web on the same system, don't trust the CA system-wide, and point KojiHubCA at the CA path (/etc/cacert/cert in Leonardo's case.)

2) It would be really nice if koji-web could print the entire index.py traceback instead of just showing the requests.exceptions.SSLError. That would provide more information for my theory above. If there is some mod_wsgi setting or something for this, it would be good to document this.

3) It would be cool to get more information about the exact cert chain that caused SSLError. Not sure if python-requests exposes this to us somehow.