#8910 Nightly failure (updates-testing) in test_ipahealthcheck.py::TestIpaHealthCheck::test_source_ipahealthcheck_ipa_host_check_ipahostkeytab
Closed: worksforme by frenaud. Opened by frenaud.

The nightly test test_ipahealthcheck.py::TestIpaHealthCheck::test_source_ipahealthcheck_ipa_host_check_ipahostkeytab is failing with ipa-healthcheck 0.9-2, see PR #995 with the following logs and report:

self = <ipatests.test_integration.test_ipahealthcheck.TestIpaHealthCheck object at 0x7fce0fb60610>
restart_service = <function restart_service.<locals>._stop_service at 0x7fce10286040>
    def test_source_ipahealthcheck_ipa_host_check_ipahostkeytab(
        self, restart_service
    ):
        """
        Testcase checks behaviour of check IPAHostKeytab in source
        ipahealthcheck.ipa.host when dirsrv service is stopped and
        running on IPA master
        """
        msgs = (
            (
                "Failed to obtain host TGT: Major (851968): "
                "Unspecified GSS failure.  "
                "Minor code may provide more information, "
                "Minor (2529638972): Generic error (see e-text)"
            ),
            (
                "Failed to obtain host TGT: Major (458752): "
                "No credentials were supplied, or the credentials "
                "were unavailable or inaccessible, "
                "Minor (2529638972): Generic error (see e-text)"
            ),
        )
        restart_service(self.master, "dirsrv")
        dirsrv_ipactl_status = 'Directory Service: STOPPED'
        result = self.master.run_command(
            ["ipactl", "status"],
            raiseonerr=False)
>       returncode, data = run_healthcheck(
            self.master,
            "ipahealthcheck.ipa.host",
            "IPAHostKeytab",
        )
test_integration/test_ipahealthcheck.py:522: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
test_integration/test_ipahealthcheck.py:189: in run_healthcheck
    data = json.loads(result.stdout_text)
/usr/lib64/python3.9/json/__init__.py:346: in loads
    return _default_decoder.decode(s)
/usr/lib64/python3.9/json/decoder.py:337: in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
self = <json.decoder.JSONDecoder object at 0x7fce129ab790>
s = "Source 'ipahealthcheck.ipa.host' is missing one or more requirements 'krb5kdc, dirsrv'\n"
idx = 0
    def raw_decode(self, s, idx=0):
        """Decode a JSON document from ``s`` (a ``str`` beginning with
        a JSON document) and return a 2-tuple of the Python
        representation and the index in ``s`` where the document ended.
        This can be used to decode a JSON document from a string that may
        have extraneous data at the end.
        """
        try:
            obj, end = self.scan_once(s, idx)
        except StopIteration as err:
>           raise JSONDecodeError("Expecting value", s, err.value) from None
E           json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
/usr/lib64/python3.9/json/decoder.py:355: JSONDecodeError
 -------------------------------Captured log call-------------------------------- 
INFO     ipatests.pytest_ipa.integration.host.Host.master.IPAOpenSSHTransport:transport.py:391 RUN ['systemctl', 'stop', 'dirsrv@IPA-TEST.service']
DEBUG    ipatests.pytest_ipa.integration.host.Host.master.cmd217:transport.py:513 RUN ['systemctl', 'stop', 'dirsrv@IPA-TEST.service']
DEBUG    ipatests.pytest_ipa.integration.host.Host.master.cmd217:transport.py:217 Exit code: 0
INFO     ipatests.pytest_ipa.integration.host.Host.master.IPAOpenSSHTransport:transport.py:391 RUN ['ipactl', 'status']
DEBUG    ipatests.pytest_ipa.integration.host.Host.master.cmd218:transport.py:513 RUN ['ipactl', 'status']
DEBUG    ipatests.pytest_ipa.integration.host.Host.master.cmd218:transport.py:557 Directory Service: STOPPED
DEBUG    ipatests.pytest_ipa.integration.host.Host.master.cmd218:transport.py:557 1 service(s) are not running
DEBUG    ipatests.pytest_ipa.integration.host.Host.master.cmd218:transport.py:557 krb5kdc Service: RUNNING
DEBUG    ipatests.pytest_ipa.integration.host.Host.master.cmd218:transport.py:557 kadmin Service: RUNNING
DEBUG    ipatests.pytest_ipa.integration.host.Host.master.cmd218:transport.py:557 named Service: RUNNING
DEBUG    ipatests.pytest_ipa.integration.host.Host.master.cmd218:transport.py:557 httpd Service: RUNNING
DEBUG    ipatests.pytest_ipa.integration.host.Host.master.cmd218:transport.py:557 ipa-custodia Service: RUNNING
DEBUG    ipatests.pytest_ipa.integration.host.Host.master.cmd218:transport.py:557 pki-tomcatd Service: RUNNING
DEBUG    ipatests.pytest_ipa.integration.host.Host.master.cmd218:transport.py:557 ipa-otpd Service: RUNNING
DEBUG    ipatests.pytest_ipa.integration.host.Host.master.cmd218:transport.py:557 ipa-dnskeysyncd Service: RUNNING
DEBUG    ipatests.pytest_ipa.integration.host.Host.master.cmd218:transport.py:217 Exit code: 3
INFO     ipatests.pytest_ipa.integration.host.Host.master.IPAOpenSSHTransport:transport.py:391 RUN ['ipa-healthcheck', '--source', 'ipahealthcheck.ipa.host', '--check', 'IPAHostKeytab', '--output-type', 'json']
DEBUG    ipatests.pytest_ipa.integration.host.Host.master.cmd219:transport.py:513 RUN ['ipa-healthcheck', '--source', 'ipahealthcheck.ipa.host', '--check', 'IPAHostKeytab', '--output-type', 'json']
DEBUG    ipatests.pytest_ipa.integration.host.Host.master.cmd219:transport.py:557 Source 'ipahealthcheck.ipa.host' is missing one or more requirements 'krb5kdc, dirsrv'
DEBUG    ipatests.pytest_ipa.integration.host.Host.master.cmd219:transport.py:217 Exit code: 1

The test succeeds with freeipa-healthcheck-0.8-5 but fails with freeipa-healthcheck-0.9-2 because 0.9.2 introduced a dependency on dirsrv running (see commit 501554e). As a consequence, freeipa-healthcheck now returns an error message (not json output format) when the dependency is not met:

# ipa-healthcheck --output-type json --source ipahealthcheck.ipa.host --check IPAHostKeytab
Source 'ipahealthcheck.ipa.host' is missing one or more requirements 'krb5kdc, dirsrv'

while it previously executed the test:

# ipa-healthcheck --output-type json --source ipahealthcheck.ipa.host --check IPAHostKeytab
[
  {
    "source": "ipahealthcheck.ipa.host",
    "check": "IPAHostKeytab",
    "result": "ERROR",
    "uuid": "9be8748f-2164-4a44-848c-cee14c53896e",
    "when": "20210703121152Z",
    "duration": "0.020035",
    "kw": {
      "msg": "Failed to obtain host TGT: Major (458752): No credentials were supplied, or the credentials were unavailable or inaccessible, Minor (2529638972): Generic error (see e-text)"
    }
  }
]

The test should not try to parse the json output in this case but rather expect the error message. freeipa-healthcheck 0.9.2 is now available in the "updates" repo of all our supported platforms (f33, f34, rawhide), meaning that we should update the images used for testing + fix the test.


Seems I was late checking this issue...
@slev already fixed the assumption in https://pagure.io/freeipa/c/3597ab9bac6afbb5f055b2d62cb60f3e2850b31e?branch=master
Closing as works for me

Metadata Update from @frenaud:
- Issue close_status updated to: worksforme
- Issue status updated to: Closed (was: Open)

Metadata