Issue #58: sealert/setroubleshootd is broken on the latest Rawhide due to the `formatter` module being removed from Python 3.10 onwards - setroubleshoot

setroubleshoot

#58 sealert/setroubleshootd is broken on the latest Rawhide due to the `formatter` module being removed from Python 3.10 onwards

Closed: Fixed Jul 27, 2021 by plautrba. Opened Jul 14, 2021 by mrc0mmand.

Hey!

I was trying to debug some AVC denials on the latest Rawhide and noticed that neither sealert nor setroubleshootd work. After a quick search it looks like the formatter module (which is imported by html_util.py) is no longer available since Python 3.10 - see the deprecation notice in Python 3.9:

Python 3.9.5 (default, May 14 2021, 00:00:00) 
[GCC 11.1.1 20210428 (Red Hat 11.1.1-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import formatter
<stdin>:1: DeprecationWarning: the formatter module is deprecated

Python 3.10.0b4 (default, Jul 11 2021, 00:00:00) [GCC 11.1.1 20210623 (Red Hat 11.1.1-6)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import formatter
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'formatter'

sealert/setroubleshootd errors:

# sealert -S
Unable to init server: Could not connect: Connection refused
Unable to init server: Could not connect: Connection refused
Opps, sealert hit an error!
Traceback (most recent call last):
  File "/usr/bin/sealert", line 691, in <module>
    run_app(username)
  File "/usr/bin/sealert", line 99, in run_app
    from setroubleshoot.serverconnection import ServerConnectionHandler
  File "/usr/lib/python3.10/site-packages/setroubleshoot/serverconnection.py", line 22, in <module>
    from setroubleshoot.rpc_interfaces import SETroubleshootServerInterface
  File "/usr/lib/python3.10/site-packages/setroubleshoot/rpc_interfaces.py", line 21, in <module>
    from setroubleshoot.signature import *
  File "/usr/lib/python3.10/site-packages/setroubleshoot/signature.py", line 72, in <module>
    from setroubleshoot.html_util import *
  File "/usr/lib/python3.10/site-packages/setroubleshoot/html_util.py", line 39, in <module>
    import formatter as Formatter
ModuleNotFoundError: No module named 'formatter'

Jul 14 12:08:59 fedora setroubleshootd[101356]: Traceback (most recent call last):
Jul 14 12:08:59 fedora setroubleshootd[101356]:   File "/usr/sbin/setroubleshootd", line 105, in <module>
Jul 14 12:08:59 fedora setroubleshootd[101356]:     from setroubleshoot.server import RunFaultServer
Jul 14 12:08:59 fedora setroubleshootd[101356]:   File "/usr/lib/python3.10/site-packages/setroubleshoot/server.py", line 70, in <module>
Jul 14 12:08:59 fedora setroubleshootd[101356]:     from setroubleshoot.analyze import (PluginReportReceiver,
Jul 14 12:08:59 fedora setroubleshootd[101356]:   File "/usr/lib/python3.10/site-packages/setroubleshoot/analyze.py", line 43, in <module>
Jul 14 12:08:59 fedora setroubleshootd[101356]:     from setroubleshoot.avc_audit import *
Jul 14 12:08:59 fedora setroubleshootd[101356]:   File "/usr/lib/python3.10/site-packages/setroubleshoot/avc_audit.py", line 43, in <module>
Jul 14 12:08:59 fedora setroubleshootd[101356]:     from setroubleshoot.audit_data import *
Jul 14 12:08:59 fedora setroubleshootd[101356]:   File "/usr/lib/python3.10/site-packages/setroubleshoot/audit_data.py", line 48, in <module>
Jul 14 12:08:59 fedora setroubleshootd[101356]:     from setroubleshoot.html_util import *
Jul 14 12:08:59 fedora setroubleshootd[101356]:   File "/usr/lib/python3.10/site-packages/setroubleshoot/html_util.py", line 39, in <module>
Jul 14 12:08:59 fedora setroubleshootd[101356]:     import formatter as Formatter
Jul 14 12:08:59 fedora setroubleshootd[101356]: ModuleNotFoundError: No module named 'formatter'
Jul 14 12:08:59 fedora systemd[1]: dbus-:1.3-org.fedoraproject.Setroubleshootd@125.service: Main process exited, code=exited, status=1/FAILURE
Jul 14 12:08:59 fedora systemd[1]: dbus-:1.3-org.fedoraproject.Setroubleshootd@125.service: Failed with result 'exit-code'.
Jul 14 12:08:59 fedora audit[1]: SERVICE_STOP pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=dbus-:1.3-org.fedoraproject.Setroubleshootd@125 comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? a>
Jul 14 12:08:59 fedora systemd[1]: Started dbus-:1.3-org.fedoraproject.Setroubleshootd@126.service.
Jul 14 12:08:59 fedora audit[1]: SERVICE_START pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=dbus-:1.3-org.fedoraproject.Setroubleshootd@126 comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? >
Jul 14 12:08:59 fedora setroubleshootd[101357]: Traceback (most recent call last):
Jul 14 12:08:59 fedora setroubleshootd[101357]:   File "/usr/sbin/setroubleshootd", line 105, in <module>
Jul 14 12:08:59 fedora setroubleshootd[101357]:     from setroubleshoot.server import RunFaultServer
Jul 14 12:08:59 fedora setroubleshootd[101357]:   File "/usr/lib/python3.10/site-packages/setroubleshoot/server.py", line 70, in <module>
Jul 14 12:08:59 fedora setroubleshootd[101357]:     from setroubleshoot.analyze import (PluginReportReceiver,
Jul 14 12:08:59 fedora setroubleshootd[101357]:   File "/usr/lib/python3.10/site-packages/setroubleshoot/analyze.py", line 43, in <module>
Jul 14 12:08:59 fedora setroubleshootd[101357]:     from setroubleshoot.avc_audit import *
Jul 14 12:08:59 fedora setroubleshootd[101357]:   File "/usr/lib/python3.10/site-packages/setroubleshoot/avc_audit.py", line 43, in <module>
Jul 14 12:08:59 fedora setroubleshootd[101357]:     from setroubleshoot.audit_data import *
Jul 14 12:08:59 fedora setroubleshootd[101357]:   File "/usr/lib/python3.10/site-packages/setroubleshoot/audit_data.py", line 48, in <module>
Jul 14 12:08:59 fedora setroubleshootd[101357]:     from setroubleshoot.html_util import *
Jul 14 12:08:59 fedora setroubleshootd[101357]:   File "/usr/lib/python3.10/site-packages/setroubleshoot/html_util.py", line 39, in <module>
Jul 14 12:08:59 fedora setroubleshootd[101357]:     import formatter as Formatter
Jul 14 12:08:59 fedora setroubleshootd[101357]: ModuleNotFoundError: No module named 'formatter'

Metadata Update from @plautrba:
- Issue assigned to plautrba

Jul 16, 2021

adamwill commented Jul 26, 2021

Downstream bug:

https://bugzilla.redhat.com/show_bug.cgi?id=1972391

upstream's recommendation is for users to copy the classes they use(!), but it might be better to see if we can use something else for what these formatters are actually used for? Ultimately it seems like both formatters are only used in the html_to_text function, and that is only used in one place, in browser.BrowserApplet.details(). server.py imports it, but does not actually use it (presumably it did at one point, but that usage got taken out?)