#403 fixEncoding for changelogs
Merged by mikem. Opened by tkopecek.
tkopecek/koji issue349  into  master

Download 403.patch

Some real-world changelogs contains non-printable characters or invalid
unicode ones. xmlrpc fails on such strings, so we sanitize changelog
strings before passing it to client.

Related: https://pagure.io/koji/issue/349

typo here:

result['text'] = koji.fixEncoding(result['author'])

If I correct author to text the error returns. Using fixEncoding isn't wrong, but it's not going to remove control characters.

With the correct field, fixEncoding doesn't actually change the result of the handler (before encoding).

Hmm, typo was the reason, why it helped in my case...

rebased

except for \r\n\t, ascii characters below \x20 are not legal in xmlrpc. Luckily these never appear in multibyte utf8 representations. I think we can just filter them out.

ctrl = [chr(i) for i in range(32)]
filter = ''.join([c for c in ctrl if c not in '\r\n\t'])
good = bad.translate(None, filter)

1 new commit added

  • less restrictive filter

If we're going to apply this that broadly, we really need to check all the uses.

I'm really not sure that silently stripping these characters in all situations is the right thing. We're doing this to work around an xmlrpc limitation, but fixEncoding is used for other purposes.

So, what about setting remove_nonpritable to False as default and for now use it only in getChangelogEntries?

1 new commit added

  • don't remove nonprintable characters by default

looks like remove_nonprintable is not propagated through the recursion (or the fallback value for that matter)

1 new commit added

  • propagate parametere recursively

Added also fallback encoding to recursion.

4 new commits added

  • propagate parameters recursively
  • don't remove nonprintable characters by default
  • less restrictive filter
  • remove non-printable characters in fixEncoding

subscribe

This patch is working fine in our production instance and fixes #349.

You need to also add it to the getRPMHeaders call end.

What is #349? its marked as private apparently and I can not access it

rebased

Commit a0941767 fixes this pull-request

Pull-Request has been merged by mikem@redhat.com

Metadata