What if we broke the part of guess_encoding that builds the sorted list of Guess objects into its own function and used it here. Then loop through the encodings in order of most confident to least confident and attempt to decode the data?

I'm actually quite surprised chardet is returning an encoding that isn't even capable of decoding the data.

Regardless of how we fall back, I would prefer we didn't do this. It breaks the API of this function, which promises a unicode string object.

It might be better to raise a Pagure exception here which informs the user that the file cannot be decoded.

This doesn't seem to be an exceptional case to me. There are going to be files we can't decode and that's fine. Maybe info would be a better log level. It'd also be good if the log message included information about which file was problematic (repo, tree, and filename or something like that).

Same comment about exception vs info

I don't think 400 is appropriate here. There was nothing wrong with the request, it's just that the server was unable to satisfy it.

s/emtpy/empty

You could just do _log.debug('Trying encoding: ' + encoding) since encoding should be a string. I don't mind either way, though.

The advantage is that exception will send me an email with all the information (full url, user...) so I can then see if the situation can be improved.

It is actually such an email that started this PR.
So at least for now, I prefer keeping LOG.exception(). If it becomes too spammy, then I'll likely open a PR to change this :)

Would you prefer 500 then?

You could just do _log.debug('Trying encoding: ' + encoding) since encoding should be a string. I don't mind either way, though.

I dropped the call to str() but I was under the impression that using %s
in the log calls was the proper way. Is it wrong?

You could just do _log.debug('Trying encoding: ' + encoding) since encoding should be a string. I don't mind either way, though.

I dropped the call to str() but I was under the impression that using %s
in the log calls was the proper way. Is it wrong?

I'm actually not sure, I've never heard that before. I can't see how it would behave any different, but I'm fine leaving it as it is!

It's a tough call, since it's not really the client's fault so 4xx codes seem wrong, and there's no 5xx code that is right either. Maybe 415 is the least weird? I'm not sure.

Other than the HTTP return code this is fine with me. No HTTP code looks particularly right, so I don't mind if it's something like HTTP 500. At least that way the client doesn't think they're at fault.

Let's do 500 then :)

:thumbsup: