#306 Python 3 compatibility for strings
Merged by abompard. Opened by abompard.
abompard/fedora-hubs py3-compat  into  develop

Download 306.patch

There were still a lot of mixup between bytestrings and unicode strings. This changeset uses the from __future__ import unicode_literals construct which makes Python 2 interpret all strings in the code as unicode, as Python 3 does. This helps a lot reducing the differences between Python 2 and 3. It also uses six to check string types.

Unit tests pass with both Python 2 and Python 3.

Why was this changed? I don't object to it (especially since the docs indicate this behaves identically), I'm just curious.

Good question! It made the unit test flaky, because since dicts arent's ordered, the URL was rendered sometimes as ?s=s&d=retro, sometimes as ?d=retro&s=s. Since the test was checking the final URL, it failed sometimes. Python3 just made it more visible.

I think it would be better to use http://flask.pocoo.org/docs/0.12/api/#flask.Request.get_json here. I would expect that to automatically decode the request data using the client-provided content encoding header and parse it nicely.

I think this way would cause problems if the data isn't utf-8 encoded because to_str assumes everything is utf-8.

I agree, I'll fix it. to_str makes sense in the parts where Flask is optional, but if we're dealing with Request instances, get_json() is definitely better.

My only concern here is this might get used on something that wasn't utf-8 encoded. Admittedly, I think that's an edge case because I assume Flask automatically decodes request data to unicode strings (the documentation just uses the word "strings" and I haven't yet checked what actually happens) and with the from __future__ import unicode_literals imports there's not a lot of other avenues for encoded text to sneak in.

It might be nice to add a encoding='utf-8' kwarg and use whatever is provided in data.decode('utf-8').

As an aside, I'm very surprised Werkzeug responses don't automatically decode the responses. If we're only doing decoding those in tests it's probably fine as it is, but you could add an additional function specifically for them that inspects the charset the server sent and decode the data using that.

This looks great and it's a great to do this as early as possible! Just a few comments (although one got a little long and rambly, sorry!), but no real issues.

It's a little tough to navigate all the mixins in Werkzeug, but I think http://werkzeug.pocoo.org/docs/0.11/wrappers/#werkzeug.wrappers.BaseResponse.get_data with as_text=True would solve everything.

I too agree with the usage of get_data(), incoming update! :-)

Done, thanks for your attentive review.

1 new commit added

  • Use request.get_json() and .get_data() where approriate

Perfect, merge away!

Pull-Request has been merged by abompard

Metadata