Related: https://pagure.io/koji/issue/464
xmlrpclib.dumps is also used in kojihub.py. Not sure if we want to move patched Marshaller to library and use it in both places.
xmlrpclib.dumps
kojihub.py
I am somewhat worried about requiring clients to support yet another xmlrpclib extension. I guess they are going to get an error either way in this case, though in the former case case they'll get a valid Fault, in this case they'll likely get an unhelpful parsing error.
So perhaps my worries are unfounded
Why they should be an error? Due to unsupported extension, or do you think, that koji will fail in some other place due to big file?
For extension I would say, that at least python should be safe. python2.2+ xmlrpclib/xmlrpc.client can decode i8. They just don't use it in Marshaller.
Otherwise we can serve big integers as text, but it would be cumbersome as it will need some additional type conversion in client.
1 new commit added
If we go this way, it will be tempting to retire the encode_int function
2 new commits added
Yep, I forgot about that at all. I've added commit + note to API usage docs.
Too hasty, I'll need to make more fixes for that.
rebased
On further though, I think it would be safer to leave the encode_int bits in place for now (or at least some of them) and have a longer discussion about the impact of that. I'm thinking specifically of non-python clients that may be using some of the calls when encode_int is used.
Such a client may already be relying on this workaround. If so the should be prepared for either an int or string value. That much is fine, however, the non-python clients may not support i8. In such a case, we would be reverting a workaround for them. They would go from getting a string value they can parse to an i8 value that gives them an error.
Affected functions include: get_rpm, readTaggedRPMS, readTaggedArchives, maven_tag_archives, list_rpms, list_archives, checkUpload, and handle_upload, many of these are likely to be used by non-koji clients.
get_rpm
readTaggedRPMS
readTaggedArchives
maven_tag_archives
list_rpms
list_archives
checkUpload
handle_upload
In particular, the changes in kojikamid won't work, because kojikamid uses xmlrpclib.ServerProxy to talk with kojivmd (which uses SimpleXMLRPCServer).
I'm not sure if the dump_datetime override makes sense outside of the hub. That is a workaround for psycopg2 vs pygresql behavior
Also, not sure if we should be monkey patching xmlrpclib in our main library
The more I look at this, the less confidence I have. It's too deep to properly evaluate this close to freeze. I think we have to leave it out of 1.13 and hopefully sort it out soon after
I did a little playing around here. Still not sure about it though
https://github.com/mikem23/koji-playground/commits/issue464
If it is used for some calls, is there reason to still have encode_int for others?
encode_int
I've been fiddling with this and this is kind of where I'm at
Not done yet, but appears to work at least. Should probably take the opportunity to write unit tests for xmlrpcplus.
Closing this in favor of PR #571
Pull-Request has been closed by mikem
Related: https://pagure.io/koji/issue/464