fedora-qa / fedora_openqa

#20 Come up with some alternative to `urlgrabber` for ISO downloads
Closed: Fixed Opened by adamwill.

Closed: Fixed
Reopen Issue

As discussed in D547, we probably should stop using urlgrabber since it's had that bug ever since we wrote the darn thing, and anyway it's an external dependency just to do one pretty simple thing.

jsedlak suggested urllib.urlretrieve but I have a bit of a problem with that: "Warning When opening HTTPS URLs, it does not attempt to validate the server certificate. Use at your own risk!" - from https://docs.python.org/2/library/urllib.html . So how about urllib2.urlopen() instead? fedfind uses it already (to check whether a given URL exists).

For the record, the URL we get from fedfind will always be HTTP. It can't ever be FTP. On some level it's theoretically possible for it to be rsync, but as the code's written that would never actually happen.

of course if I ever get my openQA ISO downloading magic unicorn merged we could just ditch this entirely, I guess :P

This ticket had assigned some Differential requests:
D597

I don't think that urllib2.urlopen() is a good idea for larger files. I don't know any way how to pipe it into a file directly and using .read() and write() would load whole file into RAM. That's the reason I'm more inclined to use urllib.urlretrieve. But I'm open for discussion :-).

C'mon, don't you go to the University of StackOverflow too? ;) You use a buffer:

https://stackoverflow.com/questions/22676/how-do-i-download-a-file-over-http-using-python

This issue has been migrated to Fedora Forge:
https://forge.fedoraproject.org/quality/fedora_openqa/issues/20

Please continue any further discussion there.
Metadata
None
None
None
None
Low
Static archive. pagure.io was sunset in 2026; this is a read-only snapshot.
Red Hat Logo

Fedora is sponsored by Red Hat. Learn more about the relationship between Red Hat and Fedora »

© Red Hat, Inc. and others.