koji

#1576 builder/kojid: revert using only 2 processors for xz compression and lower preset to 6
Closed by tkopecek. Opened by kevin.
kevin/koji xz-use-all-cpus  into  master

Download 1576.patch

In c360c791ec24b567a16553dca38b24645dd80a36 we changed xz to use only
2 cpus for compression. That commit notes that xz threads are really
memory hungry. However, preset -9 is many times as memory hungry as
preset -6. If we switch to preset 6, xz compressed artifacts will be
a little bit larger, but will compress (and decompress!) a lot larger.
This is obviously a trade off, but I think lowering the preset and
using more cpus will be a better tradeoff.

This commit moves xz to preset 6 and using all cpus the builder has.

Signed-off-by: Kevin Fenzi kevin@scrye.com

:thumbsup: It makes sense. I wonder if we should make this configurable. Nowadays we have very heterogeneous builders (tons of CPUs + relatively small memory pool vs 2 CPUs and tons of memory).
Maybe even easier is to just drop it and rely on XZ_DEFAULTS env variable?
Edited by tkopecek

Side note: the change referred to is from PR #404

I agree it would make more sense for this to be configurable.

Using XZ_DEFAULTS is an interesting idea, but I'm not sure where that would best be set. We do propagate our environ in log_output at least. Also, it might be a bit confusing if this setting was lost when running kojid manually in the foreground.

What about putting it as a parameter to kojid.conf? xz_options=XZ_DEFAULTS as default behaviour, or something else xz_options=-z6T0 ? Does it look like bad place/behaviour?

@julian8628 @mikem

+1 for xz_options=-z6T0 as default value in kojid.conf

I think it's easier to understand and more consistent than xz_options=XZ_DEFAULTS

I'd say make it a configurable in kojid.conf. Using -z6T0 seems reasonable. I'd avoid checking the environment variable for now since we do not historically configure kojid that way. It seems like a potentially very confusing thing. We can always add support for xz_options=XZ_DEFAULTS later if it seems needed.

Ok, rewritten in #2028

Pull-Request has been closed by tkopecek
Metadata
None
No Tags
Static archive. pagure.io was sunset in 2026; this is a read-only snapshot.
Red Hat Logo

Fedora is sponsored by Red Hat. Learn more about the relationship between Red Hat and Fedora »

© Red Hat, Inc. and others.