#3069 basic kiwi support
Merged by tkopecek. Opened by tkopecek.
tkopecek/koji kiwi  into  master

Download 3069.patch

%SUBJ% - work in progress

We should probably capture all the logs, like we do for other image build tasks...

Is it possible to make it so that Koji would tail the log files while the build is running? That way it's possible to see the detailed execution while it's happening...

Is it possible to make it so that Koji would tail the log files while the build is running? That way it's possible to see the detailed execution while it's happening...

yes, it is being uploaded as for other tasks (broot does that automatically on background)

1 new commit added

  • json results

:thumbsup:

Metadata Update from @julian8628:
- Pull-request tagged with: testing-ready

2 new commits added

  • kiwi: check include directive
  • basic kiwi docs

This should probably be called "Image builds using kiwi"

Would it make sense documenting a minimum version for this? kiwi-9.24.2 will have the JSON results feature, and possibly also the --releasever support...

1 new commit added

  • update docs

rebased onto aba798cb7b14ba9a67274282010518281256d7e6

1 new commit added

  • use json results

can we actually do it (possibly optionally?) other way around? generate version in koji using some kind of getNextRelease() like we do for some other types and then pass it to xml?

results = {str(k): v for k, v in results.items()}

I did not have chance to test this PR, but from code it seems that the "NVR" includes only name, version and release but not profile which would mean that you essentially can't build/tag same image with different profiles. Is my understanding correct?

Yeah, the "name" recorded by Koji should probably include the profile too...

instead of sorting those repos, would not it make sense to mirror koji's repository inheritance (with weight)?
From kiwi documentation it supports repository priorities:

priority: An integer priority for all packages in this repository. If the same package is available in more than one repository, then the one with the highest priority is used.

wouldn't it be better to just compare it with (none) and otherwise just convert it to int()?

but not all of them are external? I mean, if some "external" repos are used in build tag, some of them would be... but otherwise some RPMs are directly taken from koji. Or is my assumption wrong?

I think in 2021 it is better to use argparse :) Curious when optparse will be removed from stdlib…

can we actually do it (possibly optionally?) other way around? generate version in koji using some kind of getNextRelease() like we do for some other types and then pass it to xml?

I understand kiwi's file more like a spec. If nvr is already there, maintainer should bump it in description file if he wants higher release. It seems more right to me than have some confusing commit with nvrA and resulting build with nvrB.

can we actually do it (possibly optionally?) other way around? generate version in koji using some kind of getNextRelease() like we do for some other types and then pass it to xml?

I understand kiwi's file more like a spec. If nvr is already there, maintainer should bump it in description file if he wants higher release. It seems more right to me than have some confusing commit with nvrA and resulting build with nvrB.

Fair point, but being able to rewrite that for snapshot releases is probably useful.

I think in 2021 it is better to use argparse :) Curious when optparse will be removed from stdlib…

I would stay with optparse for now. We will have to make a leap with whole codebase to argparse or something else (migrate CLI commands to click (or whatever else)). Until that it is better to have same lib everywhere.

but not all of them are external? I mean, if some "external" repos are used in build tag, some of them would be... but otherwise some RPMs are directly taken from koji. Or is my assumption wrong?

markExternalRPMS is a helper function which will find external rpms in the list and alter their buildinfos with link to external repo, etc. It is used in the same way for all archive/image builds. So, yes, most rpms are expected to be from koji directly.

instead of sorting those repos, would not it make sense to mirror koji's repository inheritance (with weight)?
From kiwi documentation it supports repository priorities:

priority: An integer priority for all packages in this repository. If the same package is available in more than one repository, then the one with the highest priority is used.

Sorting there is just for human-reading convenience. It doesn't have any impact. We're not using priorities anywhere now. But I'm not against adding it here. Question would be - what is the priority of buildroot repo vs. repos which are supplied by the user? I was thinking if even --repo options should exist here if we shouldn't limit it to buildroot repo itself. But as for other image types it is probably useful to have it configurable. But it similarly to those open door to "untracked" content not present in koji.

Yeah, the "name" recorded by Koji should probably include the profile too...

I'm hesitating here. I was thinking that maybe we want one build with more profiles? So, something like koji kiwi-build --profile=Live,Virtual,Disk .... and have one NVR with multiple attached images. Second option is to link name somehow with profile. name = f"{name}-{profile" and third (I don't like this one) to allow user to override the name from CLI. Do you think that first variant is not practical?

1 new commit added

  • code simplification

can we actually do it (possibly optionally?) other way around? generate version in koji using some kind of getNextRelease() like we do for some other types and then pass it to xml?

I understand kiwi's file more like a spec. If nvr is already there, maintainer should bump it in description file if he wants higher release. It seems more right to me than have some confusing commit with nvrA and resulting build with nvrB.

Fair point, but being able to rewrite that for snapshot releases is probably useful.

I'm looking again into http://osinside.github.io/kiwi/image_description/elements.html#preferences-version - It seems that we can treat it as "version" (of NVR) only. In such case release for us could be a different field handled by --release or getNextRelease and we don't need to touch xml in such case. Does it make sense?

Yeah, the "name" recorded by Koji should probably include the profile too...

I'm hesitating here. I was thinking that maybe we want one build with more profiles? So, something like koji kiwi-build --profile=Live,Virtual,Disk .... and have one NVR with multiple attached images. Second option is to link name somehow with profile. name = f"{name}-{profile" and third (I don't like this one) to allow user to override the name from CLI. Do you think that first variant is not practical?

I suppose the first variant is practical. That would allow them to be built in parallel too, right?

rebased onto 3f51428c98f14634232e48b2848bc9a92cd0b181

2 new commits added

  • alter name by selected profile
  • fix signature

pretty please pagure-ci rebuild

bandit tests failed: https://jenkins-fedora-infra.apps.ocp.ci.centos.org/job/koji/3335/

rebased onto 71e1dc13b998eabcecd97529d50fa7f3b1cd8403

1 new commit added

  • fix bandit

1 new commit added

  • expand docs

1 new commit added

  • fix flake8

1 new commit added

  • fix docs

rebased onto d31306e2f673a89a7791cf209720f52318502816

Commit 101433a7 fixes this pull-request

Pull-Request has been merged by tkopecek

Metadata