@ngompa So, release-version is same as version now. Is it ok or do you need to set it to something different (so have --releasever CLI option)?

/for testing it with the latest kiwi, I've included also json PR commits here - they'll not be a part of final PR/

Is that how it works for livecd, appliance, livemedia, and image tasks?

I want the final description and image to be versioned <version>.<datestamp>.<buildcount> while release-version is set to <version> where <version> is the Fedora release version and <datestamp> is that extra datestamp to indicate when it was built and <buildcount> is an incrementing integer for the day.

So, for example, a F36 image built today would get the following XML:

<version>36.20220105.1</version>
<release-version>36</version>

Does that make sense?

Here are a couple of examples of what I'm talking about using the livemedia task:

• https://koji.fedoraproject.org/koji/buildinfo?buildID=1849945
• https://koji.fedoraproject.org/koji/buildinfo?buildID=1872468

Is that how it works for livecd, appliance, livemedia, and image tasks?
Yes, for livemedia it is that iso name is <name>-<arch>-<version>-<release>.iso, while livemedia-creator is being run with same --releasever <version> and it is used only for bootloader modification (https://weldr.io/lorax/livemedia-creator.html).

Other tools don't modify this value as they're not doing bootstrapping and run installation directly.

More general problem is the absence of release field in kiwi descriptions. Current code path will just use autoincrement always. We can "set a standard" that version in input file would be "36-20220105.1", so it could be split to version-release or add release tag support to kiwi descriptions. In such case, releasever could be just version part. Other option is to supply --release via CLI which would mean that we can have more releases based on the same commit which is not that nice. Any other idea?

@tkopecek could you bring this up in the kiwi issue? https://github.com/OSInside/kiwi/issues/1918

@ngompa I've updated code to use 'result bundle' command. It has following sideeffects:
1) You can use --release (or autoincrement) as for other image types
2) release is not reflected in description file, just in build
3) It overrides behaviour of #3198 as only files with use_for_bundle: true are renamed to NVR, so only subset of files which were in results are copied there. Anyway, it should be configurable on profile level. (@ignatenkobrain)

@tkopecek ad. 3 - please make sure that something like https://pagure.io/koji/pull-request/3200 is included so that we don't have issue with having different image types.

kiwi and kiwi-cli are the same package. I would just use kiwi-cli and kiwi-systemdeps here.

kiwi and kiwi-cli are the same package. I would just use kiwi-cli and kiwi-systemdeps here.

kiwi is here name of build group not the package. I'll rename it to kiwi-build so it is in consistence with 'rpm-build', 'image-build', etc.

So I have tested this, it does not fail… although I don't get any image files :)

https://ignatenkobrain.fedorapeople.org/Screenshot%20from%202022-01-15%2010-59-56.png

From the logs:

DEBUG util.py:542:  Executing command: ['kiwi-ng', 'result', 'bundle', '--target-dir', '/builddir/result/image', '--bundle-dir', '/builddir/result/bundle', '--id', '1'] with env {'TERM': 'vt100', 'SHELL': '/bin/bash', 'HOME': '/builddir', 'HOSTNAME': 'mock', 'PATH': '/usr/bin:/bin:/usr/sbin:/sbin', 'PROMPT_COMMAND': 'printf "\\033]0;<mock-chroot>\\007"', 'PS1': '<mock-chroot> \\s-\\v\\$ ', 'LANG': 'C.UTF-8'} and shell False
DEBUG util.py:446:  [ INFO    ]: 10:57:54 | Reading runtime config file: '/etc/kiwi.yml'
DEBUG util.py:446:  [ INFO    ]: 10:57:54 | Bundle build results from /builddir/result/image
DEBUG util.py:446:  [ INFO    ]: 10:57:54 | Creating gdc-c9s-PrivateCloud.x86_64-0.0.0-1.raw
DEBUG util.py:446:  [ INFO    ]: 10:57:55 | --> XZ compressing
DEBUG util.py:446:  [ INFO    ]: 10:58:20 | --> Creating SHA 256 sum
DEBUG util.py:446:  [ INFO    ]: 10:58:21 | Creating gdc-c9s-PrivateCloud.x86_64-0.0.0-1.changes
DEBUG util.py:446:  [ INFO    ]: 10:58:21 | --> XZ compressing
DEBUG util.py:446:  [ INFO    ]: 10:58:21 | Creating gdc-c9s-PrivateCloud.x86_64-0.0.0-1.packages
DEBUG util.py:446:  [ INFO    ]: 10:58:21 | Creating gdc-c9s-PrivateCloud.x86_64-0.0.0-1.verified
DEBUG util.py:598:  Child return code was: 0

brain@powerful-box ~/P/g/g/gdc-c9s (c9s)> ll out/
total 915M
drwxr-xr-x. 1 root root   48 Jan 15 14:44 build/
-rw-r--r--. 1 root root 2.0M Jan 15 14:46 gdc-c9s.x86_64-0.0.0.changes
-rw-r--r--. 1 root root  18K Jan 15 14:46 gdc-c9s.x86_64-0.0.0.packages
-rw-r--r--. 1 root root 1.2G Jan 15 14:46 gdc-c9s.x86_64-0.0.0.raw
-rw-r--r--. 1 root root  644 Jan 15 14:46 gdc-c9s.x86_64-0.0.0.verified
-rw-r--r--. 1 root root  11K Jan 15 14:46 kiwi.result
-rw-r--r--. 1 root root  869 Jan 15 14:46 kiwi.result.json
brain@powerful-box ~/P/g/g/gdc-c9s (c9s)> ll bundle/
total 213M
-rw-r--r--. 1 root root 221K Jan 15 14:49 gdc-c9s.x86_64-0.0.0-1.changes.xz
-rw-r--r--. 1 root root  18K Jan 15 14:49 gdc-c9s.x86_64-0.0.0-1.packages
-rw-r--r--. 1 root root 212M Jan 15 14:49 gdc-c9s.x86_64-0.0.0-1.raw.xz
-rw-r--r--. 1 root root   96 Jan 15 14:49 gdc-c9s.x86_64-0.0.0-1.raw.xz.sha256
-rw-r--r--. 1 root root  644 Jan 15 14:49 gdc-c9s.x86_64-0.0.0-1.verified

I suppose in case of bundle we should just upload all files from the bundle's directory and not even think about parsing JSONs, etc.

What do you think?

That probably makes sense.

@ngompa @tkopecek https://pagure.io/fork/tkopecek/koji/pull-request/1

FTR I'm using this patchset and so far I haven't found any issues (though I did not go further than the scratch builds).

One thing that would be useful is to be able to run some a-la make tarball before kiwi is executed. I have 2 use-cases for it:

1. Kiwi has support for injecting files into the image (it automatically adds root folder in there and you can load any tar file via the <archive name="…"/>). Sadly the second case really requires having tar file… which one does not normally want to commit to a git and use rather uncompressed/unarchived form of it. That's where make … would be useful.
2. Whilst having automatically appended release number (-1, -2, …) is nice, I'd like to have date inside the name of the image without having to explicitly pass --release 20220117.123 in the command-line (I suppose I have to have it unique myself or it will fail).

And may be, 3rd one: Koji is wiping completely contents of the <repositories> section while I have a use-case where I want to have it there (with some more attributes like includeimage="true" profiles="Cloud" which is not possible to easily (at all) set via the command line. So basically running some make … after repos preparation that could modify a kiwi file could be handy.

And may be, 3rd one: Koji is wiping completely contents of the section while I have a use-case where I want to have it there (with some more attributes like includeimage="true" profiles="Cloud" which is not possible to easily (at all) set via the command line. So basically running some make … after repos preparation that could modify a kiwi file could be handy.

You should probably not use the kiwi config for repos and build a repos package instead.

Kiwi has support for injecting files into the image (it automatically adds root folder in there and you can load any tar file via the ). Sadly the second case really requires having tar file… which one does not normally want to commit to a git and use rather uncompressed/unarchived form of it. That's where make … would be useful.

It can also unpack a root.tar.gz if it exists instead of a root directory.

@ngompa

It can also unpack a root.tar.gz if it exists instead of a root directory.

I know, but I want exactly opposite, for different profiles, use different root directory.

You should probably not use the kiwi config for repos and build a repos package instead.

Yes, that however does not help with the image-build-time repositories (which I want different per profile).

Yes, that however does not help with the image-build-time repositories (which I want different per profile).

Would that even work correctly, given that Koji constructs a repo to use as input for the build based on what you want anyway and internet access wouldn't be available inside the execution environment for non-Koji external repos anyway...

FTR I'm using this patchset and so far I haven't found any issues (though I did not go further than the scratch builds).

One thing that would be useful is to be able to run some a-la make tarball before kiwi is executed. I have 2 use-cases for it:

1. Kiwi has support for injecting files into the image (it automatically adds root folder in there and you can load any tar file via the <archive name="…"/>). Sadly the second case really requires having tar file… which one does not normally want to commit to a git and use rather uncompressed/unarchived form of it. That's where make … would be useful.

Hmm, it is interesting. Maybe it makes more sense to extend kiwi here? To allow also direct inclusion of directory? But it has some caveats (e.g. random ownership). So, maybe adding something like "make prep" makes sense (e.g. for modifying description file also not only creatign tarballs).

1. Whilst having automatically appended release number (-1, -2, …) is nice, I'd like to have date inside the name of the image without having to explicitly pass --release 20220117.123 in the command-line (I suppose I have to have it unique myself or it will fail).

I would be different from other image commands. It needs to be unique. On the other hand you can run it without release after first one that day and it will be correctly autoincremented (same as for other images). So, if last builds has release "20220117.123" new one without explicit release will be "20220117.124". I would like to have this behaving the same way as other images.

And may be, 3rd one: Koji is wiping completely contents of the <repositories> section while I have a use-case where I want to have it there (with some more attributes like includeimage="true" profiles="Cloud" which is not possible to easily (at all) set via the command line. So basically running some make … after repos preparation that could modify a kiwi file could be handy.

Isn't CLIs --repo option enough here?

@tkopecek

So, maybe adding something like "make prep" makes sense (e.g. for modifying description file also not only creatign tarballs).

yep, that's what I was thinking about. It could cover any cases that I could come up with myself.

I would be different from other image commands. It needs to be unique. On the other hand you can run it without release after first one that day and it will be correctly autoincremented (same as for other images). So, if last builds has release "20220117.123" new one without explicit release will be "20220117.124". I would like to have this behaving the same way as other images.

So if I call --release 20220118 for the first time, it will be as-is and then if I call it again with same arguments, it will get .1?

Isn't CLIs --repo option enough here?

I don't think it can set includeimage="true" profiles="Cloud" kind of things. It is just a URL, without any properties. Probably if we allow passing those extra XML attributes, then it would be good… Or I'd even like to have some special option in XML like x-koji-keep="true" which won't remove it when rewriting those repositories, this would be probably ideal.

I don't think it can set includeimage="true" profiles="Cloud" kind of things. It is just a URL, without any properties. Probably if we allow passing those extra XML attributes, then it would be good… Or I'd even like to have some special option in XML like x-koji-keep="true" which won't remove it when rewriting those repositories, this would be probably ideal.

This, however, can be covered completely by the make prep :)

I would be different from other image commands. It needs to be unique. On the other hand you can run it without release after first one that day and it will be correctly autoincremented (same as for other images). So, if last builds has release "20220117.123" new one without explicit release will be "20220117.124". I would like to have this behaving the same way as other images.

So if I call --release 20220118 for the first time, it will be as-is and then if I call it again with same arguments, it will get .1?

Not in this case - such release is treated as an integer, so next one would be 20220119 (see https://pagure.io/koji/blob/master/f/hub/kojihub.py#_4415 getNextRelease logic)

Isn't CLIs --repo option enough here?

I don't think it can set includeimage="true" profiles="Cloud" kind of things. It is just a URL, without any properties. Probably if we allow passing those extra XML attributes, then it would be good… Or I'd even like to have some special option in XML like x-koji-keep="true" which won't remove it when rewriting those repositories, this would be probably ideal.

Ok, I'll give it a thought.

I don't think it can set includeimage="true" profiles="Cloud" kind of things. It is just a URL, without any properties. Probably if we allow passing those extra XML attributes, then it would be good… Or I'd even like to have some special option in XML like x-koji-keep="true" which won't remove it when rewriting those repositories, this would be probably ideal.

This, however, can be covered completely by the make prep :)

If I implement "make prep" I still would like to modify description after that. So, weeping repos would still happen.

If I implement "make prep" I still would like to modify description after that. So, weeping repos would still happen.

Can we have then make prep-pre and make prep-post or just execute make prep after we modify description?

should not this be either executed within a desc_path (inside a buildroot) or at least get this info somehow inside?

Commit facfdb0d fixes this pull-request