So, let's try this again?

I went with the subvariants we had before, but...maybe we could rethink them. Is the "base" in "IoT_bootc_base" appropriate? Might there be other, less base-y IoT bootc images in future? By the same token, should the other one say any more than just "bootc"?

I used the 'name' field we added to pungi to try and get sensible names. Since the bootc image is not in any way IoT specific, I used the 'name' field to cut "IoT" out of its name, so it'll just be "Fedora-bootc-whatever". For the IoT one, I used the name field to avoid "IoT" being duplicated (otherwise it'd be Fedora-IoT-IoT_bootc_base-whatever).

oh, to be clear, the pungi PR isn't just merged, I backported it and a Pungi with it included is stable in Rawhide.

Ideally this is all reversed: Those are both bootc images, one is the base subvariant and the other is the iot one.

^bootc$

IoT

base

Fedora-IoT

I think we need to keep IoT for the lclassic ostree ones and bootc for the new ones with my suggestions

With the suggested changes from @siosm , LGTM. Thank you both for looking at this.

I have some notes on that, but I'm in a meeting rn. I want to be sure we are all on the same page about what the consequences of that would be.

So, I think the easiest way to explain is just to mock up the three states. Here is the directory layout we currently get (well it's not quite this as the currently-installed pungi doesn't have my patch to include the arch in the bootc filenames, but if we had that pungi deployed, this is what we'd get):

compose/
    bootc/
        (arch)/
            images/
                Fedora-IoT-bootc-(version)-(arch).ociarchive
    bootc-base/
        (arch)/
            images/
                Fedora-IoT-bootc-base-(version)-(arch).ociarchive
    IoT/
        (arch)
            images/
                Fedora-IoT-raw-(version).(arch).raw.xz
            iso/
                Fedora-IoT-ostree-(version).(arch).iso
                Fedora-IoT-provisioner-(version).(arch).iso

Here is the condensed metadata layout:

"images": {
    "IoT": {
        (arch): [
            {
                "arch": (arch),
                "format": "raw.xz",
                "path": "IoT/(arch)/images/Fedora-IoT-raw-(version).(arch).raw.xz",
                "subvariant": "IoT",
                "type": "raw-xz",
            },
            {
                "arch": (arch),
                "format": "iso",
                "path": "IoT/(arch)/iso/Fedora-IoT-ostree-(version).(arch).iso",
                "subvariant": "IoT",
                "type": "dvd-ostree",
            },
            {
                "arch": (arch),
                "format": "iso",
                "path": "IoT/(arch)/iso/Fedora-IoT-provisioner-(version).(arch).iso",
                "subvariant": "IoT_Simplified_Provisioner",
                "type": "dvd-ostree",
            }
        ]...
    },
    "bootc": {
        (arch): [
            {
                "arch": (arch),
                "format": "ociarchive",
                "path": "bootc/(arch)/images/Fedora-IoT-bootc-42.20241016.0.ociarchive",
                "subvariant": "bootc",
                "type": "ociarchive",
            }
        ]...
    },
    "bootc-base": {
        (arch): [
            {
                "arch": (arch),
                "format": "ociarchive",
                "path": "bootc-base/(arch)/images/Fedora-IoT-bootc-base-42.20241016.0.ociarchive",
                "subvariant": "bootc-base",
                "type": "ociarchive",
            }
        ]...
    }
}

Here is the directory tree we'd get with my PR as it stands:

compose/
    IoT/
        (arch)/
            images/
                Fedora-bootc-(version)-(arch).ociarchive
                Fedora-IoT-bootc_base-(version)-(arch).ociarchive
                Fedora-IoT-raw-(version).(arch).raw.xz
            iso/
                Fedora-IoT-ostree-(version).(arch).iso
                Fedora-IoT-provisioner-(version).(arch).iso

and the metadata:

"images": {
    "IoT": {
        (arch): [
            (existing images),
            {
                "arch": (arch),
                "format": "ociarchive",
                "path": "IoT/(arch)/images/Fedora-bootc-(version)-(arch).ociarchive",
                "subvariant": "bootc",
                "type": "ociarchive",
            },
            {
                "arch": (arch),
                "format": "ociarchive",
                "path": "IoT/(arch)/images/Fedora-IoT-bootc_base-(version)-(arch).ociarchive",
                "subvariant": "IoT_bootc_base",
                "type": "ociarchive",
            }
        ]...
    }
}

Here is the tree for @siosm 's proposal:

compose/
    bootc/
        (arch)/
            images/
                Fedora-bootc-(version)-(arch).ociarchive
                Fedora-IoT-(version)-(arch).ociarchive
    IoT/
        (as now)

and the metadata:

"images": {
    "IoT": {
        (as now)
    },
    "bootc": {
        (arch): [
            {
                "arch": (arch),
                "format": "ociarchive",
                "path": "bootc/(arch)/images/Fedora-bootc-(version)-(arch).ociarchive",
                "subvariant": "base",
                "type": "ociarchive",
            },
            {
                "arch": (arch),
                "format": "ociarchive",
                "path": "bootc/(arch)/images/Fedora-IoT-(version)-(arch).ociarchive",
                "subvariant": "IoT",
                "type": "ociarchive",
            }
        ]...
    }
}

Looking at siosm's proposal, here's what I have concerns about. The subvariant for the IoT base image is just "IoT", and its filename is "Fedora-IoT-(version)-(arch).ociarchive". Nothing beyond the fact that it's in the "bootc" variant/directory identifies it as a bootc image.

This is potentially confusing to a person looking at the file outside of the tree, or to someone parsing metadata and ignoring variants. I find it's good to keep that in mind, because if you look at the metadata, the variant is not included directly in the individual image dict, and it's a pain to 'keep it in mind' programatically - the easy thing to do is just extract the individual image dicts and use those. So if you look at the existing metadata for all composes, the subvariant tends to duplicate any relevant information from the variant. (This is 'canonicalized' to some extent in productmd, because 'subvariant' is in the set of properties considered to uniquely define an image, but 'variant 'is not). Looking at that filename and the metadata in the image dict, how do I know this is a bootc image and not just a 'normal' IoT container image?

Additionally, an image having a subvariant which is identical to a variant but not being part of that variant is rather odd. If an image does not have a subvariant explicitly set, productmd makes its subvariant identical to its variant. So when you see an image with a subvariant that's the same as a variant, you assume it's an image from that variant without an explicit subvariant.

Similarly, the subvariant of just 'base' for the generic base image is a bit thin - outside of the variant and an element of the filename (and we really try to avoid needing to parse filenames), nothing identifies it as a bootc image.

'bootc' as a variant is a bit unusual. It's trivial, but - every existing variant (AFAIK) starts with a capital letter. Variants are often about the content, not the format; we have Server, Everything, Cloud, Silverblue etc. Although there is the proposal to roll all the atomic desktops into a variant which would be a kinda hybrid format/content variant, I guess, and we do have the Spins and Labs variants, which again are a bit hybrid. We also have the Container variant in the main compose, which would kinda rub up against a 'bootc' variant oddly if we moved the generic base bootc image into the main compose; would we just put it in 'Container' in the main compose, but keep a 'bootc' variant in the IoT compose for the IoT bootc image? Or have 'Container' and 'bootc' variants in the main compose?

Concerns with my PR as it stands: I don't love the subvariant of just "bootc" for the generic base image. It could be "bootc-base", or just "base" if we had the "bootc" type (see below). The generic base image is inside the IoT "variant", though I intentionally planned it to be easy to move out into the main compose, as that feels like where it logically ought to be unless I'm missing something.

One thing that occurs to me writing this is that a type value of "bootc" (or something like that) would solve a lot of the concerns. As mentioned elsewhere I don't like both the type and format of these images being "ociarchive" anyway. Changing that would need more pungi patching, though (and I think maybe a productmd patch as well, I think the 'valid' types are defined in productmd).

The generic base image is inside the IoT "variant", though I intentionally planned it to be easy to move out into the main compose, as that feels like where it logically ought to be unless I'm missing something.

As a part of this we should look at moving the generic "bootc" image to the main compose. We added "base" to our image name to differentiate between the two, with the "name" field added we can drop that in the IoT specific image and include just "bootc" in the name to help identify it.

One thing that occurs to me writing this is that a type value of "bootc" (or something like that) would solve a lot of the concerns. As mentioned elsewhere I don't like both the type and format of these images being "ociarchive" anyway. Changing that would need more pungi patching, though (and I think maybe a productmd patch as well, I think the 'valid' types are defined in productmd).

That sounds like the best option.

sorry, got distracted by other stuff. Here's a PR to add the 'bootc' type to productmd: https://github.com/release-engineering/productmd/pull/181

that got merged, so I guess next I'll look at whether I want to patch pungi to use the new types by default anywhere, then come back to this and propose something using them. edit - pungi pr: https://pagure.io/pungi/pull-request/1810

when I finally get back to this, we can also close out https://pagure.io/releng/issue/12142 (this is a note to self so I don't need to keep both tabs open any more)

OK, I've made sure productmd changes got stable now, and sent a Pungi update with the Pungi changes which I'll deploy on the rawhide builder...

What's the intersection of this and https://gitlab.com/fedora/bootc/base-images/-/merge_requests/70 ?

I'd really like to stop using pungi invoke rpm-ostree and instead have the entrypoint to the base image build be via podman build.

I would say no intersection, except that if we go to konflux builds as the Official Thing, this becomes kinda obsolete. at least, this does nothing to block that. this is all about pungi config.

OK, sorry for the delay. I have now tweaked this again; it's still more or less my version, but now the subvariants and base names can be better. Here's the summary from the commit message:

With this change, the variant for both images will be "IoT" and
they will both appear in the IoT directory in the tree. The IoT
image's subvariant will also be "IoT". The base image's
subvariant will be "base". The format for both images will be
"ociarchive", as it currently is. From tomorrow, the type for
both images will be "bootable-container". The filename for the
IoT image will start with "Fedora-IoT-bootc", and the filename
for the base image will start with "Fedora-base-bootc". So we
will have these basename - subvariant - type - formats:
* Fedora-IoT-bootc - IoT - bootable-container - ociarchive
* Fedora-base-bootc - base - bootable-container - ociarchive

@pwhalen ping?

Thanks for working on this Adam! However

if we go to konflux builds as the Official Thing, this becomes kinda obsolete

Is much closer now since https://gitlab.com/fedora/bootc/tracker/-/issues/33 is progressing.

@pwhalen ping?

Thanks Adam, sorry for the delay.

LGTM, needs a rebase.

Thanks!

Commit d1728cd3 fixes this pull-request