We normally only keep images that are newer than 30 days on candidate-registry... ones that go stable move to registry.
However, the cleanup job isn't working anymore:
registry garbage-collect /etc/docker-distribution/registry/config.yml ... fedora: marking blob sha256:04064b4419f6b00bf312a1d9f3ee6f5cba384b6725d05357722d8898dcc671a1 fedora: marking blob sha256:78a4d8c241dd8e79cf0fa1a0ca722449cfbe9e92c53f5b0c611aa617b7c5eca4 fedora: marking manifest sha256:fff9a474898b88a7e12f3448482cdf696f3cc2d57c77f9d964e6dd20e1f83160 fedora: marking blob sha256:b0d6b2c835bf5c9779c5eb66d2e66f9340db3f44158a56e46dacb29034e4b426 fedora: marking blob sha256:193ad33abeefed1950a93fa7e04d766ad73ff9eca4494dd71ff6f0a33031f8ec fedora: marking manifest sha256:fffe902dfec4d261089333bf63bb2a87048562fc188bc6aeac95b5f5153bb55f fedora: marking blob sha256:1c45f1475ba51efd10aae9e992b7ae0ac74343a3e0d2e1b65bb05c11afb44993 fedora: marking blob sha256:7bc4a5b1233f9f017006c181907b2a6b31d0af94073d07c06efda8fca53153a0 fedora: marking manifest sha256:ffff5c6f62344f26f5c9e91d48a2b01996af386efed7da1700363ab898718442 fedora: marking blob sha256:e0fc4c21643c4d49cfffdcfb653af308d508a0fb21a52c4c0b7e8e4ea1077dde fedora: marking blob sha256:f5fd767d156b3deb7eba6ce68dd6ea8e84b06258f01b201c68b1934dd82dcb93 fedora-container-image fedora-kinoite fedora-kinoite: marking manifest sha256:5a7f074e6c481708937e9e11dd0f814f7117ec17faf2f1bc2609d0c51191c3cc failed to garbage collect: failed to mark: failed to retrieve manifest for digest sha256:5a7f074e6c481708937e9e11dd0f814f7117ec17faf2f1bc2609d0c51191c3cc: errors verifying manifest: unrecognized manifest content type
CC: @walters @siosm any ideas?
What's the code backing registry garbage-collect? Is this docker-distribution?
registry garbage-collect
I think the basic problem here is likely to be that the rpm-ostree stack generates OCI images by default, whereas the application base image i.e. quay.io/fedora/fedora:latest is still docker schema 2:
quay.io/fedora/fedora:latest
$ skopeo inspect --raw docker://quay.io/fedora/fedora:latest | jq .mediaType "application/vnd.docker.distribution.manifest.list.v2+json" $
The latest docker-distribution should support OCI AFAIK.
Yeah, it's:
docker-distribution-2.6.2-18.git48294d9.fc36.x86_64
Will look for an update! Thanks for the quick response...
Metadata Update from @phsmoura: - Issue tagged with: medium-gain, medium-trouble, ops
ok, upgrading to 2.8.1 does indeed fix this. It also breaks a local script/ansible module we have, but I'll file an infra issue on that one. ;)
Metadata Update from @kevin: - Issue untagged with: medium-gain, medium-trouble, ops - Issue close_status updated to: Fixed - Issue status updated to: Closed (was: Open)