A few builders have no space free, free up space on them and investigate why this is happening.
at least:
buildvm-s390x-20.s390.fedoraproject.org | CHANGED | rc=0 | (stdout) Filesystem Size Used Avail Use% Mounted on\n/dev/vda3 96G 91G 167M 100% / buildvm-s390x-22.s390.fedoraproject.org | CHANGED | rc=0 | (stdout) Filesystem Size Used Avail Use% Mounted on\n/dev/vda3 96G 89G 2.5G 98% / buildvm-s390x-24.s390.fedoraproject.org | CHANGED | rc=0 | (stdout) Filesystem Size Used Avail Use% Mounted on\n/dev/vda3 96G 83G 8.0G 92% /
reported on fedora-devel.
So, it looks like this was just unfortunate timing, some large failed builds hitting at the same time. I cleared all the builders than were over 90%.
Upstream koji is taking about how better to handle recycling failed builds space, so hopefully out of that will come a better method than we have now.
Metadata Update from @kevin: - Issue close_status updated to: Fixed - Issue status updated to: Closed (was: Open)
JFYI https://koji.fedoraproject.org/koji/taskinfo?taskID=40345807 / https://koschei.fedoraproject.org/build/7569428 / https://koschei.fedoraproject.org/package/pypy3?collection=f32 failed with missing space on aarch64. I will open another ticket if it proves to be persistent.
@churchyard the current ticket on this is https://pagure.io/fedora-infrastructure/issue/8522