From 693a116875bbf21ab84c839d8930f2e581830327 Mon Sep 17 00:00:00 2001 From: Anton Medvedev Date: Oct 02 2025 12:09:28 +0000 Subject: This is part of releng repo refactoring -> clean_amis.py --- diff --git a/modules/ROOT/nav.adoc b/modules/ROOT/nav.adoc index 4cfcd26..6739668 100644 --- a/modules/ROOT/nav.adoc +++ b/modules/ROOT/nav.adoc @@ -67,7 +67,6 @@ *** xref:releng_misc_guide:sop_request_an_automation_user.adoc[Request an Automation User] *** xref:releng_misc_guide:sop_process_dist_git_requests.adoc[Process `fedora-scm-requests` Tickets] *** xref:releng_misc_guide:sop_signing_builds.adoc[Sign the Packages] -*** xref:releng_misc_guide:sop_clean_amis.adoc[Clean AMIs] *** xref:releng_misc_guide:sop_pushing_updates.adoc[Push Updates] *** xref:releng_misc_guide:sop_sigul_client_setup.adoc[Sigul Client Setup] *** xref:releng_misc_guide:sop_troubleshooting.adoc[General Troubleshooting Guide] diff --git a/modules/releng_misc_guide/pages/index.adoc b/modules/releng_misc_guide/pages/index.adoc index 5ec08a0..06ba739 100644 --- a/modules/releng_misc_guide/pages/index.adoc +++ b/modules/releng_misc_guide/pages/index.adoc @@ -22,7 +22,6 @@ Here are some of the miscellaneous tasks performed by the Fedora Release Enginee * link:sop_requesting_task_automation_users[Requesting Automation Users] * link:sop_process_dist_git_requests[Process fedora-scm-requests tickets] * link:sop_signing_builds[Sign the packages] -* link:sop_clean_amis[Clean AMIs Process] * link:sop_pushing_updates[Pushing Updates] * link:sop_package_blocking[Package Blocking] * link:sop_sigul_client_setup[Sigul Client Setup] diff --git a/modules/releng_misc_guide/pages/sop_clean_amis.adoc b/modules/releng_misc_guide/pages/sop_clean_amis.adoc deleted file mode 100644 index af17010..0000000 --- a/modules/releng_misc_guide/pages/sop_clean_amis.adoc +++ /dev/null @@ -1,59 +0,0 @@ -= Clean AMIs Process - -== Description - -The Fedora AMIs are uploaded on a daily basis to Amazon Web Services. -Over time the number of AMIs piles up and have to be removed manually. -Manual removal comes with it's own set of issues where missing to delete -the AMIs is a viable issue. - -The goal of the script is to automate the process and continue regular -removal of the AMIs. The report of the script is pushed to a -https://pagure.io/ami-purge-report[Pagure repo] - -== Action - -There is a script in the https://pagure.io/releng[Fedora RelEng repo] -named `clean-amis.py` under the `scripts` directory. - -The script runs as a cron job within the Fedora Infrastructure to delete -the old AMIs. The permission of the selected AMIs are changed to -private. This is to make sure that if someone from the community raises -an issue we have the option to get the AMI back to public. After 10 -days, if no complaints are raised the AMIs are deleted permanently. - -The complete process can be divided in couple of parts: - -* Fetching the data from datagrepper. Based on the [.title-ref]#--days# -param, the script starts fetching the fedmsg messages from datagrepper -for the specified timeframe i.e. for lasts [.title-ref]#n# days, where -[.title-ref]#n# is the value of [.title-ref]#--days# param. The queried -fedmsg topic [.title-ref]#fedimg.image.upload#. -* Selection of the AMIs: After the AMIs are parsed from datagrepper. The -AMIs are filtered to remove Beta, Two-week Atomic Host and GA released -AMIs. Composes with [.title-ref]#compose_type# set to -[.title-ref]#nightly# are picked up for deletion. Composes which contain -date in the [.title-ref]#compose label# are also picked up for deletion. -GA composes also have the compose_type set to production. So to -distinguish then we filter them if the compose_label have date in them. -The GA composes dont have date whereas they have the version in format -of X.Y -* Updated permissions of AMIs The permissions of the selected AMIs are -changed to private. -* Deletion of AMIs After 10 days, the private AMIs are deleted. - -In order to change the permissions of the AMIs use the command given -below, add [.title-ref]#--dry-run# argument test the command works. -Adding [.title-ref]#--dry-run# argument will print the AMIs to console. - -.... -AWS_ACCESS_KEY={{ ec2_image_delete_access_key_id }} AWS_SECRET_ACCESS_KEY={{ ec2_image_delete_access_key }} PAGURE_ACCESS_TOKEN={{ ami_purge_report_api_key }} ./clean-amis.py --change-perms --days 7 --permswaitperiod 5 -.... - -In order to delete the AMIs whose launch permissions have been removed, -add [.title-ref]#--dry-run# argument test the command works. Adding -[.title-ref]#--dry-run# argument will print the AMIs to console. - -.... -AWS_ACCESS_KEY={{ ec2_image_delete_access_key_id }} AWS_SECRET_ACCESS_KEY={{ ec2_image_delete_access_key }} PAGURE_ACCESS_TOKEN={{ ami_purge_report_api_key }} ./clean-amis.py --delete --days 17 --deletewaitperiod 10 -....