From c69ded5a2461b187c847a85e2164792da9edfb2a Mon Sep 17 00:00:00 2001 From: Tomas Kopecek Date: Oct 15 2020 12:27:34 +0000 Subject: [PATCH 1/4] Release notes 1.23 Fixes: https://pagure.io/koji/issue/2446 --- diff --git a/docs/source/migrations/migrating_to_1.23.rst b/docs/source/migrations/migrating_to_1.23.rst new file mode 100644 index 0000000..d837f14 --- /dev/null +++ b/docs/source/migrations/migrating_to_1.23.rst @@ -0,0 +1,24 @@ +Migrating to Koji 1.23 +====================== + +You should consider the following changes when migrating to 1.23: + +DB Updates +---------- + +There is one minor schema change in this release. We've dropped NOT NULL +restriction on tag_extra value column to allow block these values in hierarchy +(`PR#2495 `_). + +As in previous releases, we provide a migration script that updates the database. + +:: + + # psql koji koji < /usr/share/doc/koji/docs/schema-upgrade-1.22-1.23.sql + + +Other changes +------------- + +There are numerous other changes in 1.23 that should not have a direct impact on migration. For +details see: :doc:`../release_notes/release_notes_1.23` diff --git a/docs/source/migrations/migrations.rst b/docs/source/migrations/migrations.rst index 303d099..920dc8c 100644 --- a/docs/source/migrations/migrations.rst +++ b/docs/source/migrations/migrations.rst @@ -5,6 +5,7 @@ Migrations .. toctree:: :maxdepth: 1 + migrating_to_1.23 migrating_to_1.22 migrating_to_1.21 migrating_to_1.20 diff --git a/docs/source/release_notes/release_notes.rst b/docs/source/release_notes/release_notes.rst index 222a2ae..5d0f5a2 100644 --- a/docs/source/release_notes/release_notes.rst +++ b/docs/source/release_notes/release_notes.rst @@ -5,6 +5,7 @@ Release Notes .. toctree:: :maxdepth: 1 + release_notes_1.23 release_notes_1.22.1 release_notes_1.22 release_notes_1.21.1 diff --git a/docs/source/release_notes/release_notes_1.23.rst b/docs/source/release_notes/release_notes_1.23.rst new file mode 100644 index 0000000..043d0e6 --- /dev/null +++ b/docs/source/release_notes/release_notes_1.23.rst @@ -0,0 +1,241 @@ +Koji 1.23.0 Release notes +========================= + +All changes can be found at `pagure `_. +Most important changes are listed here. + + +Migrating from Koji 1.22/1.22.1 +------------------------------- + +For details on migrating see :doc:`../migrations/migrating_to_1.23` + + +Security Fixes +-------------- + +None + + +Client Changes +-------------- + +**cli: clone-tag fails on failed multicalls** + +| PR: https://pagure.io/koji/pull-request/2464 + +Previously some errors could have been hidden which could have led to missing +content in target tag. Now we are failing on any error. + +**improved download_file** + +| PR: https://pagure.io/koji/pull-request/2395 +| PR: https://pagure.io/koji/pull-request/2461 +| PR: https://pagure.io/koji/pull-request/2471 + +In the last version we've introduced unified ``download_file`` method which also +checks downloaded content. There are minor updates improving this function +especially in reaction to errors. + +**cli: add --task and --source options to list-builds command** + +| PR: https://pagure.io/koji/pull-request/2458 + +Builds now can be queried via ``task`` or ``source`` fields. + +**show log urls for failed tasks** + +| PR: https://pagure.io/koji/pull-request/2505 + +When watching for task progress, failed tasks will also display links to +relevant logs. + +**load plugins also from /usr/lib64** + +| PR: https://pagure.io/koji/pull-request/2525 + +Fix of the bug which missed plugins in /usr/lib64 prefix. + +**clone-tag --config clones also extra info** + +| PR: https://pagure.io/koji/pull-request/2472 + +Formerly cloning skipped extra values. Nevertheless, extra values are becoming +important part of config, so from now we are cloning also those. + +Library Changes +--------------- + +**getRPMHeaders now fetch all headers by default** + +| PR: https://pagure.io/koji/pull-request/2388 + +Simple change of library function default behaviour. + +API Changes +----------- + +**lowercase sigkeys during import/query where needed** + +| PR: https://pagure.io/koji/pull-request/2459 + +Case of signature hashes is properly ignored. + +**tagChangedSince reacts on changes in extra** + +| PR: https://pagure.io/koji/pull-request/2439 + +It is probably mostly used by kojira, but it can be of some interest to +automation scripts. + +**getAverageBuildDuration sliding window** + +| PR: https://pagure.io/koji/pull-request/2421 + +``getAverageBuildDuration`` returned average for all package builds. Old data +could be irrelevant to new version of packages, so age in months can now be +specified to limit data from which average is computed. + +**getBuildConfig returns inheritance history** + +| PR: https://pagure.io/koji/pull-request/2493 + +Additional option return inheritance chain for extra values and architectures. +This behaviour is meant to be used with next change. + +**blocking inherited extra** + +| PR: https://pagure.io/koji/pull-request/2495 + +Inherited tag extra fields could have been overriden but not removed. Now it can +be done via CLIs ``edit-tag`` or ``editTag2`` respectively which has new +``block_extra`` option. + +**deprecate getGlobalInheritance** + +| PR: https://pagure.io/koji/pull-request/2407 + +It should be replaced by ``readFullInheritance`` which is more general and +properly reflects various inheritance options. Call will be completely removed +in koji 1.25. + +**remove deprecated list-tag-history / tagHistory** + +| PR: https://pagure.io/koji/pull-request/2405 + +Final removal. + +**Remove deprecated host.getTask call** + +| PR: https://pagure.io/koji/pull-request/2406 + +Final removal. + + +Builder Changes +--------------- + +**builder: configurable TTL for buildroots** + +| PR: https://pagure.io/koji/pull-request/2485 + +Buildroots were cleaned from builder after hard-coded amount of time. Basic +cleanup was done after two minutes and completely removed after a day. Now both +options are configurable as ``buildroot_basic_cleanup_delay`` and +``buildroot_final_cleanup_delay``. + +**livemedia-creator: pass --nomacboot on non-x86_64** + +| PR: https://pagure.io/koji/pull-request/2373 + +Additional option was needed for booting on non-x86_64 archs. + +**builder: handle btrfs subvolumes in ApplianceTask** + +| PR: https://pagure.io/koji/pull-request/2365 + +BTRFS needed special handling in ``ApplianceTask`` to work. + +**kojid: fix extra-boot-args option** + +| PR: https://pagure.io/koji/pull-request/2452 + +Bug which prevented proper usage of ``bootloader --append`` in kickstarts. + +**kojid: waitrepo on deleted tag** + +| PR: https://pagure.io/koji/pull-request/2417 + +If tag was deleted during ``waitrepo`` task, it waited until long timeout. Now +it can fail immediately. + +System Changes +-------------- + +**dropping python 2.6 / RHEL6 / yum support** + +| PR: https://pagure.io/koji/pull-request/2490 + +One of the most important changes is dropping support for older python. We still +support python 2.7 for builder (other components are python 3 only). It +effectively means ending support for RHEL/CentOS 6 builders. In the same moment +we are dropping yum support (it was used only with dist-repos) as RHEL7 and +newer have full dnf stack. + +**report versions of components** + +| PR: https://pagure.io/koji/pull-request/2438 + +There is a new API call ``getVersion`` (don't confuse with ``getAPIVersion``) +which returns version of hub being connected to. Similarly, basic library +provides ``koji.__version__`` field. + +Plugins +------- + +**proton: persistent message queue** + +| PR: https://pagure.io/koji/pull-request/2441 + +As qpid (or other amqps broker) can be unreachable for longer periods of time +we've implemented local db queue, so no messages are lost. This behaviour needs +to be turned on - check the documentation. + +Utilities Changes +----------------- + +Kojira +...... + +**parallel rmtree** + +| PR: https://pagure.io/koji/pull-request/2443 + +Deleting old repos is now done in parallel. + + +Documentation +------------- + +**PostgreSQL requirements for partitioning** + +| PR: https://pagure.io/koji/pull-request/2508 + + +**release process** + +| PR: https://pagure.io/koji/pull-request/2462 + + +**more info about permission system** + +| PR: https://pagure.io/koji/pull-request/2415 + + +**setting rpm macros for build tags** + +| PR: https://pagure.io/koji/pull-request/2410 + +**livecd/livemedia updates** + +| PR: https://pagure.io/koji/pull-request/2500 diff --git a/koji.spec b/koji.spec index d65ef6d..78d1863 100644 --- a/koji.spec +++ b/koji.spec @@ -78,7 +78,7 @@ %define release %{baserelease} %endif Name: koji -Version: 1.22.1 +Version: 1.23.0 Release: %{release}%{?dist} License: LGPLv2 and GPLv2+ # the included arch lib from yum's rpmUtils is GPLv2+ diff --git a/koji/_version.py b/koji/_version.py index b7888a6..a2e4fe1 100644 --- a/koji/_version.py +++ b/koji/_version.py @@ -1,2 +1,2 @@ -__version_info__ = (1, 22, 1) +__version_info__ = (1, 23, 0) __version__ = '.'.join([str(x) for x in __version_info__]) From 1a853e4172bc69738c65310e271bc55f3182feb4 Mon Sep 17 00:00:00 2001 From: Mike McLean Date: Oct 22 2020 07:44:47 +0000 Subject: [PATCH 2/4] update release notes --- diff --git a/docs/source/plugins.rst b/docs/source/plugins.rst index 2299af8..a9baebb 100644 --- a/docs/source/plugins.rst +++ b/docs/source/plugins.rst @@ -159,6 +159,8 @@ And in scripts, you can use following calls: ks.gssapi_login() ks.createSideTag('f30-build') +.. _protonmsg-config: + Proton messaging ================ diff --git a/docs/source/release_notes/release_notes_1.23.rst b/docs/source/release_notes/release_notes_1.23.rst index 043d0e6..7d6e81e 100644 --- a/docs/source/release_notes/release_notes_1.23.rst +++ b/docs/source/release_notes/release_notes_1.23.rst @@ -33,7 +33,7 @@ content in target tag. Now we are failing on any error. | PR: https://pagure.io/koji/pull-request/2461 | PR: https://pagure.io/koji/pull-request/2471 -In the last version we've introduced unified ``download_file`` method which also +In the previous version we introduced unified ``download_file`` method which also checks downloaded content. There are minor updates improving this function especially in reaction to errors. @@ -56,12 +56,12 @@ relevant logs. Fix of the bug which missed plugins in /usr/lib64 prefix. -**clone-tag --config clones also extra info** +**clone-tag --config also clones extra info** | PR: https://pagure.io/koji/pull-request/2472 Formerly cloning skipped extra values. Nevertheless, extra values are becoming -important part of config, so from now we are cloning also those. +an important part of config, so from now we are cloning them. Library Changes --------------- @@ -88,13 +88,21 @@ Case of signature hashes is properly ignored. It is probably mostly used by kojira, but it can be of some interest to automation scripts. +Because of this, kojira will now correctly regenerate repos when tag extra +values change. + **getAverageBuildDuration sliding window** | PR: https://pagure.io/koji/pull-request/2421 -``getAverageBuildDuration`` returned average for all package builds. Old data -could be irrelevant to new version of packages, so age in months can now be -specified to limit data from which average is computed. +The ``getAverageBuildDuration`` hub call returned an average for all builds for +the given package. +However, old data could be irrelevant to new versions of packages. +Now the call offers an ``age`` option to limit the query (specified as number +of months). + +The koji builder daemon now uses this option with a value of 6 months when +adjusting the weight of ``buildArch`` tasks. **getBuildConfig returns inheritance history** @@ -107,17 +115,17 @@ This behaviour is meant to be used with next change. | PR: https://pagure.io/koji/pull-request/2495 -Inherited tag extra fields could have been overriden but not removed. Now it can -be done via CLIs ``edit-tag`` or ``editTag2`` respectively which has new +Inherited tag extra fields could have been overridden but not removed. Now it can +be done via CLIs ``edit-tag`` or ``editTag2`` respectively which has a new ``block_extra`` option. **deprecate getGlobalInheritance** | PR: https://pagure.io/koji/pull-request/2407 -It should be replaced by ``readFullInheritance`` which is more general and -properly reflects various inheritance options. Call will be completely removed -in koji 1.25. +This call was never used in Koji. +Clients should instead use the ``readFullInheritance`` call. +The ``getGlobalInheritance`` call will be completely removed in Koji 1.25. **remove deprecated list-tag-history / tagHistory** @@ -139,10 +147,17 @@ Builder Changes | PR: https://pagure.io/koji/pull-request/2485 -Buildroots were cleaned from builder after hard-coded amount of time. Basic -cleanup was done after two minutes and completely removed after a day. Now both -options are configurable as ``buildroot_basic_cleanup_delay`` and -``buildroot_final_cleanup_delay``. +Previously these times were hard coded. +The ``buildroot_basic_cleanup_delay`` setting controls how long the builder will +wait before basic cleanup of the buildroot (removing most content but leaving +the directory). The default value is two minutes. +The ``buildroot_final_cleanup_delay`` setting controls how long the build will +wait before final cleanup of the buildroot (removing the rest). +The default value is one day. +Both values are specified in seconds. + +For historical context on why there are two separate delays, see +`this bug `. **livemedia-creator: pass --nomacboot on non-x86_64** @@ -166,8 +181,9 @@ Bug which prevented proper usage of ``bootloader --append`` in kickstarts. | PR: https://pagure.io/koji/pull-request/2417 -If tag was deleted during ``waitrepo`` task, it waited until long timeout. Now -it can fail immediately. +Previously, if a tag was deleted while a ``waitrepo`` task was watching it, the +task would not notice and wait until the timeout expired. +Now it will fail when it detects that the tag has been deleted. System Changes -------------- @@ -176,19 +192,23 @@ System Changes | PR: https://pagure.io/koji/pull-request/2490 -One of the most important changes is dropping support for older python. We still -support python 2.7 for builder (other components are python 3 only). It -effectively means ending support for RHEL/CentOS 6 builders. In the same moment -we are dropping yum support (it was used only with dist-repos) as RHEL7 and + +One of the most significant changes in this release is dropping support for +older python versions. +Koji no longer supports python 2.6, and only supports python 2.7 for the +builder and cli. + +This effectively means ending support for RHEL/CentOS 6 builders. +We are dropping yum support (it was used only with dist-repos) as RHEL7 and newer have full dnf stack. **report versions of components** | PR: https://pagure.io/koji/pull-request/2438 -There is a new API call ``getVersion`` (don't confuse with ``getAPIVersion``) -which returns version of hub being connected to. Similarly, basic library -provides ``koji.__version__`` field. +There is a new hub API call named ``getVersion`` (don't confuse with +``getAPIVersion``) which returns the version of Koji that the hub is running. +Similarly, the ``koji`` library provides its version in ``koji.__version__``. Plugins ------- @@ -197,9 +217,10 @@ Plugins | PR: https://pagure.io/koji/pull-request/2441 -As qpid (or other amqps broker) can be unreachable for longer periods of time -we've implemented local db queue, so no messages are lost. This behaviour needs -to be turned on - check the documentation. +As qpid (or other amqps broker) can be unreachable for long periods of time +we've implemented a fallback queue in the database to avoid lost messages. +This behaviour needs to be enabled - see +:ref:`the documentation `. Utilities Changes ----------------- From 70886db32418aa86141f1b58d9420f515be887ee Mon Sep 17 00:00:00 2001 From: Mike McLean Date: Oct 22 2020 07:44:51 +0000 Subject: [PATCH 3/4] update migration notes --- diff --git a/docs/source/migrations/migrating_to_1.23.rst b/docs/source/migrations/migrating_to_1.23.rst index d837f14..67a4fcd 100644 --- a/docs/source/migrations/migrating_to_1.23.rst +++ b/docs/source/migrations/migrating_to_1.23.rst @@ -6,9 +6,18 @@ You should consider the following changes when migrating to 1.23: DB Updates ---------- -There is one minor schema change in this release. We've dropped NOT NULL -restriction on tag_extra value column to allow block these values in hierarchy -(`PR#2495 `_). +This release includes some minor schema changes. + +We've dropped the ``NOT NULL`` restriction on the ``value`` column of the +``tag_extra`` table as part of the changes to allow blocking these values in +the inheritance (see `PR#2495 `_). + +We've also added a new ``proton_queue`` table that is used by the ``protonmsg`` +plugin when it is configured to queue messages +(see `PR#2441 `_) + +Lastly, we've added a new index for the ``task`` table to improve performance +(see `PR#2419 `_) As in previous releases, we provide a migration script that updates the database. From 6543a20deaa5f1de8795d640d7a401884b08dfc6 Mon Sep 17 00:00:00 2001 From: Mike McLean Date: Oct 22 2020 07:44:56 +0000 Subject: [PATCH 4/4] formatting --- diff --git a/docs/source/release_notes/release_notes_1.22.1.rst b/docs/source/release_notes/release_notes_1.22.1.rst index 8cf5762..abcecd3 100644 --- a/docs/source/release_notes/release_notes_1.22.1.rst +++ b/docs/source/release_notes/release_notes_1.22.1.rst @@ -32,7 +32,7 @@ Hub Changes In some cases (especially ``hastag`` policy in combination with ``clone-tag``) there were allocated db cursors but not freed. -**stricter config file permissions +**stricter config file permissions** | PR: https://pagure.io/koji/pull-request/2474