fedora-infrastructure

#12909 Multiple pods crashlooping in staging OpenShift
Closed: Migrated to Fedora Forge by ryanlerch. Opened by zlopez.

Closed: Migrated to Fedora Forge
Reopen Issue

Describe what you would like us to do:

Today with the log01 running out of space I noticed that there is a really big log file for staging OpenShift worker and found out that there are 6 pods crashlooping.

Here is the list of the pods:

  • compose-tracker
2025-11-11 14:04:16,169 INFO fedora_messaging.cli - Starting consumer with compose_tracker:Consumer callback
2025-11-11 14:04:16,169 INFO compose_tracker - Using detected token to talk to pagure.
2025-11-11 14:04:16,169 INFO compose_tracker - Targeting repo releng/failed-composes on https://stg.pagure.io/
2025-11-11 14:04:18,171 ERROR ogr.services.pagure.service - {'error': 'Invalid or expired token. Please visit https://stg.pagure.io/settings#nav-api-tab to get or renew your API token.', 'error_code': 'EINVALIDTOK', 'errors': 'Expired token'}
Traceback (most recent call last):
  File "/usr/bin/fedora-messaging", line 33, in <module>
    sys.exit(load_entry_point('fedora-messaging==3.4.1', 'console_scripts', 'fedora-messaging')())
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/fedora_messaging/cli.py", line 133, in consume
    _consume(exchange, queue_name, routing_key, callback, app_name)
  File "/usr/lib/python3.11/site-packages/fedora_messaging/cli.py", line 169, in _consume
    deferred_consumers = api.twisted_consume(
                         ^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/fedora_messaging/api.py", line 151, in twisted_consume
    callback = _check_callback(callback)
               ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/fedora_messaging/api.py", line 75, in _check_callback
    callback_object = callback()
                      ^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/compose_tracker.py", line 138, in __init__
    self.gitproject = gitservice.get_project(
                      ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/ogr/abstract.py", line 116, in wrapper
    __check_for_internal_failure(ex)
  File "/usr/lib/python3.11/site-packages/ogr/abstract.py", line 58, in __check_for_internal_failure
    raise ex
  File "/usr/lib/python3.11/site-packages/ogr/abstract.py", line 106, in wrapper
    return function(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/ogr/services/pagure/service.py", line 100, in get_project
    username=self.user.get_username(),
             ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/ogr/abstract.py", line 116, in wrapper
    __check_for_internal_failure(ex)
  File "/usr/lib/python3.11/site-packages/ogr/abstract.py", line 58, in __check_for_internal_failure
    raise ex
  File "/usr/lib/python3.11/site-packages/ogr/abstract.py", line 106, in wrapper
    return function(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/ogr/services/pagure/user.py", line 23, in get_username
    return_value = self.service.call_api(url=request_url, method="POST", data={})
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/ogr/abstract.py", line 116, in wrapper
    __check_for_internal_failure(ex)
  File "/usr/lib/python3.11/site-packages/ogr/abstract.py", line 58, in __check_for_internal_failure
    raise ex
  File "/usr/lib/python3.11/site-packages/ogr/abstract.py", line 106, in wrapper
    return function(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/ogr/services/pagure/service.py", line 174, in call_api
    raise PagureAPIException(
ogr.exceptions.PagureAPIException: Pagure API returned an error when calling 'https://stg.pagure.io//api/0/-/whoami': Invalid or expired token. Please visit https://stg.pagure.io/settings#nav-api-tab to get or renew your API token. - Expired token
  • flatpak-indexer-quay-io
---> Running application from script (app.sh) ...
Traceback (most recent call last):
  File "/opt/app-root/lib64/python3.9/site-packages/flatpak_indexer/fedora_monitor.py", line 332, in _run
    self._wait_for_messages()
  File "/opt/app-root/lib64/python3.9/site-packages/flatpak_indexer/fedora_monitor.py", line 252, in _wait_for_messages
    queue_name_raw = self.thread_redis_client.get('fedora-messaging-queue')
  File "/opt/app-root/lib64/python3.9/site-packages/redis/commands/core.py", line 1834, in get
    return self.execute_command("GET", name, keys=[name])
  File "/opt/app-root/lib64/python3.9/site-packages/redis/client.py", line 657, in execute_command
    return self._execute_command(*args, **options)
  File "/opt/app-root/lib64/python3.9/site-packages/redis/client.py", line 663, in _execute_command
    conn = self.connection or pool.get_connection()
  File "/opt/app-root/lib64/python3.9/site-packages/redis/utils.py", line 196, in wrapper
    return func(*args, **kwargs)
  File "/opt/app-root/lib64/python3.9/site-packages/redis/connection.py", line 2601, in get_connection
    connection.connect()
  File "/opt/app-root/lib64/python3.9/site-packages/redis/connection.py", line 846, in connect
    self.connect_check_health(check_health=True)
  File "/opt/app-root/lib64/python3.9/site-packages/redis/connection.py", line 869, in connect_check_health
    self.on_connect_check_health(check_health=check_health)
  File "/opt/app-root/lib64/python3.9/site-packages/redis/connection.py", line 941, in on_connect_check_health
    auth_response = self.read_response()
  File "/opt/app-root/lib64/python3.9/site-packages/redis/connection.py", line 1133, in read_response
    response = self._parser.read_response(disable_decoding=disable_decoding)
  File "/opt/app-root/lib64/python3.9/site-packages/redis/_parsers/resp2.py", line 15, in read_response
    result = self._read_response(disable_decoding=disable_decoding)
  File "/opt/app-root/lib64/python3.9/site-packages/redis/_parsers/resp2.py", line 38, in _read_response
    raise error
redis.exceptions.AuthenticationError: AUTH <password> called without any password configured for the default user. Are you sure your configuration is correct?
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
  File "/opt/app-root/bin/flatpak-indexer", line 8, in <module>
    sys.exit(cli())
  File "/opt/app-root/lib64/python3.9/site-packages/click/core.py", line 1161, in __call__
    return self.main(*args, **kwargs)
  File "/opt/app-root/lib64/python3.9/site-packages/click/core.py", line 1082, in main
    rv = self.invoke(ctx)
  File "/opt/app-root/lib64/python3.9/site-packages/click/core.py", line 1697, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/opt/app-root/lib64/python3.9/site-packages/click/core.py", line 1443, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/opt/app-root/lib64/python3.9/site-packages/click/core.py", line 788, in invoke
    return __callback(*args, **kwargs)
  File "/opt/app-root/lib64/python3.9/site-packages/click/decorators.py", line 33, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/opt/app-root/lib64/python3.9/site-packages/flatpak_indexer/cli.py", line 47, in daemon
    updater.start()
  File "/opt/app-root/lib64/python3.9/site-packages/flatpak_indexer/datasource/fedora/updater.py", line 70, in start
    self.change_monitor.start()
  File "/opt/app-root/lib64/python3.9/site-packages/flatpak_indexer/fedora_monitor.py", line 78, in start
    self._maybe_reraise_failure("Failed to start connection to fedora-messaging")
  File "/opt/app-root/lib64/python3.9/site-packages/flatpak_indexer/fedora_monitor.py", line 180, in _maybe_reraise_failure
    raise RuntimeError(msg) from self.failure
RuntimeError: Failed to start connection to fedora-messaging
  • flatpak-indexer-registry-fedora
---> Running application from script (app.sh) ...
Traceback (most recent call last):
  File "/opt/app-root/lib64/python3.9/site-packages/flatpak_indexer/fedora_monitor.py", line 332, in _run
    self._wait_for_messages()
  File "/opt/app-root/lib64/python3.9/site-packages/flatpak_indexer/fedora_monitor.py", line 252, in _wait_for_messages
    queue_name_raw = self.thread_redis_client.get('fedora-messaging-queue')
  File "/opt/app-root/lib64/python3.9/site-packages/redis/commands/core.py", line 1834, in get
    return self.execute_command("GET", name, keys=[name])
  File "/opt/app-root/lib64/python3.9/site-packages/redis/client.py", line 657, in execute_command
    return self._execute_command(*args, **options)
  File "/opt/app-root/lib64/python3.9/site-packages/redis/client.py", line 663, in _execute_command
    conn = self.connection or pool.get_connection()
  File "/opt/app-root/lib64/python3.9/site-packages/redis/utils.py", line 196, in wrapper
    return func(*args, **kwargs)
  File "/opt/app-root/lib64/python3.9/site-packages/redis/connection.py", line 2601, in get_connection
    connection.connect()
  File "/opt/app-root/lib64/python3.9/site-packages/redis/connection.py", line 846, in connect
    self.connect_check_health(check_health=True)
  File "/opt/app-root/lib64/python3.9/site-packages/redis/connection.py", line 869, in connect_check_health
    self.on_connect_check_health(check_health=check_health)
  File "/opt/app-root/lib64/python3.9/site-packages/redis/connection.py", line 941, in on_connect_check_health
    auth_response = self.read_response()
  File "/opt/app-root/lib64/python3.9/site-packages/redis/connection.py", line 1133, in read_response
    response = self._parser.read_response(disable_decoding=disable_decoding)
  File "/opt/app-root/lib64/python3.9/site-packages/redis/_parsers/resp2.py", line 15, in read_response
    result = self._read_response(disable_decoding=disable_decoding)
  File "/opt/app-root/lib64/python3.9/site-packages/redis/_parsers/resp2.py", line 38, in _read_response
    raise error
redis.exceptions.AuthenticationError: AUTH <password> called without any password configured for the default user. Are you sure your configuration is correct?
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
  File "/opt/app-root/bin/flatpak-indexer", line 8, in <module>
    sys.exit(cli())
  File "/opt/app-root/lib64/python3.9/site-packages/click/core.py", line 1161, in __call__
    return self.main(*args, **kwargs)
  File "/opt/app-root/lib64/python3.9/site-packages/click/core.py", line 1082, in main
    rv = self.invoke(ctx)
  File "/opt/app-root/lib64/python3.9/site-packages/click/core.py", line 1697, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/opt/app-root/lib64/python3.9/site-packages/click/core.py", line 1443, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/opt/app-root/lib64/python3.9/site-packages/click/core.py", line 788, in invoke
    return __callback(*args, **kwargs)
  File "/opt/app-root/lib64/python3.9/site-packages/click/decorators.py", line 33, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/opt/app-root/lib64/python3.9/site-packages/flatpak_indexer/cli.py", line 47, in daemon
    updater.start()
  File "/opt/app-root/lib64/python3.9/site-packages/flatpak_indexer/datasource/fedora/updater.py", line 70, in start
    self.change_monitor.start()
  File "/opt/app-root/lib64/python3.9/site-packages/flatpak_indexer/fedora_monitor.py", line 78, in start
    self._maybe_reraise_failure("Failed to start connection to fedora-messaging")
  File "/opt/app-root/lib64/python3.9/site-packages/flatpak_indexer/fedora_monitor.py", line 180, in _maybe_reraise_failure
    raise RuntimeError(msg) from self.failure
RuntimeError: Failed to start connection to fedora-messaging
  • resultsdb-ci-listener
[fedora_messaging.cli INFO] Starting consumer with resultsdb_listener.consumer:Consumer callback
[fedora_messaging.twisted.service INFO] Authenticating with server using x509 (certfile: /etc/pki/rabbitmq/crt/resultsdb-ci-listener.crt, keyfile: /etc/pki/rabbitmq/key/resultsdb-ci-listener.key)
[fedora_messaging.cli ERROR] The TCP connection appears to have started, but the TLS or AMQP handshake with the broker failed; check your connection and authentication parameters and ensure your user has permission to access the vhost
  • koschei - watcher
/usr/share/koschei/koschei/models.py:1311: SAWarning: relationship 'CoprRebuild.request' will copy column copr_rebuild_request.id to column copr_rebuild.request_id, which conflicts with relationship(s): 'CoprRebuildRequest.rebuilds' (copies copr_rebuild_request.id to copr_rebuild.request_id). If this is not the intention, consider if these relationships should be linked with back_populates, or if viewonly=True should be applied to one or more if they are read-only. For the less common case that foreign key constraints are partially overlapping, the orm.foreign() annotation can be used to isolate the columns that should be written towards.   To silence this warning, add the parameter 'overlaps="rebuilds"' to the 'CoprRebuild.request' relationship. (Background on this error at: https://sqlalche.me/e/14/qzyx)
  CoprRebuild.request = relationship(CoprRebuildRequest)
watcher started
Log opened.
Loading configuration from /etc/koschei/fedora-messaging.toml
Authenticating with server using x509 (certfile: /etc/koschei/rabbitmq-client.crt, keyfile: /etc/koschei/rabbitmq-client.key)
Starting factory FedoraMessagingFactoryV2(parameters=<URLParameters host=rabbitmq.stg.fedoraproject.org port=5671 virtual_host=/pubsub ssl=True>, confirms=True)
AMQP stack terminated, failed to connect, or aborted: opened=False, error-arg=None; pending-error=ConnectionDone()
Connection setup terminated due to ConnectionDone()
<twisted.internet.tcp.Connector instance at 0x7f7949a30290 disconnected IPv4Address(type='TCP', host='rabbitmq.stg.fedoraproject.org', port=5671)> will retry in 2 seconds
Stopping factory FedoraMessagingFactoryV2(parameters=<URLParameters host=rabbitmq.stg.fedoraproject.org port=5671 virtual_host=/pubsub ssl=True>, confirms=True)
Consuming raised an unexpected error, please report a bug:
Traceback (most recent call last):
--- <exception caught here> ---
  File "/usr/lib/python3.12/site-packages/fedora_messaging/api.py", line 166, in _twisted_consume_wrapper
    consumers = yield twisted_consume(callback, bindings=bindings, queues=queues)
  File "/usr/lib/python3.12/site-packages/fedora_messaging/twisted/factory.py", line 291, in consume
    protocol = yield self.when_connected()
  File "/usr/lib/python3.12/site-packages/fedora_messaging/twisted/factory.py", line 203, in when_connected
    yield self._client_deferred
fedora_messaging.exceptions.ConnectionException: 
Service watcher crashed.
Traceback (most recent call last):
  File "/usr/share/koschei/koschei/backend/main.py", line 58, in main
    svc(backend.KoscheiBackendSession()).run_service()
  File "/usr/share/koschei/koschei/backend/service.py", line 100, in run_service
    self.main()
  File "/usr/share/koschei/koschei/plugins/fedmsg_plugin/backend/services/watcher.py", line 72, in main
    fedmsg.consume(callback)
  File "/usr/lib/python3.12/site-packages/fedora_messaging/api.py", line 235, in consume
    eventual_result.wait(timeout=2**31)
  File "/usr/lib/python3.12/site-packages/crochet/_eventloop.py", line 196, in wait
    result.raiseException()
  File "/usr/lib/python3.12/site-packages/twisted/python/failure.py", line 505, in raiseException
    raise self.value.with_traceback(self.tb)
fedora_messaging.exceptions.ConnectionException
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/usr/share/koschei/koschei/backend/main.py", line 67, in <module>
    main()
  File "/usr/share/koschei/koschei/backend/main.py", line 58, in main
    svc(backend.KoscheiBackendSession()).run_service()
  File "/usr/share/koschei/koschei/backend/service.py", line 100, in run_service
    self.main()
  File "/usr/share/koschei/koschei/plugins/fedmsg_plugin/backend/services/watcher.py", line 72, in main
    fedmsg.consume(callback)
  File "/usr/lib/python3.12/site-packages/fedora_messaging/api.py", line 235, in consume
    eventual_result.wait(timeout=2**31)
  File "/usr/lib/python3.12/site-packages/crochet/_eventloop.py", line 196, in wait
    result.raiseException()
  File "/usr/lib/python3.12/site-packages/twisted/python/failure.py", line 505, in raiseException
    raise self.value.with_traceback(self.tb)
fedora_messaging.exceptions.ConnectionException

I scaled them all to 0, but if anybody has spare time to look at them, please do.

When do you need this to be done by? (YYYY/MM/DD)

Not urgent, for now they are disabled

Metadata Update from @zlopez:
- Issue tagged with: sprint-0

Metadata Update from @zlopez:
- Issue assigned to zlopez

@yselkowitz - for the flatpak indexer ones

perhaps @lenkaseg might look at the others?

I resolved the koschei watcher pod, the problem there was that the user and queue were missing for fedora messaging. So after some debugging and then running the playbook I was able to get it up again.

resultsdb-ci-listener had expired fedora-messaging certificate. Fixed that as well.

I checked the flatpak-indexer as well, but I don't really understand what is happening there. It seems like the redis is not reachable or the password is not correctly read. I will try to check it out more tomorrow.

Fixed compose-tracker. It was expired pagure token, so I updated it with new expiration date.

I looked again at the flatpak-indexer, the redis was still running on some really old build and when I tried to rebuild it to latest staging it started failing. I also noticed that the Dockerfile on staging branch is 3 years older than the one in main branch, so I tried to build it from main, but it started failing as well although with different error.

I don't think I will be able to resolve this by just changing configuration in OpenShift, this will need changes upstream. Good first step would be to update staging branch to latest main. I will probably do this in my own fork and then create PR to @yselkowitz repository.

After some discussion with @yselkowitz I decided to work with upstream repository instead. I have my own fork of the upstream and trying to get the deployment in staging fixed. Today I got Redis (Valkey) running in staging, although I needed to backup the old database as the format wasn't compatible with Valkey (I assume the flatpak indexer will recreate the redis database anyway).

The rest of the deployment will take some time to figure out as the upstream has a lot of changes that are not reflected in current OpenShift deployment.

I got it finally running. All the flatpak-indexer pods are now working. The changes I did are in following PRs:

  • https://pagure.io/fedora-infra/ansible/pull-request/3028
  • https://pagure.io/fedora-infra/ansible/pull-request/3027
  • https://pagure.io/fedora-infra/ansible/pull-request/3030
  • https://github.com/owtaylor/flatpak-indexer/pull/28

Here is what I have to do:

  • Replace redis with valkey
  • Create a backup of database so valkey can create it's own (latest redis database is not compatible with valkey and flatpak-indexer should be able to regenerate the database from scratch again)
  • Reflect the latest changes in upstream in the deployment
  • Switch deployment to owtaylor repository

Now I should deploy these changes in production. But let's wait with this till Monday.

So which ones are still to be looked at here?

@kevin None, I just trying to fix some permission issue in flatpak-indexer pods. All the original problems are already fixed.

@kevin You said it should be possible to mount the PV on sundries01. I found there are already few mounted, but I'm not sure how to mount staging PV (I'm missing the second part of server domain for staging). All those that are already mounted are production ones.

Something like:

mount ntap-rdu3-c02-fedora01-nfs01a.neta-002.prod.rdu2.dc.redhat.com:/languages-storage /mnt/
do whatever changes
umount /mnt

I am not sure why other openshift volumes are mounted there full time. A few are needed because we sync content out from them, but the rest only would be needed if we had to change file ownership or something, so IMHO they shouldn't be mounted all the time there.

I guess @abompard added them in a0d4c1f6df7 "for the next datacenter move", so alright...

I wonder what I'm doing wrong when trying to mount the volume.

[root@sundries01 ~][PROD-RDU3]# mount -v ntap-rdu3-c02-fedora01-nfs01a.neta-002.prod.rdu2.dc.redhat.com:/flatpak-indexer-storage-stg /mnt/
mount.nfs: timeout set for Wed Jan 14 14:19:59 2026
mount.nfs: trying text-based options 'vers=4.2,addr=10.16.162.11,clientaddr=10.16.163.81'
mount.nfs: mount(2): Permission denied
mount.nfs: trying text-based options 'vers=4,minorversion=1,addr=10.16.162.11,clientaddr=10.16.163.81'
mount.nfs: mount(2): Permission denied
mount.nfs: trying text-based options 'vers=4,addr=10.16.162.11,clientaddr=10.16.163.81'
mount.nfs: mount(2): Permission denied
mount.nfs: trying text-based options 'addr=10.16.162.11'
mount.nfs: prog 100003, trying vers=3, prot=6
mount.nfs: trying 10.16.162.11 prog 100003 vers 3 prot TCP port 2049
mount.nfs: prog 100005, trying vers=3, prot=17
mount.nfs: trying 10.16.162.11 prog 100005 vers 3 prot UDP port 635
mount.nfs: mount(2): Permission denied
mount.nfs: access denied by server while mounting ntap-rdu3-c02-fedora01-nfs01a.neta-002.prod.rdu2.dc.redhat.com:/flatpak-indexer-storage-stg

In openshift the volume name has -s in it, but in reality it has _s

so, you need to mount flatpak_indexer_storage_stg

also, since it's staging that needs to be on sundries01.stg :)

@kevin Something is still off :/

[root@sundries01 ~][STG]#  mount -v ntap-rdu3-c02-fedora01-nfs01a.neta-002.prod.rdu2.dc.redhat.com:/flatpak_indexer_storage_stg /mnt/
mount.nfs: timeout set for Fri Jan 16 10:02:37 2026
mount.nfs: trying text-based options 'vers=4.2,addr=10.16.162.11,clientaddr=10.16.166.33'
mount.nfs: mount(2): No such file or directory
mount.nfs: trying text-based options 'addr=10.16.162.11'
mount.nfs: prog 100003, trying vers=3, prot=6
mount.nfs: trying 10.16.162.11 prog 100003 vers 3 prot TCP port 2049
mount.nfs: prog 100005, trying vers=3, prot=17
mount.nfs: trying 10.16.162.11 prog 100005 vers 3 prot UDP port 635
mount.nfs: mount(2): No such file or directory
mount.nfs: mounting ntap-rdu3-c02-fedora01-nfs01a.neta-002.prod.rdu2.dc.redhat.com:/flatpak_indexer_storage_stg failed, reason given by server: No such file or directory

I checked the server and the name, both are correct. So I'm not sure what I'm missing.
Edited by zlopez

You want flatpak-indexer-storage-stg

The actual volume on the netapp is flatpak_indexer_storage_stg but openshift cannot handle _s in volume names, so it's setup to be mounted on the netapp as /vol/flatpak-indexer-storage-stg and thats how it's exported out to the clients. :)

It's mounted now...

This issue has been migrated to Fedora Forge:
https://forge.fedoraproject.org/infra/tickets/issues/12909

Please continue any further discussion there.

Metadata Update from @ryanlerch:
- Issue close_status updated to: Migrated to Fedora Forge
- Issue status updated to: Closed (was: Open)
Metadata

low-gain medium-trouble sprint-0

None
None
Normal
Static archive. pagure.io was sunset in 2026; this is a read-only snapshot.
Red Hat Logo

Fedora is sponsored by Red Hat. Learn more about the relationship between Red Hat and Fedora »

© Red Hat, Inc. and others.