I found out this issue while I was testing MTS on Openshift stg. Following steps are what I did to encounter this issue:
durable = True
auto_delete = False
routing_keys
['org.fedoraproject.*.mbs.module.state.change', 'org.fedoraproject.*.buildsys.task.state.change']
org.fedoraproject.*.buildsys.task.state.change
durable = False
auto_delete = True
Can you please have a look in the server side? I'm not sure if the queue was deleted correctly. Queue name is mts.stg.
mts.stg
Thank you very much!
@abompard could you help @cqi with this?
Metadata Update from @pingou: - Issue assigned to abompard - Issue priority set to: Waiting on Assignee (was: Needs Review)
Hey @cqi. A few things here: - the queue was not deleted, you can do it in ansible but it's a on-off thing and it's easier to just ask infra to delete the queue. - queues can't be switched from durable to non-durable, so it's still the durable queue that is in place in the server - there is currently no way to remove bindings (subscriptions to a topic) in ansible, we have to do it manually on the server (but it's easy ;-) ) - on the server, I see your mts.stg queue subscribed to 3 topics: buildsys.task.state.change, mbs.build.state.change and mbs.module.state.change.
buildsys.task.state.change
mbs.build.state.change
mbs.module.state.change
I'm just going to delete the queue (and the bindings), and let you recreate them all as you want.
Metadata Update from @abompard: - Issue close_status updated to: Fixed - Issue status updated to: Closed (was: Open)
If it's not already there, should we include this in the SOP? (both the text you've put here as well as the command to clear things up?)
Thanks :)
mbs.build.state.change was set incorrectly. Thanks both of you @pingou and @abompard :)
@abompard
I rerun MTS in a new pod, and notice this log from logs:
2019-09-12 12:40:33,345- fedora_messaging._session - INFO - Channel <Channel number=1 CLOSED conn=<SelectConnection OPEN transport=<pika.adapters.utils.io_services_utils._AsyncSSLTransport object at 0x7fc3fbcd81d0> params=<URLParameters host=rabbitmq.stg.fedoraproject.org port=5671 virtual_host=/pubsub ssl=True>>> closed (404): NOT_FOUND - no queue 'mts.stg' in vhost '/pubsub'
Metadata Update from @cqi: - Issue status updated to: Open (was: Closed)
Fixed. Sorry.
Metadata Update from @cqi: - Issue close_status updated to: Fixed - Issue status updated to: Closed (was: Open)
@abompard Could you please have a look at the queue on the server again? What I see from logs is MTS connects to broker and queue mts.stg is bound to amq.topic with the org.fedoraproject.*.mbs.module.state.change key. But, it still can't receive any messages sent to the topic.
org.fedoraproject.*.mbs.module.state.change
I can get those messages with CLI fedora-messaging with the fedora.stg.toml config. I can't figure out at this moment why this happens. Thanks.
fedora-messaging
fedora.stg.toml
Could you please have a look at the queue on the server again?
I mean, is there any message in the queue, what are the messages content?
I see no messages in the queue, and only one binding: org.fedoraproject.*.mbs.module.state.change.
Please also note that the asterisk * does not match dots in AMQP. if you send a message to org.fedoraproject.dev.foobar.mbs.module.state.change it will not be routed to your queue. If you want to match dots, use the hash sign #.
*
org.fedoraproject.dev.foobar.mbs.module.state.change
#