Isn't this still needed (fixed variant msg['id'])? Otherwise we will add older messages after some which could have been inserted meanwhile and replay order will be wrong. (It can occur already as other thread can be slower e.g. by timeouting on some URL).

Otherwise :thumbsup:

Dropping the bit handling the db id field was explained in the commit message for that change.

This is a relic of an earlier version of the db work.
At this point in the code, there is never an id field.
We don't re-store messages from the db. We only delete them
after confirmed send.

At first I was looking at this part because it looked like a potential KeyError (there is never a db_id field), but then I realized the above.

@tkopecek discussed this earlier today and we both agreed this part was fine.

Testing Notes

Issue #2841 was discovered while testing on a stage system with protonmsg enabled and configured. The plugin was connecting to our stage amqps broker.

In order to test the db queue code path, I used iptables to temporarily block connections to our broker hosts. E.g.

# for ip in $BROKER_IPS; do  iptables -A INPUT -s "$ip" -j DROP ; done
# iptables -L INPUT

A short time later, the tracebacks from #2841 started to appear.

Once I'd explored the problem, I undid the ip block

# for ip in $BROKER_IPS; do  iptables -D INPUT -s "$ip" -j DROP ; done
# iptables -L INPUT

Removing the block doesn't solve the issue. Once there are entries in the proton_queue table, the plugin will error on every call that manages to get the table lock, losing new messages in the process.

I've found a separate issue in testing. The messages are never deleted from the db.

New issue is #2846
Will append a fix shortly

I ran these current changes (including the last) on the same system where I discovered the bug. The behavior was as expected. With ampq connections blocked, the db queue started to fill up with message. When the connections were unblocked, the queued messages were sent (once) and removed from the db.

ouch, :thumbsup:

:thumbsup:

Commit 8b07fcf5 fixes this pull-request