#12644 Mailing list moderation consistently fails
Closed: Insufficient data by zlopez. Opened by rcritten.

Describe what you would like us to do:

Mailing list moderation of the freeipa-users list, https://lists.fedoraproject.org/admin/lists/freeipa-users.lists.fedorahosted.org/held_messages , consistently fails with a 503 Error, Backend fetch.

Most moderation involves discarding spam and even on failure somtimes the number to moderate goes down even with a failure, but I'm concerned about losing messages to the list that need to be approved.

This could be related to the datacenter move and will clear itself up.

When do you need this to be done by? (YYYY/MM/DD)


It is a slow boil but the longer it takes to address the worse it will become. It will just happen slowly assuming the current rate of spam continues.


I was able to open the URL without error, could you collaborate what exactly doesn't work?

Metadata Update from @zlopez:
- Issue priority set to: Waiting on Reporter (was: Needs Review)
- Issue tagged with: lists

The URL works but if you try to moderate, say select the first 10 and discard it times out after some time (one minute?) and nothing is apparently done.

I tried approving one of the held messages and I saw it show on the list but it still appeared in the moderation list. Then it went away, I think? I'm worried that messages are going to disappear.

I can confirm that, I got 503 when trying to reject one of the spams. I will need to investigate the mailman logs to see what is happening.

Metadata Update from @zlopez:
- Issue assigned to zlopez
- Issue priority set to: Waiting on Assignee (was: Waiting on Reporter)
- Issue tagged with: Needs investigation, medium-gain, ops

When I tried to debug the issue I restarted the mailman3 service and the moderation works now. Not sure what happened. @rcritten Could you check if it's working for you? Feel free to close this ticket and in case it stops working again reopen it.

I tried and got a 503.

You are right, I tried it again now and it throws 503, but I was able to reject 3 mails during testing. I will try to investigate more.

It's strange, I enabled the debug log again and the moderation works. I disabled it and it still works. I assume it will stop working after some time, but it's really hard to debug

I'm not able to reproduce it anymore, I rejected almost all the spam e-mails in held messages and still didn't trigger the issue :/

It's working for me now as well. Ok I guess we can close this? My list gets a fair amount of spam. If it gets stuck again I'll re-open.

Let me close it for now as insufficient data and feel free to re-open when the issue happens again.

Metadata Update from @zlopez:
- Issue close_status updated to: Insufficient data
- Issue status updated to: Closed (was: Open)

FWIW now the entire service is down.

$ curl https://lists.fedoraproject.org/admin/lists/freeipa-users.lists.fedorahosted.org/held_messages
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>503 Service Unavailable</title>
</head><body>
<h1>Service Unavailable</h1>
<p>The server is temporarily unable to service your
request due to maintenance downtime or capacity
problems. Please try again later.</p>
</body></html>

You probably hit https://pagure.io/fedora-infrastructure/issue/12651, which takes some time to restart.

Metadata