fedora-infrastructure

#11116 FAS Mail is not working since 2023-01-30
Closed: Fixed by fantom. Opened by fantom.

Closed: Fixed
Reopen Issue

The mail redirection, the mail alias, "fas@fp.o" is not working at all since 2023-01-30 at 23:00:03 UTC.

Last mail delivery was 2023-01-30 at 23:00:03 UTC from bastion-iad01.fedoraproject.org IP 38.145.60.11 (about my fas@fp.o address).

To try to troubbleshoot the issue, I tried to send an email to my fas@fp.o from another mail provider. The mail got lost.

Severity: Urgent

Metadata Update from @zlopez:
- Issue tagged with: Needs investigation, medium-gain

The problem is that your domain is causing issues:

Feb  2 13:40:40 bastion01 postfix/error[564262]: B552430363BF: to=<casper@casperlefantom.net>, orig_to=<fantom@fedoraproject.org>, relay=none, delay=185321, delays=185321/0.62/0/0, dsn=4.4.3, status=deferred (delivery temporarily suspended: Host or domain name not found. Name service error for name=casperlefantom.net type=MX: Host not found, try again)
# host casperlefantom.net
Host casperlefantom.net not found: 2(SERVFAIL)

However going out of the dc, I can get to it

 host casperlefantom.net
casperlefantom.net has address 51.15.177.140
casperlefantom.net has IPv6 address 2001:bc8:3fec:500:7ea::
casperlefantom.net mail is handled by 40 mx2.casperlefantom.net.
casperlefantom.net mail is handled by 30 mx1.casperlefantom.net.

Something is off in DNS in the cage. I am not sure what yet. Doing a dig +trace I get a broken chain at

casperlefantom.net.     172800  IN      NS      nsd.casperlefantom.net.
casperlefantom.net.     172800  IN      NS      nsb.casperlefantom.net.
casperlefantom.net.     172800  IN      NS      nse.casperlefantom.net.
casperlefantom.net.     172800  IN      NS      nsf.casperlefantom.net.
casperlefantom.net.     172800  IN      NS      nsh.casperlefantom.net.
casperlefantom.net.     172800  IN      NS      nsa.casperlefantom.net.
casperlefantom.net.     86400   IN      DS      8589 7 2 F215C2B663D10A231A24845F6BF25F1FFED44C30BEAA4768FCB16D6D 49CB4EFA
casperlefantom.net.     86400   IN      DS      8589 7 1 81F4B2C00A5FDB3CA26E9C2D2D98D0F2158E1904
casperlefantom.net.     86400   IN      RRSIG   DS 8 2 86400 20230208070514 20230201055514 27254 net. RfJokMEqeit52Cns+1rEpCUSbnQwtVSWdCzt/NFkU3UadnXItJzj3Lck a2bOtfDQSmqNISYJUXTEo4aahbZnLCNrPWlWTLfu4r82n8Qz4xw7Vlhc BPL10HbMtn4qCBTxJGKM7kLdefoo8ZB+n5aagkjGVt6XFaSwOHk4nX+S YZA4I5GivzkWikplK4zVybfCGt+7lKhSEhu6A0cxWM6gCg==
couldn't get address for 'nsd.casperlefantom.net': not found
couldn't get address for 'nsb.casperlefantom.net': not found
couldn't get address for 'nse.casperlefantom.net': not found
couldn't get address for 'nsf.casperlefantom.net': not found
couldn't get address for 'nsh.casperlefantom.net': not found
couldn't get address for 'nsa.casperlefantom.net': not found
dig: couldn't get address for 'nsd.casperlefantom.net': no more

how it is working at

casperlefantom.net. 86400   IN  A   51.15.177.140
casperlefantom.net. 86400   IN  RRSIG   A 7 2 86400 20230223075830 20230121075830 21299 casperlefantom.net. S5rcHdfzyuUTIVSEl5En2qyAcqraEKaVq7pK6YyVRT+wpRQsaY3tTnD0 /VcXZtj4HbK/y1zH9el5V+UZF8nwItaX2rJo3yDux82WXVlpY/ldKzlt raCijLJwevjHVQvhWSADW22Po+stHvZtgNbbzNYID6r1DfULnzdApMkT 24fs4WjbreMb6HQyLeCdPrBsyqA4lQWMbg6/vsEoRIZtheQrxUJ8IT7H ubzatYhHaF0VAsqUDQcCkaHEKqFaRw3EukiRh6U3R1mAyJPLl9X7HWNf 19C/HviCLxOXQOkjNUHEJ8G9HOubq2NEVJBwxxkS5rnL/H7nE2tq91uf g+D7mQ==
casperlefantom.net. 86400   IN  NS  nsb.casperlefantom.net.
casperlefantom.net. 86400   IN  NS  nsh.casperlefantom.net.
casperlefantom.net. 86400   IN  NS  nsa.casperlefantom.net.
casperlefantom.net. 86400   IN  NS  nsf.casperlefantom.net.
casperlefantom.net. 86400   IN  NS  nsd.casperlefantom.net.
casperlefantom.net. 86400   IN  NS  nse.casperlefantom.net.
casperlefantom.net. 86400   IN  RRSIG   NS 7 2 86400 20230223075830 20230121075830 21299 casperlefantom.net. GBGnom68Cez9WFCd6l1ZiYTCa7r9NVk9ZTuQoBa8Hwvn6fp5fDuggXL4 WyPu5FwSitHMBn1UrcHhrh5DYNTm4Vc/nPxV1+g0+MXmdefu3O34rY8N hRPyerC/HZjReiLZNjTlBuJ2AxZepwwTeFjtmMgh43Ehm5XmU9A5Fkw/ zSSFV684pgrf86xvnGGUU/5kyyVs7ZnDwrFTTaozpdP/ZD2k9jTNZDXU dfUPhEMSPnAN6HYK+mOdQs30ol4J6QIo1Xi3pij2hafEkUhqHGSEOeHm Atllym/AKElUWkQH+vzVc33ze0L+QotvtUUTw0g1e+K+Do9Lq934VL50 sM9pQA==
Edited by smooge

OK I think I figured out the issue. A lot of DNS lookups are failing with errors about envelope routines::invalid digest:crypto/evp/pmeth_lib.c and similar. This is due to the fact that keys are using older DNSKEY which the server can't validate anymore (my guess is SHA1).

I am looking to see what can be done for this.

OK the issue is that various domains are using keys which can no longer be validated by the DEFAULT EL9 (and possibly Fedora) operating system. This caused the nameservers to start returning SERVFAIL for a long list of domains and their email to have issues and probably lost.

I just received about 50 emails at the same moment (including the message sent for test purpose). Thank you for the fix.

At the begining of DNSSEC, there was only RSASHA1-NSEC3-SHA1 available. But it seems to be fine for online validators:

https://dnssec-debugger.verisignlabs.com/casperlefantom.net

Yeah, I expect we will need to make it DEFAULT:SHA1 for various systems which deal with DNS regularly.
For reference, this is what was turned off in the current crypto policy which is in EL9 and probably in upcoming Fedora.

  • DH with parameters < 2048 bits
  • RSA with key size < 2048 bits
  • DSA (any size)
  • 3DES
  • RC4
  • FFDHE-1024
  • DHE-DSS
  • Camellia
  • SHA-1 in digital signatures and certificates
  • CBC mode ciphers

Metadata Update from @smooge:
- Issue assigned to smooge

Metadata Update from @smooge:
- Issue tagged with: dns

Metadata Update from @smooge:
- Issue priority set to: Waiting on Assignee (was: Needs Review)

OK to close this ticket? Kevin is going to fix it permanently with an ansible policy.

Everything is working now, yes :)

Metadata Update from @fantom:
- Issue close_status updated to: Fixed
- Issue status updated to: Closed (was: Open)
Metadata

Needs investigation dns medium-gain

None
None
Normal
Static archive. pagure.io was sunset in 2026; this is a read-only snapshot.
Red Hat Logo

Fedora is sponsored by Red Hat. Learn more about the relationship between Red Hat and Fedora »

© Red Hat, Inc. and others.