Our EPEL repo files contain the following for each repo:
metalink=https://mirrors.fedoraproject.org/metalink?repo=testing-source-epel7&arch=$basearch&infra=$infra&content=$contentdir
Starting earlier today, yum installs began to fail for specific hosts. Using curl, we tracked this down to a stale repomd.xml file returned by the URL above. After inspecting the headers, all the failures have X-Fedora-ProxyServer: proxy30.fedoraproject.org in common. The nodes that are working have a different proxy host listed here.
X-Fedora-ProxyServer: proxy30.fedoraproject.org
Failure: Note the old epoch timestamp.
[root@host-4 ~]# curl -vvv "https://mirrors.fedoraproject.org/metalink?repo=epel-7&arch=x86_64&infra=container&content=centos" 2>&1 | grep -e timestamp -e X-Fedora-ProxyServer < X-Fedora-ProxyServer: proxy30.fedoraproject.org <mm0:timestamp>1579999421</mm0:timestamp> <mm0:timestamp>1579887751</mm0:timestamp>
Success: Note the recent timestamp.
[root@host-3 ~]# curl -vvv "https://mirrors.fedoraproject.org/metalink?repo=epel-7&arch=x86_64&infra=container&content=centos" 2>&1 | grep -e timestamp -e X-Fedora-ProxyServer < X-Fedora-ProxyServer: proxy09.fedoraproject.org <mm0:timestamp>1580775676</mm0:timestamp> <mm0:timestamp>1580689019</mm0:timestamp> <mm0:timestamp>1580608900</mm0:timestamp>
Please see additional discussion in the fedora-admins IRC channel.
ASAP (2020/2/5), as our images have this URL baked in, so it is impacting quite a few automated tests.
Fixed. Thanks for letting us know.
proxy30 wasn't correctly getting the mirrorlist updates. The mirrormanager backend needed an ansible playbook run to add it to the list of proxies to update.
Let us know if you still see any issues with it.
Metadata Update from @kevin: - Issue close_status updated to: Fixed - Issue status updated to: Closed (was: Open)