fedora-infrastructure

#12964 git repository seems unhealthy
Closed: Fixed by kevin. Opened by mtasaka.

Closed: Fixed
Reopen Issue

As of 2025-12-07 00:00 JST(=UTC+9h), fedora git repository seems very unstable or unreachable.

For example, $ git pull takes very long time, or sometimes returns:

Error during lookup request: status: 503
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.

And this seems to be affecting bodhi build using rpkg, for example:
https://copr.fedorainfracloud.org/coprs/mtasaka/rubygems-40-all-02/build/9880709/

builder-live.log.gz says:

cmd: ['git', 'clone', 'https://src.fedoraproject.org/rpms/rubygem-aruba.git', '/var/lib/copr-rpmbuild/workspace/workdir-8hg6n96r/rubygem-aruba', '--depth', '500', '--no-single-branch', '--recursive']
cwd: .
rc: 128
stdout: 
stderr: Cloning into '/var/lib/copr-rpmbuild/workspace/workdir-8hg6n96r/rubygem-aruba'...
error: RPC failed; HTTP 503 curl 22 The requested URL returned error: 503
fatal: the remote end hung up unexpectedly
Cloning into '/var/lib/copr-rpmbuild/workspace/workdir-8hg6n96r/rubygem-aruba'...
error: RPC failed; HTTP 503 curl 22 The requested URL returned error: 503
fatal: the remote end hung up unexpectedly

Would you investigate the current git server status? Thank you in advance.

I also encountered the similar instability issue, when git from command line or on fedora package sources website.

I'm also running into this issue:

https://koji.fedoraproject.org/koji/taskinfo?taskID=139758169
=>
https://kojipkgs.fedoraproject.org//work/tasks/8217/139758217/checkout.log
=>

$ git clone -n https://src.fedoraproject.org/rpms/python-inema.git /var/lib/mock/f42-build-64258467-6623784/root/chroot_tmpdir/scmroot/python-inema
Cloning into '/var/lib/mock/f42-build-64258467-6623784/root/chroot_tmpdir/scmroot/python-inema'...
fatal: unable to access 'https://src.fedoraproject.org/rpms/python-inema.git/': The requested URL returned error: 503

https://src.fedoraproject.org/rpms/python-inema.git/
=> Firefox get on my workstation currently yields:

Error 503 Backend fetch failed
Backend fetch failed
Guru Meditation:
XID: 1477804044
Varnish cache server

https://www.fedorastatus.org/ claims that everything is fine, though:
=>

All Systems Operational

Visiting the front page https://src.fedoraproject.org/ in a browser takes very long and ends with an unavailable page, right now:

Sorry! This service is currently unavailable.

And https://www.fedorastatus.org/ still reports that everything is fine.
Either that service doesn't health check src.fedoraproject.org or that check isn't very thorough.

Yes, it's scrapers. ;(

And https://www.fedorastatus.org/ still reports that everything is fine.

Yes, it's manually updated when we know about a problem.
Luckily in my sleep I didn't know about this. ;( (nightmare?)

will start looking now.

Metadata Update from @kevin:
- Issue assigned to kevin
- Issue priority set to: Waiting on Assignee (was: Needs Review)
- Issue tagged with: high-gain, high-trouble

ok, the problem seems to have been that scrapers were (aggressively as usual) crawling the kernel package. They were hitting /blame/ and /history/ endpoints on every commit page they fetched, then recursively doing so on those and so on. ;(

I adjusted a bunch of things (more memory for pkgs01 backend, more apache threads/workers, caching of ttf files), but in the end the thing that got things back to normal was just blocking the /blame/ and /history/ endpoints. So, the web interface will no longer offer those (at least for now). If you need to do those things, you can git clone locally and do them easily.

So, things should be back to normal now and hopefully will stay that way.

Metadata Update from @kevin:
- Issue close_status updated to: Fixed
- Issue status updated to: Closed (was: Open)
Metadata

high-gain high-trouble

None
None
Normal
Static archive. pagure.io was sunset in 2026; this is a read-only snapshot.
Red Hat Logo

Fedora is sponsored by Red Hat. Learn more about the relationship between Red Hat and Fedora »

© Red Hat, Inc. and others.