Currently, several packages have been orphaned because the original maintainer bundled the main package they were dependencies to, impacting 49 packages. If these orphaned packages are retired, it will affect additional packages.

With Fedora 42 branching in three weeks and the orphaned packages not set to be retired until February 22nd, any decisions made now should primarily impact Fedora 43.

A new tool called go-vendor-tools, developed by @gotmax23, helps bundle dependencies and can be integrated with Packit. However it requires more development based on feedback from the main developer. Some packages, , such as opentofu, helm, gh or kubernetes stack, already depend on it.

Other packages, like tailscale and glow, use a shell script to create the bundle tarball. As go-vendor-toolsis not yet ready for EPEL, this method is used mainly by packagers that also work in EPEL.

If bundling becomes the default approach, many dependencies would effectively become leaf packages and could reasonably be retired. However, if the bundling policy were to be reversed in the future, it might require packaging hundreds of dependencies again.

There may be an alternative involving enabling Go modules, as discussed by @qulogic, though its feasibility is uncertain.

I realize you want fesco's opinions on this, but perhaps this would be best started as a discssion on the devel list or discussion area?
That way we can gather more feedback than just fesco folks.

I do also realize this has been discussed a fair bit in the past...

Disclaimer: I have been involved in many of those conversations since I have been in the Go SIG for many years.

my2cc on the topic:

The goal of Fedora is to create the best possible distro for our stakeholders.
For this specific conversation, I think the four kind of stakeholders we should consider are:

1. The Fedora users that would like to use some applications that incidentally are written in Go
2. The Fedora users that want to develop an application (or library, etc) in Go
3. The people that are creating derived distros (both the EL world and "other" distros, such as Asahi, Bluefin, etc)
4. The Fedora Packagers involved in golang packahges

Also, we should do this, by adhering to the four foundations: Freedom, Friends, Features, First.

The question, imho, should be: is vendoring bringing us closer or further from serving our stakeholders and onoring our foundations?

There are a few aspects to consider.

1. Our users (and downstreamer) would benefit from us being able to adhere better to the "First" foundation. At the moment we are having some complexity in this, since often a new version will require new versions of dependencies (or new dependecies), which makes it hard to promptly package those new updates.

2. Having to manage 10x the number of package (yes, the proportion is roughly 1:10 between the applications and the dependencies) is tolling on packagers. Managing 10x should provide us a big advantage, otherwise it is not worth it.

3. Libraries packages are only used at packages compile time, since Go is statically links libraries, so the end user only installs the binary package that wants to use.

4. As far as I know, no Go developer installs the libraries from Fedora packages, since it is way easier and common to use go mod to pull down the dependencies during the development phase, rather than getting them from te OS packaging system.

5. Having so many packages, each of which needs to have a main admin, and so many dependencies, exposes Fedora and the packagers to huge waves of orphan packages when a "core" Go SIG member becomes unresponsive (it happened a year or so ago, and it was not pretty)

6. Go allows cyclical dependencies between projects, which means that there are packages in Fedora that need to have special code in the spec file to handle a bootstrap mode to allow the installation with limited dependencies, which then needs to be re-built in "standard" mode, after all the required dependencies are available. Also, this makes it more complex to perform rebuilds and recover from broken updates or similar scenarios.

7. We (Fedora) needs to ensure that we are able to update and rebuild all relevant packages when there are CVEs or similar security issues. The current process with many library packages has not made this easy. We need to ensure that, whatever the new process is going to be, it will make it easier to handle those cases. The CVEs can come either from the compiler/stdlib (which simply requires to re-build all binaries packages) or from a dependency, which requires to bump the library and rebuild all binaries depending on that library, or from a package, in this case the requirement is simply to upare the source and rebuild the single pakcage.

Personally, I think the best solution given all previous consideration is:

1. Ensure go-vendor-tools is available for all branches (including EPEL, see https://gitlab.com/fedora/sigs/go/go-vendor-tools/-/issues/10)

2. Ensure that we have enough tooling so that (7) works well.

3. Migrate all packages to go-vendor-tools

4. Retire all packages that are leaf libraries (basically golang-*)

In the process I think there will be the need of at least 1 change proposal (potentially in can be more than one if we go for a more staged approach). I'm also willing to work on change proposals for this.

Generally it seems like a good idea to do this. I've worked with some Go packages which indeed have large dependency lists and are a big chore to maintain each and every dependency separately. Submitting them all would overload the review system and all the points made by @fale apply afterwards. This keeps me away from even trying to package these.

I'm going to take a look at using go-vendor-tools for my spec files as well; currently it's a custom shell script to generate the bundle requires :)

I'm in agreement with this change, and I think @fale has done an excellent job of laying out the issues.

I think it is more important to make it easier to package projects written in go, and maintain them, than it is to make sure their dependencies are fedora packages. We should be following the best practices of upstream go developers rather than trying to impose Fedora's packaging rules onto an ecosystem that just doesn't operate that way. It will make it easier to update things quickly, leading to more projects being available in Fedora and better security.

I realize you want fesco's opinions on this, but perhaps this would be best started as a discssion on the devel list or discussion area?
That way we can gather more feedback than just fesco folks.

I do also realize this has been discussed a fair bit in the past...

+1, fesco ticket != discussion forum.

While I appreciate why you want to do this, you're making it sound all land-of-flowing-milk-and-honey like, there definitely are downsides to building with vendored dependencies that everybody should be aware of. I'll put them all into the discussion thread that you'll start instead of listing them here ;)

Linking the discussion in the devel-list for reference: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/DXEQZBARCOVV6O7BHIVFXAQ3LRKORVGG/

We had multiple people try and package https://github.com/spiffe related projects (properly) and burnt themselves out before even having prototype level packages. Big plus one from me for updated rules.

@fale I think it's fair to say that the discussion in Fedora showed that a large fraction of people fall under the "sad but true" conclusion wrt. vendoring by default. Can you please prepare some formal proposal and PR for the Packaging Guidelines so that we can finish this?

@fale ping

The Go SIG is planning to publish a Change Proposal for Fedora 43 once the go-vendor-tools project is mostly feature complete. I think we can close this, as there's nothing left to discuss with FESCo until we submit the Change.

Thanks, I'll close this issue. Looking forward to the CP.