The packaging guidelines say:
"If a JavaScript library typically is shipped as minified or compiled code, it MUST be compiled or minified as part of the RPM build process. Shipping pre-minified or pre-compiled code is unacceptable in Fedora."
I think anaconda-webui is violating this.
If I try and build anaconda-webui with a patch that changes one of its javascript source files - e.g. https://patch-diff.githubusercontent.com/raw/rhinstaller/anaconda-webui/pull/1037.patch - the build fails like this:
+ cd anaconda-webui + /usr/bin/make install DESTDIR=/builddir/build/BUILD/anaconda-webui-49-build/BUILDROOT 'INSTALL=/usr/bin/install -p' PREFIX=/usr /usr/bin/make VERSION.txt && \ /usr/bin/make package-lock.json && \ NODE_ENV=production ./build.js make[1]: Entering directory '/builddir/build/BUILD/anaconda-webui-49-build/anaconda-webui' make[1]: Leaving directory '/builddir/build/BUILD/anaconda-webui-49-build/anaconda-webui' make[1]: *** No rule to make target 'packaging/anaconda-webui.spec.in', needed by 'anaconda-webui.spec'. Stop.
that is, when the package's %install step runs make install, some part of the upstream build system notices that a node.js 'build' step - NODE_ENV=production ./build.js - is required, and tries to run it (and it fails, clearly indicating the spec is not set up to handle actually doing the compilation).
%install
make install
NODE_ENV=production ./build.js
In the logs of official builds of anaconda-webui - e.g. this build.log from this build - there is no trace of any node.js compilation happening in the %install step, which implies that the build system realizes the pre-compiled assets in the tarball - they're in the anaconda-webui/dist subdirectory - are up to date with the sources, and skips compiling them. That means the build works. But it is, IMO, very clearly against the guidelines.
anaconda-webui/dist
The 'meat' of the shipped packages appears to be the files:
/usr/share/cockpit/anaconda-webui/index.css.gz /usr/share/cockpit/anaconda-webui/index.css.map /usr/share/cockpit/anaconda-webui/index.js.gz /usr/share/cockpit/anaconda-webui/index.js.map
and these files are just copied straight from the dist/ folder of the tarball, by the looks of it.
dist/
As well as being a violation of the guidelines, this is a practical problem, because it makes it very difficult for me to test changes to anaconda-webui by doing a modified package build...because I can't do one.
CC @kkoukiou
oh, I should note there is a similar rule for CSS which the package also appears to violate:
"Pure CSS frameworks can be included as-is. CSS frameworks that use an alternative language that compiles to CSS, such as LESS, must compile to CSS as part of the build process.
Packages containing CSS should make the best effort to regenerate any precompiled/minimized CSS wherever possible, as this leads to more maintainable packages. Where this would result in a significant hardship, the bundled pregenerated CSS may be shipped with a specfile comment explaining the decision. This does not eliminate the requirement to validate licenses of bundled content."
To me that is poorly drafted, as it's unclear how the "best effort" language in the second paragraph interacts with the "must" language in the first paragraph, but either way, the package doesn't comply, as there's no spec file comment explaining any "significant hardship".
The initial package review does not appear to have ever considered the situation wrt these guidelines.
I think this is a cockpit thing in general, probably all the cockpit packages need to be re-evaluated.
Yes, I was about to file another issue saying the same thing: cockpit appears to violate the same guidelines. Exactly the same test proves it: try building a cockpit package with a patch to any of the javascript sources in the pkg/ directory. The build will fail:
pkg/
cd . && NODE_ENV='production' tools/termschutz ./build.js make[1]: Leaving directory '/builddir/build/BUILD/cockpit-348-build/cockpit-348' make[1]: *** [Makefile:5947: dist/static/manifest.json] Error 127
CC @martinpitt
The guidelines also say
Packages containing JavaScript should make the best effort to regenerate any precompiled/minimized JS wherever possible, as this leads to more maintainable packages. Where this would result in a significant hardship, the bundled pregenerated JS may be shipped with a specfile comment explaining the decision. This does not eliminate the requirement to validate licenses of bundled code.
and there's a similar guideline for CSS. So it seems the guidelines are self-contradictory here. At the very least, there needs to be a specfile comment explaining the decision, which seems to also be missing from the anaconda-webui package.
In general, I think there needs to be more guidance and tooling for packaging "web stuff." It's good to have rules, but in this case in particular, they're not backed up with clear guidance on how an everyday packager (e.g., a Go packager packaging an app such as openbao or prometheus or forgejo that also includes web assets alongside Go source code or a Python packager building Sphinx docs or a package like cockpit/anaconda-webui) can handle web assets while following the bundling and licensing guidelines.
The reason the Go packaging change is taking so long (in addition to me needing to find a block of time where I can sit down and finish writing the guidelines) is because we wanted to make sure there was tooling and practical guidance in place to handle bundling and licensing and security updates before changing the guidelines and retiring all the library packages.
I (at the time), after some discussion and seeing other packages use the same idiom, felt this to be under the 'hardship' part of the guideline and didn't fully consider the implications to downstream patching.
Apologies since that discussion should've taken place in the BZ and the comment should've been there.
In general, I think there needs to be more guidance and tooling for packaging "web stuff." It's good to have rules, but in this case in particular, they're not backed up with clear guidance on how an everyday packager (e.g., a Go packager packaging an app such as openbao or prometheus or forgejo that also includes web assets alongside Go source code or a Python packager building Sphinx docs or a package like cockpit/anaconda-webui) can handle web assets while following the bundling and licensing guidelines. The reason the Go packaging change is taking so long (in addition to me needing to find a block of time where I can sit down and finish writing the guidelines) is because we wanted to make sure there was tooling and practical guidance in place to handle bundling and licensing and security updates before changing the guidelines and retiring all the library packages.
Agreed, I'm working currently on packaging a web application with associated (and upstream patched and built javascript assets) and I've been having a hard time finding the normal approach though this issue clarifies some bits :)
Yeah, we are painfully aware of this situation. We've been trying to take stabs at this for years. I made quite some progress in December here https://github.com/martinpitt/cockpit-files/tree/distrobuild , but it needs more work. The biggest blocker is https://bugzilla.redhat.com/show_bug.cgi?id=2387212 - without that, we'd have to ship esbuild in node_modules/ as part of the Fedora upstream tarball, and that sucks as it's an arch specific binary.
In Debian, I got node_modules/ down to basically PatternFly and dart-sass, plus very few very small extra modules which are unproblematic . Unfortunately PatternFly itself has a zillion dependencies, and dart-sass is quite big as well (see https://bugs.debian.org/1051470 and https://bugs.debian.org/998024 , same situation in Fedora). Fedora also does not package React (Debian does).
So the current plan to fix this on our side is: - Fix esbuild in Fedora (https://bugzilla.redhat.com/show_bug.cgi?id=2387212) - Start shipping upstream releases with the minimal (still huge) set of node_modules/ build dependencies included.
That's a lot of bundling (which is also frowned upon), but the best that we can realistically do.
TL/DR: Packaging web apps in distros is hard 😢
Thanks for the info. Is there a note in the spec explaining this? I took a quick look and didn't see one, but I may have missed it.
I worked on Fedora's esbuild package today, and got it working. Updated https://bugzilla.redhat.com/show_bug.cgi?id=2387212 and sent https://src.fedoraproject.org/rpms/golang-github-evanw-esbuild/pull-request/13
Once that lands in current Fedora releases, that resolves the biggest blocker of building the js bundles during rpm build. There's some more missing bits, but I'm working on them.
More yak shaving, I'm afraid. Before we can fix the esbuild package (without hacks), we need to fix nodejs packaging: https://bugzilla.redhat.com/show_bug.cgi?id=2404330
I guess @martinpitt forgot to update this, but good news: this is fixed in cockpit now! Many thanks for that.
Hopefully anaconda-webui can follow suit.