We now have at least three implementations of BLAS/LAPACK in Fedora/EPEL: netlib, atlas, and openblas. Due to implementation differences, it is important that all components of a particular software stack link to the same BLAS/LAPACK implementation. Also, users may want to choose a particular implementation that works best for them at run time.
I'm starting a draft here: https://fedoraproject.org/wiki/PackagingDrafts:BLAS_LAPACK
That makes use of environment modules where possible, and multiple sub-packages where not.
While working on julia recently it is becoming clear that this needs to get addressed soon. julia makes use of arpack, which currently links to tatlas, and openblas directly. This seems like a disaster waiting to strike. We are also working on making ILP64 (64-bit integer) versions available so it's a good time to set some standards.
Prior discussion in ticket #352
Although BLAS is AFAIK pretty much standardized (as it only implements fundamental linear algebra operations: vector and matrix addition and products and so on), the API of LAPACK still seems to be changing to some extent.
So, if you'd like to enforce changeability of BLAS/LAPACK implementations, you have just two options:
1) standardize version of LAPACK shipped in Fedora release X. All of the different implementations must use this version. [might even use the object files from lapack-static as we have done till now] 2) build BLAS-only packages, and use them with the netlib version of LAPACK. This guarantees perfect transferability, although some performance is lost when optimized versions of LAPACK routines are available.
I don't really see much of an issue with a system level choice of the library. Or even at the architecture level. Because OpenBLAS > ATLAS >> netlib BLAS. Although this will still leave the question of the use of a sequential vs a parallel implementation.
IMHO libraries should leave the choice of the BLAS/LAPACK library to the link phase. And for applications, the library can be chosen by the packager.
(as a scilab packager)
Great comment Susi, AFAIK optimized implementation of LAPACK are always partial and only provide a reduced set of functions. The OpenBLAS and ATLAS build systems will mostly re-use netlib LAPACK. So I guess the 1/ solution is the good one : netlib LAPACK will be used both as an ABI and as the reference implementation ; each optimized version might provide its own set of specialized implementation preserving the ABI by linking against LAPACK-static.
I build Scilab using only netlib BLAS and netlib LAPACK allowing the user to choose the runtime using LD_PRELOAD (this is very effective while benchmarking). I guess this module approach is compatible but can you confirm ?
Ok, let us know when the draft is ready.
Hopefully we can enforce 1 - consistent LAPACK API. Using static lapack to link is useful here. We've dropped it at the moment in openblas because currently lapack does not provide an ILP64 interface with 64_ suffixed symbols, but hopefully that will be fixed.
It would be nice to not link libraries directly to blas/lapack. This is proving problematic in my brief testing for things like julia that dlopen() libraries, but perhaps this can be fixed with the proper dlopen() options.
LD_LIBRARY_PATH should work much like LD_PRELOAD by putting the chosen libraries first in the linker search path.
I've updated the draft to add system level library selection via alternatives.
I've setup https://copr.fedoraproject.org/coprs/g/scitech/blas/ to test this proposal out.
I guess i should ping on this to see if anything has happened or if there's any point in discussing it yet.
Metadata Update from @tibbs: - Issue close_status updated to: None - Issue tagged with: needinfo
No, this will require a lot of work in the BLAS/LAPACK providing packages. Hopefully someday, but...
This has been in needinfo for well over two years now. I'm going to go ahead and close it. Please feel free to reopen or file a new ticket if any progress is made.
Metadata Update from @tibbs: - Issue untagged with: needinfo - Issue close_status updated to: nothingtodo - Issue status updated to: Closed (was: Open)