From 38fa9d0ef55a6ca71c728703f08a3c7be0a142ec Mon Sep 17 00:00:00 2001 From: Cristian Le Date: Feb 23 2024 10:45:13 +0000 Subject: [PATCH 1/3] Fix MPI link --- diff --git a/guidelines/modules/ROOT/pages/MPI.adoc b/guidelines/modules/ROOT/pages/MPI.adoc index 9a488fd..2f2ac8c 100644 --- a/guidelines/modules/ROOT/pages/MPI.adoc +++ b/guidelines/modules/ROOT/pages/MPI.adoc @@ -18,9 +18,8 @@ The files of MPI compilers MUST be installed in the following directories: |File type |Placement |Binaries |`+%{_libdir}/%{name}/bin+` |Libraries |`+%{_libdir}/%{name}/lib+` -|[[PackagingDrafts/Fortran |Fortran modules]] |`+%{_fmoddir}/%{name}+` -|[[Packaging/Python |Python modules]] |`+%{python2_sitearch}/%{name}+` -`+%{python3_sitearch}/%{name}+` +|Fortran modules |`+%{_fmoddir}/%{name}+` +|link:Python.adoc[Python modules] |`+%{python3_sitearch}/%{name}+` |Config files |`+%{_sysconfdir}/%{name}-%{_arch}+` |=================================================================== From 6d7ffbb90f4d2430ad5e64d1a0d5c0e27de61c90 Mon Sep 17 00:00:00 2001 From: Cristian Le Date: Feb 23 2024 15:06:21 +0000 Subject: [PATCH 2/3] Refactor MPI document --- diff --git a/guidelines/modules/ROOT/pages/MPI.adoc b/guidelines/modules/ROOT/pages/MPI.adoc index 2f2ac8c..12d28bc 100644 --- a/guidelines/modules/ROOT/pages/MPI.adoc +++ b/guidelines/modules/ROOT/pages/MPI.adoc @@ -1,13 +1,26 @@ = Message Passing Interface +:mpi_wiki: https://en.wikipedia.org/wiki/Message_Passing_Interface +:openmpi: https://www.open-mpi.org[OpenMPI] +:mpich: https://www.mpich.org[MPICH] +:mvapich: https://mvapich.cse.ohio-state.edu[MVAPICH1 and MVAPICH2] + == Introduction -Message Passing Interface (MPI) is an API for parallelization of programs across multiple nodes and has been around since 1994 https://en.wikipedia.org/wiki/Message_Passing_Interface[1]. MPI can also be used for parallelization on SMP machines and is considered very efficient in it too (close to 100% scaling on parallelizable code as compared to ~80% commonly obtained with threads due to unoptimal memory allocation on NUMA machines). Before MPI, about every manufacturer of supercomputers had their own programming language for writing programs; MPI made porting software easy. +Message Passing Interface (MPI) is an API for parallelization of programs across multiple nodes and has been around +since 1994footnote:[link:{mpi_wiki}[]]. MPI can also be used for parallelization on SMP machines and is +considered very efficient in it too (close to 100% scaling on parallelizable code as compared to ~80% commonly obtained +with threads due to unoptimal memory allocation on NUMA machines). Before MPI, about every manufacturer of +supercomputers had their own programming language for writing programs; MPI made porting software easy. -There are many MPI implementations available, such as https://www.open-mpi.org/[Open MPI] (the default MPI compiler in Fedora and the MPI compiler used in RHEL), https://www.mpich.org/[MPICH] (in Fedora and RHEL) and -https://mvapich.cse.ohio-state.edu/[MVAPICH1 and MVAPICH2] (in RHEL but not yet in Fedora). +There are many MPI implementations available, such as link:{openmpi} (the default MPI compiler in Fedora and the MPI +compiler used in RHEL), link:{mpich} (in Fedora and RHEL) and link:{mvapich} (in RHEL but not yet in Fedora). -As some MPI libraries work better on some hardware than others, and some software works best with some MPI library, the selection of the library used must be done in user level, on a session specific basis. Also, people doing high performance computing may want to use more efficient compilers than the default one in Fedora (gcc), so one must be able to have many versions of the MPI compiler each compiled with a different compiler installed at the same time. This must be taken into account when writing spec files. +As some MPI libraries work better on some hardware than others, and some software works best with some MPI library, the +selection of the library used must be done in user level, on a session specific basis. Also, people doing high +performance computing may want to use more efficient compilers than the default one in Fedora (gcc), so one must be able +to have many versions of the MPI compiler each compiled with a different compiler installed at the same time. This must +be taken into account when writing spec files. == Packaging of MPI compilers @@ -23,7 +36,10 @@ The files of MPI compilers MUST be installed in the following directories: |Config files |`+%{_sysconfdir}/%{name}-%{_arch}+` |=================================================================== -As include files and manual pages are bound to overlap between different MPI implementations, they MUST also placed outside normal directories. It is possible that some man pages or include files (either those of the MPI compiler itself or of some MPI software installed in the compiler's directory) are architecture specific (e.g. a definition on a 32-bit arch differs from that on a 64-bit arch), the directories that MUST be used are as follows: +As include files and manual pages are bound to overlap between different MPI implementations, they MUST also be placed +outside normal directories. It is possible that some man pages or include files (either those of the MPI compiler itself +or of some MPI software installed in the compiler's directory) are architecture specific (e.g. a definition on a 32-bit +arch differs from that on a 64-bit arch), the directories that MUST be used are as follows: [cols=",",options="header",] |================================================ @@ -32,13 +48,17 @@ As include files and manual pages are bound to overlap between different MPI imp |Include files |`+%{_includedir}/%{name}-%{_arch}+` |================================================ -Architecture independent parts (except headers which go into `+-devel+`) MUST be placed in a `+-common+` subpackage that is `+BuildArch: noarch+`. +Architecture independent parts (except headers which go into `+-devel+`) MUST be placed in a `+-common+` subpackage that +is `+BuildArch: noarch+`. -The runtime of MPI compilers (mpirun, the libraries, the manuals etc) MUST be packaged into %\{name}, and the development headers and libraries into %\{name}-devel. +The runtime of MPI compilers (mpirun, the libraries, the manuals etc) MUST be packaged into %\{name}, and the +development headers and libraries into %\{name}-devel. -As the compiler is installed outside `+PATH+`, one needs to load the relevant variables before being able to use the compiler or run MPI programs. This is done using xref:EnvironmentModules.adoc[environment modules]. +As the compiler is installed outside `+PATH+`, one needs to load the relevant variables before being able to use the +compiler or run MPI programs. This is done using xref:EnvironmentModules.adoc[environment modules]. -The module file MUST be installed under `+%{_sysconfdir}/modulefiles/mpi+`. This allows as user with only one mpi implementation installed to load the module with: +The module file MUST be installed under `+%{_sysconfdir}/modulefiles/mpi+`. This allows as user with only one mpi +implementation installed to load the module with: .... module load mpi @@ -72,11 +92,15 @@ The module file MUST also set some helper variables (primarily for use in spec f As these directories may be used by software using the MPI stack, the MPI runtime package MUST own all of them. -MUST: By default, NO files are placed in `+/etc/ld.so.conf.d+`. If the packager wishes to provide alternatives support, it MUST be placed in a subpackage along with the ld.so.conf.d file so that alternatives support does not need to be installed if not wished for. +MUST: By default, NO files are placed in `+/etc/ld.so.conf.d+`. If the packager wishes to provide alternatives support, +it MUST be placed in a subpackage along with the ld.so.conf.d file so that alternatives support does not need to be +installed if not wished for. -MUST: If the maintainer wishes for the environment module to load automatically by use of a scriptlet in /etc/profile.d or by some other mechanism, this MUST be done in a subpackage. +MUST: If the maintainer wishes for the environment module to load automatically by use of a scriptlet in /etc/profile.d +or by some other mechanism, this MUST be done in a subpackage. -MUST: The MPI compiler package MUST provide an RPM macro that makes loading and unloading the support easy in spec files, e.g. by placing the following in `+/etc/rpm/macros.openmpi+` +MUST: The MPI compiler package MUST provide an RPM macro that makes loading and unloading the support easy in spec +files, e.g. by placing the following in `+/etc/rpm/macros.openmpi+` .... %_openmpi_load \ @@ -90,19 +114,30 @@ MUST: The MPI compiler package MUST provide an RPM macro that makes loading and loading and unloading the compiler in spec files is as easy as `+%{_openmpi_load}+` and `+%{_openmpi_unload}+`. -Automatic setting of the module loading path in python interpreters is done using a `+.pth+` file placed in one of the directories normally searched for modules (`+%{python2_sitearch}+`, `+%{python3_sitearch}+`). Those `+.pth+` files should append the directory specified with $MPI_PYTHON2_SITEARCH or $MPI_PYTHON3_SITEARCH environment variable, depending on the interpreter version, to `+sys.path+`, and do nothing if those variables are unset. Module files MUST NOT set PYTHONPATH directly, since it cannot be set for both Python versions at the same time. +Automatic setting of the module loading path in python interpreters is done using a `+.pth+` file placed in one of the +directories normally searched for modules (`+%{python2_sitearch}+`, `+%{python3_sitearch}+`). Those `+.pth+` files +should append the directory specified with $MPI_PYTHON2_SITEARCH or $MPI_PYTHON3_SITEARCH environment variable, +depending on the interpreter version, to `+sys.path+`, and do nothing if those variables are unset. Module files MUST +NOT set PYTHONPATH directly, since it cannot be set for both Python versions at the same time. -If the environment module sets compiler flags such as `+CFLAGS+` (thus overriding the ones exported in `+%configure+`, the RPM macro MUST make them use the Fedora optimization flags `+%{optflags}+` once again (as in the example above in which the openmpi-%\{_arch} module sets CFLAGS). +If the environment module sets compiler flags such as `+CFLAGS+` (thus overriding the ones exported in `+%configure+`, +the RPM macro MUST make them use the Fedora optimization flags `+%{optflags}+` once again (as in the example above in +which the openmpi-%\{_arch} module sets CFLAGS). == Packaging of MPI software -Software that supports MPI MUST be packaged also in serial mode [i.e. no MPI], if it is supported by upstream. (for instance: `+foo+`). +Software that supports MPI MUST be packaged also in serial mode [i.e. no MPI], if it is supported by upstream. (for +instance: `+foo+`). -If possible, the packager MUST package versions for each MPI compiler in Fedora (e.g. if something can only be built with mpich and mvapich2, then mvapich1 and openmpi packages do not need to be made). +If possible, the packager MUST package versions for each MPI compiler in Fedora (e.g. if something can only be built +with mpich and mvapich2, then mvapich1 and openmpi packages do not need to be made). -MPI implementation specific files MUST be installed in the directories used by the used MPI compiler (`+$MPI_BIN+`, `+$MPI_LIB+` and so on). +MPI implementation specific files MUST be installed in the directories used by the used MPI compiler (`+$MPI_BIN+`, +`+$MPI_LIB+` and so on). -The binaries MUST be suffixed with `+$MPI_SUFFIX+` (e.g. _openmpi for Open MPI, _mpich for MPICH and _mvapich2 for MVAPICH2). This is for two reasons: the serial version of the program can still be run when an MPI module is loaded and the user is always aware of the version s/he is running. This does not need to hurt the use of shell scripts: +The binaries MUST be suffixed with `+$MPI_SUFFIX+` (e.g. _openmpi for Open MPI, _mpich for MPICH and _mvapich2 for +MVAPICH2). This is for two reasons: the serial version of the program can still be run when an MPI module is loaded and +the user is always aware of the version s/he is running. This does not need to hurt the use of shell scripts: .... # Which MPI implementation do we use? @@ -121,19 +156,31 @@ mpirun -np 4 bar${MPI_SUFFIX} -process bar -collect .... -The MPI enabled bits MUST be placed in a subpackage with the suffix denoting the MPI compiler used (for instance: `+foo-openmpi+` for Open MPI [the traditional MPI compiler in Fedora] or `+foo-mpich+` for MPICH). For directory ownership and to guarantee the pickup of the correct MPI runtime, the MPI subpackages MUST require the correct MPI compiler's runtime package. +The MPI enabled bits MUST be placed in a subpackage with the suffix denoting the MPI compiler used (for instance: +`+foo-openmpi+` for Open MPI [the traditional MPI compiler in Fedora] or `+foo-mpich+` for MPICH). For directory +ownership and to guarantee the pickup of the correct MPI runtime, the MPI subpackages MUST require the correct MPI +compiler's runtime package. -Each MPI build of shared libraries SHOULD have a separate -libs subpackage for the libraries (e.g. foo-mpich-libs). As in the case of MPI compilers, library configuration (in `+/etc/ld.so.conf.d+`) MUST NOT be made. +Each MPI build of shared libraries SHOULD have a separate -libs subpackage for the libraries (e.g. foo-mpich-libs). As +in the case of MPI compilers, library configuration (in `+/etc/ld.so.conf.d+`) MUST NOT be made. -In case the headers are the same regardless of the compilation method and architecture (e.g. 32-bit serial, 64-bit Open MPI, MPICH), they MUST be split into a separate `+-headers+` subpackage (e.g. 'foo-headers'). Fortran modules are architecture specific and as such are placed in the (MPI implementation specific) `+-devel+` package (foo-devel for the serial version and foo-openmpi-devel for the Open MPI version). +In case the headers are the same regardless of the compilation method and architecture (e.g. 32-bit serial, 64-bit +OpenMPI, MPICH), they MUST be split into a separate `+-headers+` subpackage (e.g. 'foo-headers'). Fortran modules are +architecture specific and as such are placed in the (MPI implementation specific) `+-devel+` package (foo-devel for the +serial version and foo-openmpi-devel for the Open MPI version). -Each MPI build MUST have a separate -devel subpackage (e.g. foo-mpich-devel) that includes the development libraries and `+Requires: %{name}-headers+` if such a package exists. The goal is to be able to install and develop using e.g. 'foo-mpich-devel' without needing to install e.g. openmpi or the serial version of the package. +Each MPI build MUST have a separate -devel subpackage (e.g. foo-mpich-devel) that includes the development libraries and +`+Requires: %{name}-headers+` if such a package exists. The goal is to be able to install and develop using e.g. +'foo-mpich-devel' without needing to install e.g. openmpi or the serial version of the package. -Files must be shared between packages as much as possible. Compiler independent parts, such as data files in `+%{_datadir}/%{name}+` and man files MUST be put into a `+-common+` subpackage that is required by all of the binary packages (the serial package and all of the MPI packages). +Files must be shared between packages as much as possible. Compiler independent parts, such as data files in +`+%{_datadir}/%{name}+` and man files MUST be put into a `+-common+` subpackage that is required by all of the binary +packages (the serial package and all of the MPI packages). === A sample spec file -.... +[source,spec] +---- # Define a macro for calling ../configure instead of ./configure %global dconfigure %(printf %%s '%configure' | sed 's!\./configure!../configure!g') @@ -206,4 +253,4 @@ make -C $MPI_COMPILER install DESTDIR=%{buildroot} INSTALL="install -p" CPPROG=" %files openmpi # All openmpi linked files %files mpich # All mpich linked files -.... +---- From 69d28a24f32429ad8e003cedd28bdc5a07c414e4 Mon Sep 17 00:00:00 2001 From: Cristian Le Date: Feb 23 2024 15:11:24 +0000 Subject: [PATCH 3/3] Remove the requirement for `MPI_SUFFIX` in binaries --- diff --git a/guidelines/modules/ROOT/pages/MPI.adoc b/guidelines/modules/ROOT/pages/MPI.adoc index 12d28bc..cd3b1d1 100644 --- a/guidelines/modules/ROOT/pages/MPI.adoc +++ b/guidelines/modules/ROOT/pages/MPI.adoc @@ -135,27 +135,6 @@ with mpich and mvapich2, then mvapich1 and openmpi packages do not need to be ma MPI implementation specific files MUST be installed in the directories used by the used MPI compiler (`+$MPI_BIN+`, `+$MPI_LIB+` and so on). -The binaries MUST be suffixed with `+$MPI_SUFFIX+` (e.g. _openmpi for Open MPI, _mpich for MPICH and _mvapich2 for -MVAPICH2). This is for two reasons: the serial version of the program can still be run when an MPI module is loaded and -the user is always aware of the version s/he is running. This does not need to hurt the use of shell scripts: - -.... -# Which MPI implementation do we use? - -#module load mpi/mvapich2-i386 -#module load mpi/openmpi-i386 -module load mpi/mpich-i386 - -# Run preprocessor -foo -preprocess < foo.in -# Run calculation -mpirun -np 4 foo${MPI_SUFFIX} -# Run some processing -mpirun -np 4 bar${MPI_SUFFIX} -process -# Collect results -bar -collect -.... - The MPI enabled bits MUST be placed in a subpackage with the suffix denoting the MPI compiler used (for instance: `+foo-openmpi+` for Open MPI [the traditional MPI compiler in Fedora] or `+foo-mpich+` for MPICH). For directory ownership and to guarantee the pickup of the correct MPI runtime, the MPI subpackages MUST require the correct MPI