Let users load what a target installs on top of EESSI#
Make what the targets of a site file install on top of the European Environment for Scientific Software Installations (EESSI) loadable by the users of each target’s nodes, in their shells and in their jobs.
The EESSI module adds a site tree to MODULEPATH from EESSI_SITE_SOFTWARE_PREFIX as it loads: PREFIX/versions/RELEASE/software/linux/CPU_TARGET/modules/all, for the CPU target it loads for, when that directory exists.
Without the variable, it takes the host_injections tree of the EESSI repository instead, so a run that installed every line can leave users without any of it.
Before you start#
A site file whose targets install on top of EESSI, with
[modules.eessi]andsite_install = true, as theeessi-sitetemplate of The site file templates has.A way to set a variable in the environment of users on the nodes before EESSI loads there, such as a file in
/etc/profile.d.For step 2, where eessi-arch-guard is the Slurm
TaskProlog, a shell with EESSI loaded.
The site file on this page builds for the zen2 and zen4 nodes of Example University, each with a site prefix of its own:
# Example University: the zen2 and zen4 nodes of the cpu partition, each
# with a site prefix of its own, installing on top of EESSI 2026.06.
schema_version = 1
kind = "site"
[site]
name = "example"
build_account = "alice"
default_group = "alice"
[scheduler]
kind = "slurm"
launch = "srun"
job_name_template = "{project}-{build}-{phase}-{target}"
[phase_defaults]
plan = "-p cpu -N 1 --ntasks-per-node=1 -c 1 -t 15:00"
short = "-p cpu -N 1 --ntasks-per-node=1 -c 1 -t 15:00"
build = "-p cpu -N 1 --ntasks-per-node=1 -c 16 -t 1-00:00:00"
cpu = "-p cpu -N 1 --ntasks-per-node=1 -c 16 -t 1-00:00:00"
cache = "-p cpu -N 1 --ntasks-per-node=1 -c 1 -t 15:00"
[[targets]]
name = "zen2"
build_lists = ["zen2.list"]
cpu_target = "x86_64/amd/zen2"
site_prefix = "/opt/site/zen2"
[targets.phases]
build = "-p cpu -C zen2 -N 1 --ntasks-per-node=1 -c 16 -t 1-00:00:00"
[[targets]]
name = "zen4"
build_lists = ["zen4.list"]
cpu_target = "x86_64/amd/zen4"
site_prefix = "/opt/site/zen4"
default = true
[targets.phases]
build = "-p cpu -C zen4 -N 1 --ntasks-per-node=1 -c 16 -t 1-00:00:00"
[filesystems]
scratch = "/scratch/alice"
tmp = "/scratch/alice/tmp"
# A build directory and a download directory per target, so that the
# targets can build at once.
build_path = "/scratch/alice/build/{target}"
source_paths = ["/scratch/alice/sources/{target}", "/opt/site/sources"]
install_root = "/opt/site"
log_root = "/opt/site/logs"
[modules]
init = ["EESSI/2026.06", "EESSI-extend/2026.06-easybuild"]
[modules.eessi]
site_install = true
Steps#
Read the prefix and the CPU target that each target’s builds install for, and the tree that EESSI-extend installs them into:
for t in $(eb-pipeline targets --site site.toml); do eb-pipeline explain --site site.toml --target "$t" | grep -E "install_note|EESSI_SITE_SOFTWARE_PREFIX|EESSI_SOFTWARE_SUBDIR_OVERRIDE" done
install_note = "EESSI-extend replaces EASYBUILD_INSTALLPATH when it loads: /opt/site/zen2/versions/<release>/software/linux/x86_64/amd/zen2" EESSI_SITE_SOFTWARE_PREFIX = "/opt/site/zen2" EESSI_SOFTWARE_SUBDIR_OVERRIDE = "x86_64/amd/zen2" install_note = "EESSI-extend replaces EASYBUILD_INSTALLPATH when it loads: /opt/site/zen4/versions/<release>/software/linux/x86_64/amd/zen4" EESSI_SITE_SOFTWARE_PREFIX = "/opt/site/zen4" EESSI_SOFTWARE_SUBDIR_OVERRIDE = "x86_64/amd/zen4"
runexports both variables to every command of a target:EESSI_SITE_SOFTWARE_PREFIXis itssite_prefix, orfilesystems.install_rootwhen it sets none, andEESSI_SOFTWARE_SUBDIR_OVERRIDEis itscpu_target. Users see what a target installed when their EESSI module loads with the same prefix, for the same CPU target.Check that the
cpu_targetof each target is the CPU target that jobs load on its nodes. With eessi-arch-guard as the SlurmTaskProlog, that is the target the guard selects for a task, which can differ from the one EESSI selects on the same CPU; a task prints it, here on azen4node:srun -p cpu -C zen4 -N 1 printenv EESSI_SOFTWARE_SUBDIRIt should print
x86_64/amd/zen4. Any other target means thatcpu_targetdoes not match the nodes: set it to the target that the task printed, and build the target’s lists again, as a tree built for a CPU target that jobs do not load is on no user’sMODULEPATH.Set the target’s prefix before EESSI loads, in the environment that users get on its nodes, for example from a file in
/etc/profile.dthat runs before the one that loads EESSI; on thezen4nodes:export EESSI_SITE_SOFTWARE_PREFIX=/opt/site/zen4A Slurm job starts, by default, with the environment of the shell that submitted it, so a job submitted with the prefix of one target keeps it on the nodes of another, where EESSI finds no tree under it. Targets that share one
site_prefixneed one value on every node, as the tree under it is per CPU target already; with staged rebuilds, each of them then setssoftware_rootandmodules_rootto its own tree, aschecksays.
Check that it worked#
In a job on a node of the target, the EESSI module names the site tree it took, and module avail lists its module directory:
srun -p cpu -C zen4 -N 1 bash -c 'echo "$EESSI_SITE_SOFTWARE_PATH"; module -t avail 2>&1 | grep "^$EESSI_SITE_SOFTWARE_PATH"'
On a zen4 node, the first line is the tree that the install_note of zen4 names, with the EESSI release in place of <release>.
The second line is the modules/all directory of that tree, with a colon after it, once the target has installed a module there.
If it fails#
Message |
Cause |
What to do |
|---|---|---|
|
|
Set it before EESSI loads, as in step 3, and start a new shell. |
|
The job kept the prefix of the shell that submitted it. |
Submit from a shell with the prefix of the target, or give the targets one |
|
The task loaded EESSI for another CPU target than the one the target builds for. |
Set |
No second line |
Nothing is installed in that tree: the target has not installed a module yet, or the prefix is not mounted on the node. |
Compare the tree with the |
|
The shell that ran |
Load EESSI first, as users do. |
The output of step 1 comes from docs/examples/site-installs.sh; the other commands need a cluster with EESSI.