Environment variables#
The variables eb-pipeline reads in place of an option or a site file value, the ones a run reads because the site file names them, and the ones it sets for the commands it starts.
Options#
An option that is not on the command line takes its value from its variable. An option on the command line wins.
Variable |
Option |
Commands |
|---|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
--out, --result, --shard-plan, --format, --dry-run and --probe-tools have no variable.
Site file values#
EB_PIPELINE__SECTION__KEY=VALUE sets one value of the site file, after the file is read and before it is checked.
Every command that reads a site file applies these variables.
The name after
EB_PIPELINE__is split at each__, and each part, lowercased, names a table or, last, the key.EB_PIPELINE__FILESYSTEMS__SCRATCHsetsscratchin[filesystems], andEB_PIPELINE__LOGS__ROTATE__DELETE_AFTER_DAYSsetsdelete_after_daysin[logs.rotate].A part must hold letters, digits and underscores only, and none may be empty.
VALUEis read as a TOML value, such astrue,16,"16"or["a", "b"], and as a string when it is not one./scratch/aliceis the string"/scratch/alice".A table on the path that the file does not have is made.
A path that reaches into an array of tables, such as
targets, is refused: the values of a target stay in the file. So is a path that ends at a table, or that passes through a value that is not a table.The variables apply in the order of their names, whatever the order of the environment.
Each one writes
INFO [site] -: KEY set from NAMEto stderr, with the key and never the value.
The result is checked like the file itself, so an unknown key or a value of the wrong type exits 2 with invalid site file after environment overrides: and the reason.
Since every part is lowercased, these variables cannot set a key that has an upper-case letter, such as a package name under [modules.package_env].
Read during a run#
The site file names these variables; run reads them before the first build.
plan and run --dry-run show ${NAME} in their place.
Variable |
Use |
|---|---|
the one named by |
The job id that |
the one named by |
A digest that the routed commands get as |
each one named in |
Module trees, separated by |
Set for the commands of a run#
A command that run starts inherits the environment of eb-pipeline, with the variables below set over it.
The keys of [scheduler.env] come first, and a variable of the table below with the same name replaces one of them.
Variable |
Value |
Set when |
|---|---|---|
the keys of |
their values |
always |
|
|
always |
|
|
always |
|
the entries of |
|
|
the target’s |
|
|
the target’s |
|
|
the target’s |
|
|
|
one of them is set |
|
|
one of them is set |
|
the target’s |
it is set |
|
|
|
|
|
it is set |
|
the site file, as an absolute path |
always |
|
|
always |
|
the target’s name |
always |
|
the build type |
always |
|
the run id |
always |
|
the log stamp |
always |
|
|
always |
|
|
it is set |
|
|
it is set |
|
the entries of |
|
|
|
|
|
|
it is set |
|
|
|
|
|
|
the one named by |
the target’s |
|
the one named by |
the target’s |
|
|
the target’s |
|
EB_PIPELINE_SITE, EB_PIPELINE_TARGET, EB_PIPELINE_BUILD_TYPE, EB_PIPELINE_RUN_ID and EB_PIPELINE_LOG_STAMP are also the variables of the options above, so a command that runs eb-pipeline without those options gets the site file, target, build type, run id and log stamp of the run.
eb-pipeline does not read EB_PIPELINE_SITE_NAME; EB_PIPELINE__SITE__NAME, with two underscores, is the variable that sets name in the site file.
[modules.eessi] is on when the site file has the table and its enabled is not false.
[licences] is on when its enabled and enforce are not false.
EESSI stands for the European Environment for Scientific Software Installations.
Set for some commands#
Some commands get more variables, set over those above.
Variable |
Value |
Commands |
|---|---|---|
the keys of |
their values; per key, the target’s wins |
the prefetch and build commands of a line whose easyconfig is of package |
the keys of |
their values |
the same, with |
|
the group of |
the same, with |
|
the target’s |
the prefetch commands, when the target has a |
|
the |
the prefetch and build commands, when the target has a |
the keys of |
their values, with the placeholders of the route replaced |
the commands that the route wraps |
|
|
the same, when it is set |
|
the entries of |
the same, when they are not empty |
|
the value of the variable that |
the same, when it is set |
|
|
|
|
|
|
|
the |
|
|
|
the same, when it is set |
|
the |
the same, when one of them is set |
The module wrapper#
When modules.init is not empty, modules.unuse_path is true or modules.shell_init is set, the shard, prefetch, build, keep-failed and cache commands start through modules.shell, which reads three of the variables above:
EB_PIPELINE_SHELL_INIT: the file it sources first, when set.EB_PIPELINE_UNUSE_PATH: when set, it emptiesMODULEPATH.EB_PIPELINE_MODULES: it runsmodule purge, thenmodule loadfor each name in it, in order, then the command. It splits the variable at white space, whichcheckrefuses in an entry ofmodules.init, and gives each name tomodule loadas written: a name with a pattern in it, such asGCC/*, is not matched against file names.
The wrapper is a script for a POSIX shell: check accepts sh, bash, dash, ash, ksh, ksh93, mksh and yash as modules.shell, by name or by path, and refuses any other.
zsh would load the names of EB_PIPELINE_MODULES as one, since it does not split an unquoted variable, and csh, tcsh and fish cannot parse the script.
The admit and rotate commands start without it.