The log tree and its locks#
A run writes its logs under filesystems.log_root, LOG_ROOT on this page, in a directory of its own for each target.
It takes locks in the log root, and in the state root of eb-staged-rebuild and the admission lock when the site file uses them.
The log directory#
LOG_DIR is LOG_ROOT/LOG_STAMP/TARGET, and every command of a run gets it as EB_PIPELINE_LOG_DIR.
LOG_STAMP is --log-stamp or EB_PIPELINE_LOG_STAMP, else the time run starts in Coordinated Universal Time (UTC), as YYYYMMDD_HHMMSS.
plan without a stamp, and explain, show <log-stamp> in its place.
Two runs with the same stamp and target write into the same directory; the runs of one pipeline can share a stamp when each takes a different target.
After the run in docs/examples/files.sh, which builds one line and fails another with [logs] keep_failed and [logs.rotate] set, the log root holds:
find logs | sort
logs
logs/.eb-pipeline-rotate.lock
logs/20261117_090000
logs/20261117_090000/cpu
logs/20261117_090000/cpu/Broken-1.0-GCCcore-15.2.0.keep-logs.log
logs/20261117_090000/cpu/Broken-1.0-GCCcore-15.2.0.stdout.log
logs/20261117_090000/cpu/Example-1.0-GCCcore-15.2.0.stdout.log
logs/20261117_090000/cpu/rotate.log
File |
Phase |
What it holds |
|---|---|---|
|
before |
The lines that |
|
|
The output of |
|
|
The output of the prefetch of the line’s sources. |
|
|
The output of attempt |
|
|
The output of the line’s |
|
|
The logs of EasyBuild itself, which |
|
|
The output of |
|
|
The output of the cache command. |
|
|
The output of |
|
routed |
The directory |
STEM is the easyconfig file name without .eb, with .2, .3 and so on added when the list repeats a line, and UNIT is all, cpu-prerequisites, prelude or the name of a shard.
Each log holds the standard output and the standard error of its command, which starts with no standard input, and run creates it anew when the command starts.
A command that cannot start leaves one line in its log, cannot start PROGRAM: ERROR, and counts as exit 127.
The log root#
File |
What it is |
|---|---|
|
The directories of the runs with that stamp, one per target. |
|
The rotation lock of eb-pipeline; the first run with |
|
The rotation lock of eb-keep-logs, which an |
eb-keep-logs rotate compresses and deletes the run directories by the age of their stamp, and keeps files of its own in the tree; the eb-keep-logs documentation describes them.
Locks#
Lock |
Taken by |
How |
While another holder has it |
|---|---|---|---|
|
|
|
A run that starts waits while another holds it alone, as a rotation does. A run that cannot take it alone at its end leaves |
|
|
|
It waits up to |
|
|
Alone, for the whole of its build and publish |
It waits up to |
|
|
|
|
STATE_ROOT is rebuild.state_root, else the EB_STAGED_REBUILD_STATE_ROOT that eb-staged-rebuild sees, else .eb-staged-rebuild beside the software tree.
flock locks hold between nodes only where the filesystem carries them between nodes.
When a run cannot take its rotation lock at all, it logs WARN [logs] TARGET: cannot lock PATH: ERROR; the rotation does not wait for other runs and rotates without it.
Lock order#
Whatever takes both rotation locks takes them in this order:
LOG_ROOT/.eb-pipeline-rotate.lockLOG_ROOT/.eb-keep-logs-rotate.lock
A run’s rotation does, and so does a rotation outside a run that takes the first with flock(1); eb-keep-logs takes only the second.
A run waits for the first lock only at its start, before it holds any other, and at its end takes it alone only when it can do so at once.
See also#
The site file, for
[logs],[rebuild]and[admission]The plan and result files, for the statuses of the
logsphase and of a lineWhy the rotation takes two locks, for why the locks are taken as they are