ci: install deps from committed lockfiles when present

Reusing concretization between the weekly buildcache (fresh) and weekday CI
(reuse) can drift if runner externals change, causing avoidable cache misses.

- setup-deps installs from test/ci/locks/<env>-gcc<N>.lock when it exists,
  skipping concretization for byte-identical hashes; falls back to the spec
  yaml otherwise
- buildcache exports each env's spack.lock as a downloadable artifact so the
  lockfiles can be regenerated on the ubuntu-24.04 runner and committed
- document the manual regeneration flow in test/ci/locks/README.md
This commit is contained in:
Dennis Klein
2026-05-31 21:08:42 +02:00
committed by Dennis Klein
parent ffc9c60f73
commit bb5c0a998c
3 changed files with 70 additions and 0 deletions

View File

@@ -43,7 +43,25 @@ runs:
ghcr-buildcache oci://ghcr.io/fairrootgroup/fairmq-spack-buildcache
echo "::endgroup::"
- name: Resolve lockfile
id: lock
shell: bash
run: |
# Prefer a committed, fully-concretized lockfile so the hashes match the
# binaries the buildcache pushed (guaranteed cache hits, no re-solve).
# Fresh runs (the buildcache itself) must re-concretize, so skip the lock.
lock="test/ci/locks/${{ inputs.env }}-gcc${{ inputs.gcc }}.lock"
if [ "${{ inputs.fresh }}" != "true" ] && [ -f "$lock" ]; then
echo "Using committed lockfile: $lock"
echo "use=true" >> "$GITHUB_OUTPUT"
echo "path=$lock" >> "$GITHUB_OUTPUT"
else
echo "No lockfile for ${{ inputs.env }}-gcc${{ inputs.gcc }}; concretizing from spec"
echo "use=false" >> "$GITHUB_OUTPUT"
fi
- name: Install GCC
if: steps.lock.outputs.use != 'true'
shell: spack-bash {0}
run: |
echo "::group::Install GCC"
@@ -53,6 +71,7 @@ runs:
echo "::endgroup::"
- name: Install dependencies
if: steps.lock.outputs.use != 'true'
shell: spack-bash {0}
run: |
echo "::group::Install dependencies"
@@ -60,5 +79,20 @@ runs:
spack -e fairmq add gcc@${{ inputs.gcc }}
spack -e fairmq config add "packages:all:require:'%gcc@${{ inputs.gcc }}'"
spack -e fairmq install --fail-fast ${{ inputs.fresh == 'true' && '--fresh' || '' }}
echo "::endgroup::"
- name: Install dependencies (from lockfile)
if: steps.lock.outputs.use == 'true'
shell: spack-bash {0}
run: |
echo "::group::Install dependencies (from lockfile)"
spack env create fairmq ${{ steps.lock.outputs.path }}
spack -e fairmq install --fail-fast
echo "::endgroup::"
- name: Export environment
shell: spack-bash {0}
run: |
echo "::group::Export environment"
spack env activate --sh fairmq | grep '^export ' | sed 's/^export //;s/;$//' >> $GITHUB_ENV
echo "::endgroup::"

View File

@@ -50,6 +50,21 @@ jobs:
spack -e fairmq mirror set --oci-username ${{ github.actor }} --oci-password-variable GITHUB_TOKEN ghcr-buildcache
spack -e fairmq buildcache push --unsigned ghcr-buildcache
- name: Export lockfile
if: ${{ !cancelled() }}
shell: spack-bash {0}
run: |
mkdir -p test/ci/locks
cp "$(spack location -e fairmq)/spack.lock" \
"test/ci/locks/${{ matrix.env }}-gcc${{ matrix.gcc }}.lock"
- name: Upload lockfile
if: ${{ !cancelled() }}
uses: actions/upload-artifact@v4
with:
name: lock-${{ matrix.env }}-gcc${{ matrix.gcc }}
path: test/ci/locks/${{ matrix.env }}-gcc${{ matrix.gcc }}.lock
update-index:
if: github.repository == 'FairRootGroup/FairMQ' && !cancelled()
needs: build

21
test/ci/locks/README.md Normal file
View File

@@ -0,0 +1,21 @@
# Committed spack lockfiles
`setup-deps` installs from `<env>-gcc<N>.lock` here when a matching file exists,
skipping concretization so the resolved hashes match the binaries the buildcache
pushed (guaranteed cache hits). When no lockfile is present it falls back to
concretizing from `test/ci/spack-<env>.yaml`.
## Regenerating
Lockfiles pin host externals (glibc, system gcc), so they must be concretized on
the same runner image CI uses (`ubuntu-24.04`) — do **not** generate them on a
local machine.
1. Run the `Spack Buildcache` workflow (push to `dev`/`master` touching the spack
configs, or `workflow_dispatch`). It concretizes fresh (`fresh: 'true'`),
pushes binaries, and uploads each env's `spack.lock` as a `lock-<env>-gcc<N>`
artifact.
2. Download the artifacts and commit them here as `<env>-gcc<N>.lock`.
Regenerate whenever the specs in `test/ci/spack-*.yaml` change or the runner
image is bumped.