Iso4 >>> Automation

Recommendation. Automate environment maintenance with continuous integration: regenerate per-OS, per-architecture conda-lock files on every change to an environment.yml, and rebuild and publish multi-architecture container images whenever a Dockerfile changes. The maintenance burden moves from individual collaborators to one automated pipeline.

When this applies

As soon as more than one person, or more than one platform, depends on a unit's environment. Lock files times operating systems times architectures, plus container images times architectures, is combinatorial work; doing it by hand is tolerable for one unit on one platform and stops scaling immediately after that. The most common failure is not doing the work badly but forgetting it entirely: an environment.yml changes and its lock files silently go stale.

Why

Continuous integration (CI) means a service, such as GitHub Actions, runs defined jobs automatically on every change to the repository. Regenerating locks and rebuilding images is mechanical and platform-bound, which makes it exactly the kind of work CI is for: a matrix build runs the same job across several operating systems in one go, on machines nobody has to own. What lands in the repository is then always current, and a new collaborator gets working environments from a plain clone.

In practice

For lock files, a matrix job regenerates the locks whenever the environment specification changes. With GitHub Actions:

name: relock
on:
  push:
    paths: ["**/environment.yml"]

jobs:
  lock:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: mamba-org/setup-micromamba@v2
        with:
          environment-name: lock
          create-args: conda-lock
      - run: >
          conda-lock lock --file environment.yml
          --platform linux-64 --platform osx-arm64 --platform win-64
        shell: micromamba-shell {0}
      - uses: peter-evans/create-pull-request@v7
        with:
          title: "Update conda-lock files"
          branch: relock

conda-lock can resolve all platforms from a single Linux runner, so one job suffices; the result comes back as a pull request rather than a direct push, so a human still sees what changed in the resolution. For container images, a second workflow triggers on Dockerfile changes and pushes a multi-architecture build:

name: image
on:
  push:
    paths: ["Dockerfile", "src/**"]

jobs:
  build:
    runs-on: ubuntu-latest
    permissions:
      packages: write
    steps:
      - uses: actions/checkout@v4
      - uses: docker/setup-qemu-action@v3
      - uses: docker/setup-buildx-action@v3
      - uses: docker/login-action@v3
        with:
          registry: ghcr.io
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}
      - uses: docker/build-push-action@v6
        with:
          platforms: linux/amd64,linux/arm64
          tags: ghcr.io/your-org/forward-model:${{ github.sha }}
          push: true

The same pipeline can append the Apptainer conversion from Iso3, so a ready .sif is published alongside the Docker image.

There is also a useful middle ground between a bare conda environment and a hand-crafted Dockerfile: wrap a locked conda environment in a minimal container image.

FROM mambaorg/micromamba:2.0
COPY conda-lock.yml /tmp/conda-lock.yml
RUN micromamba install --name base --file /tmp/conda-lock.yml && \
    micromamba clean --all --yes

This combines conda's resolution of language-level dependencies with the container's control over the system layer, in one artefact built by the same CI pipeline. The workflow managers can even produce such images for you: Snakemake's --containerize flag emits a single Dockerfile bundling a workflow's conda environments, and Nextflow's Wave service builds an image on demand for each declared conda environment. Those conveniences are good to know about, though a per-unit image built in our own CI keeps us in control of when and how images change.

Related

What gets automated here is defined in Iso1 (lock files), Iso2 (images), and Iso3 (the .sif conversion). The GPE benchmarking use case runs exactly this setup, with CI regenerating locks on every environment change.