Appearance
Bayesian Calibration of a Mass Flow Model
This study calibrates the parameters of a geophysical mass flow simulator, one that models natural hazards such as landslides and avalanches, using Bayesian inference against field observations. Because the forward simulator is expensive to run, calibration proceeds in two stages: a surrogate is trained on simulator runs first, then MCMC inference runs against the surrogate. Two collaborators built the study: a UQ-methods researcher on Windows, working in Python and R, who wanted to apply surrogate-based calibration to a real use case with an expensive simulator; and a computational scientist and RSE on Linux, experienced with high-fidelity mass flow simulators, who builds and maintains the study.
The study spans four units across three language ecosystems. A Latin Hypercube Sampling unit in Julia generates a parameter design. The forward simulator, in Python, evaluates each input configuration from that design and returns model responses. A surrogate training unit in R fits a Gaussian process emulator to the design and its responses. A surrogate serving unit, also in R, exposes the trained emulator's predictions over HTTP, and an MCMC inference unit in Python queries it in an online loop to infer the model's parameters.
Patterns
The Latin Hypercube design, forward simulator, and surrogate training unit form Pattern A: many input configurations evaluated and fit into one emulator. The surrogate serving unit and MCMC unit then form Pattern C, an online loop where the MCMC unit queries the surrogate server directly. See Workflow Patterns for how these compose.
Recommendations applied
Each unit is isolated in its own conda environment (Iso1) and additionally containerised with Apptainer for HPC (Iso2, Iso3), so no unit code needs modification to run there.
Latin Hypercube samples and forward-model responses are exchanged as CSV files; MCMC output is written in HDF5. Both are language-agnostic formats exchanged through the filesystem, bridging the Julia, Python, and R units across the pipeline (Int1, Int2). The surrogate unit is itself split into a training sub-process and a serving sub-process, so the emulator need not be retrained on every run; they exchange the trained model as an RDS file. The surrogate server exposes predictions through the UMBridge interface, and the Python MCMC client queries it without any language bridging (Int3).
The full workflow is automated in Nextflow (Orch1), which sequences the two stages so MCMC calibration starts only once the surrogate is trained (Orch2), and skips completed simulator runs on rerun, so an interrupted run need not repeat the expensive simulation stage (Orch3). The surrogate server signals readiness by writing its network port to a sentinel file; the MCMC client connects once that file appears (Orch4).
This study is where the sentinel-file pattern for coordinating service processes (Orch4) was first identified. An R implementation of the UMBridge interface didn't exist yet, so it was implemented for this study, with an upstream contribution underway. The study has run end-to-end on Windows and Linux laptops and on a local Linux cluster.
Repository
The study is available at thealanjason/bpc_massflow.