Molecular dynamics of AEP particles for prediction of crowd behavior

Ongoing research with the Klotsa group at UNC-Chapel Hill: molecular dynamics of active particles, run at scale, as a model for crowd behavior. The project isn’t finished and the physics hasn’t been through full rigor yet, so I’m holding the results back for now. That being said, I can still talk about the machinery behind the research, and I’d like to take some time to do so.

Scale

The defining constraint is volume. The workflow runs to hundreds of gigabytes across hundreds of thousands of files, against a finite allocation of compute and storage. The phase diagram above, a single artifact amongst many, is 25,000 simulations on its own, each about 15 minutes of compute. At that ratio, scheduling is not an afterthought to the science—how work is packed across the cluster is the difference between a result this week and a result next semester.

Where I’ve gotten

• Simulation. HOOMD-blue for the unit simulation, orchestrated across UNC’s Longleaf cluster with SLURM and Bash to push tens of thousands of parallel jobs through the queue. Vectorizing the core simulation code cut its runtime by roughly 5×, which compounds quickly at this scale.
• Pipeline. A central script executes a 10+ step routine end to end, branching on data conditions at each step rather than running a fixed sequence. It sits on a persistent node and delegates, spawning child scripts that submit their own job arrays, watch arrays already in flight, or run their own reduction and cleanup passes, each reporting back so the next branch has something to decide on. Jobs spawn jobs at varying depths, and the parent is responsible for knowing what finished and what silently didn’t. Handling that tree is the piece I’ve spent the most engineering effort on, but it has paid itself off by allowing me to drastically cut down on model development cycles, as it automates a lot of manual, time-consuming work.
• Data structures. The formats and layouts I settled on for storing and reducing simulation output have also helped a lot in speeding up the research. They were not designed in advance, but rather were adpated to fit our needs and obstacles as the project took shape.
• Physics. Phenomenological models/derivations to test candidate theories and fits against some simulation data.

Because this is still an early stage, the priority has been fast, dynamic implementation: a broad search over approaches to data management and analysis rather than an early commitment to one.

What’s next

More of the parameter space, refining the model, and enough checking to be confident in what the relationships actually are. It’s a bit like assembling a jigsaw puzzle without the picture on the box. I’ll write up the physics here and provide the official report once the project concludes.