Research – Biology and chemistry

Learning to run a water simulation

A transformer that learns to advance a molecular simulation of 512 water molecules, one step at a time, and is checked against the real physics engine it learned from.

StartedNov 10, 2026
Last updateNov 16, 2026
on the true path before the model drifts 0.1 nm away0.6 ps

Question

Can a neural network learn to move a box of water forward in time, so that it behaves like water when it runs on its own? A molecular dynamics engine does this by computing every force at every 2 femtosecond tick. A learned “jump model” could be a cheaper stand-in. Water comes first because it is most of a cell. This is a first step toward simulating a cell from physics and chemistry, not a cell simulator.

Approach

The physics engine was written first, in four steps that each add one thing: one particle drifting in water (Langevin dynamics, a random-kick model of diffusion), a box of rigid three-atom water molecules, one calcium ion in that water, then calcium with two chloride ions. A fused CUDA version of the same physics produces the training data.

The model gets one token per water molecule. Each token attends to its nearest neighbours (32 at first, later 100), and everything it sees is relative: atom offsets from the molecule’s centre, velocities, and vectors to neighbours. No absolute coordinates.

System 512 water molecules at 25 C in a box that repeats in every direction
Saved data 8 training and 2 validation runs of 100 ps, one snapshot every 10 fs (later replaced by data streamed live from the engine)
First models 3.7M parameters, width 256, 6 layers, one jump = 10 fs or 100 fs
Tick models 10.9M parameters, width 384, 8 layers, one jump = one 2 fs engine tick, predicting only the change in velocity
Optimizer Muon (adjusts whole weight matrices at once) on the hidden layers, AdamW on the rest
Hardware one NVIDIA GPU in a cloud pod
Loss scale 1.00 = no better than letting every atom coast at its current velocity

Two short-term scores are tracked. Validation loss says how good one jump is. A free run says what happens when the model feeds on its own output for thousands of jumps: temperature, diffusion (how fast molecules wander, in µm²/s) and the spacing between neighbouring oxygens, all next to the real engine.

Results

Big jumps, 10 fs and 100 fs (14–15 Nov). One-jump accuracy came fast. Free runs did not hold.

Run Val loss Free run, temperature (real 23–25 C) Diffusion (real ~2,570)
10 fs, first model 0.081 247 C 526,275
10 fs, Muon optimizer 0.0048 −102 C 105
10 fs, 4 jumps in a row in training – −121 C 1,442
10 fs, sees 5 past snapshots 0.0022 −146 C 131
same, with a strong heat bath (0.05 ps) – 22 C 2,719
10 fs, sees 10 past snapshots 0.0014 −75 C (10 ps check) 725
100 fs, regression (one best guess) 0.029 −251 C 2
100 fs, generative (flow, 8 passes per jump) – blew up after 6.1 ps –

A long run (6.4 hours, 160,000 steps in four phases, training on 1 to 4 jumps in a row) never produced a free run warmer than −17 C, and one checkpoint blew up. With a strong heat bath it mostly held 23–25 C.

Fixing the energy (15 Nov). All runs below start from the same long-run checkpoint and train 10,000 more steps. “Free” is a 10 ps run with no help. “Energy snap” rescales speeds after each jump so the box’s total energy stays where it started.

Run Free, last check Energy snap, last check
control, no energy term −60 C, diffusion 801 27 C, 2,563
penalty on motion energy (weight 100) −8 C, 1,574 –
penalty on motion + stored energy −5 C, 1,463 –
snap used during training blew up at 5.2 ps 23 C, 2,529
trained on its own states, engine as the answer key −58 C, 885 34 C, 2,790
current forces given as input −88 C, 541 30 C, 3,003

The checks swing by tens of degrees between checkpoints of the same run (the motion-energy run read +42 C one check before −8 C), so none of these is a clean win.

Masked model (15 Nov). Trained from scratch to fill in hidden snapshots in a 32-snapshot window (4.0M parameters, stopped at 50,500 of 60,000 steps). Filling one gap in the middle was easy (loss 0.002). Predicting the next snapshot (0.015) was ten times worse than the dedicated 10 fs model, and free runs ranged from −178 C to +130 C.

One tick at a time (15–16 Nov). The jump shrank to one 2 fs engine tick, where the job is essentially learning the forces.

Run What changed Stays within 3 C of the engine (of 100 ticks)
tick sees 10 past ticks free runs blew up at 2–3 ps
tick8 trained on 8 ticks of its own output 5 at start, then 7–68
tick8e plus a temperature penalty (weight 100) slid from 52 to 9 in 175 steps; the same start without the penalty held 33–59
now sees only the current snapshot up to 55
pairs also gets atom-to-atom vectors shaped like force terms up to 93; loss 0.0063 vs 0.0130 for “now” at step 4,800
long100 trained on 100 ticks of its own output 51–81
own, own2 trained on its own runs up to 500 ticks deep, graded by the engine from those exact states 78–100, then 100 at every check
deep 100 neighbours, own runs up to 5,000 ticks deep 86–100

In the engine-graded runs, none of the model’s own boxes went bad, including deep runs averaging 1,400 to 2,800 ticks.

Long free runs, 50 ps = 25,000 ticks, no guardrail (16 Nov). Reference is the engine in double precision.

Checkpoint Temperature, second half Diffusion Neighbour spacing Total energy drift
real engine 26.2 C 2,384 0.273 nm –
own2 86.3 C 5,415 0.278 nm +16.0%
100 neighbours, before deep training 316.3 C 12,657 0.278 nm +72.9%
deep 59.6 C 3,585 0.278 nm +9.9%

Stray test. The model and the engine start from the same state and run side by side. The time until the model’s atoms are 0.1 nm (about a third of the gap between neighbouring molecules) off the engine’s path:

Checkpoint Time on the true path
pairs (5k, 10k, 15k steps) 0.52–0.55 ps
long100 0.63 ps
own (3 checkpoints) 0.62–0.65 ps
own2 0.63–0.64 ps
deep 0.62 ps

Findings

  • Water is chaotic. Two runs of the real engine started with velocities nudged by one part in a million also drift apart. That sets a ceiling: no model can track the exact path longer than the engine tracks itself. So the test that matters is whether the model’s water stays water (temperature, diffusion, structure), not whether it follows the same atoms.
  • The stray time stopped improving at about 0.6 ps while training kept improving everything else, which fits a chaos ceiling.
  • Low one-jump loss did not mean a stable run. At 10 fs a model that explained 99.9% of what coasting misses still froze the box to around −75 C on its own.
  • At 100 fs, the regression model averaged over possible futures and drained the motion out (−251 C). The generative version kept the motion but blew up within about 6 ps.
  • Penalising energy errors in the loss did not fix drift and sometimes hurt. What worked was showing the model the states it gets itself into and asking the engine for the right answer from there.
  • Liquid structure was the easiest part. Neighbour spacing came out within 0.005 nm of the engine in every run, even ones at the wrong temperature. The remaining problem is a slow heat gain, about 10% in total energy over 50 ps in the best model.

Next

  • A size test is queued: the same single-tick training with width 384 and 8 layers, 384 and 16, and 768 and 8.
  • Faster training code is being checked against the plain version so that it gives the same numbers.
  • Close the energy drift so 50 ps free runs hold 25 C without a guardrail.
  • The engine already makes calcium and calcium–chloride boxes. The model has only seen pure water so far.