Machine learning for biology and chemistry.
Built from the bottom up: models that read DNA, a simulated cell for that DNA to live in, and then many cells together. Inside the simulation, DNA can be watched as it moves and copies, in detail that’s hard to measure in a lab. The aim is to decode what DNA carries and find new medicine.
Biology and chemistry
All →DNA
Models that read whole genomes, one letter at a time, and learn what tends to change and what doesn’t.
The cell
A cell simulated from physics and chemistry, built up from the bottom. It starts with a box of water.
- Learning to run a water simulation0.6 ps on the true path before the model drifts 0.1 nm awayRead
- A map of everything inside a human cell195 kinds of thing catalogued, from water to virusesRead
- A 3D simulator of living cellsA browser simulator where cells take in glucose, turn it into ATP and protein inside their own volume, grow, divide and starve. Version 0, built to grow.Read
Many cells
Cells signalling, competing and organising into tissue. The third step, once a single cell runs well.
Nothing here yet.
How they connect
Each level feeds the next. A simulated cell shows how DNA moves, folds and copies, which becomes training data for the DNA models, and what those models learn goes back into the cell. Cells that behave correctly on their own become the building blocks for simulating many at once.
- Giving a frozen LLM a memory+0.355 F1 lift from injection on an 8B model, 147 held-out questions (0.43 with memory, 0.08 without)Read
- Squeezing a prompt into a few tokens16 of 2,048 numbers free per soft token, and the model still stays within KL 0.22 of reading the full promptRead
- Neurons as spinning dials2.41 bits per byte on raw web text from a 52k-parameter dial modelRead
- A 336M language model from scratch0.989 bits per byte on held-out FineWeb-Edu after one pass over 1.8B tokensRead
- Learning without backprop2.8 bits per character, the spiking baseline the later variants were measured againstRead
- New architectures, small and fast2.023 bits per byte with multiscale positions, vs 2.037 for the plain 1-layer control, with 9% fewer parametersRead
- What one training step does0.04 average cosine between one gradient and the next on weight matrices (about 0.7 on the final norm)Read
- Learning the update rule2.89 bits per byte for the best learned optimizer, vs 2.04 for Muon on the same modelRead
- Parameter Golf, a 16 MB language model1.170 bits per byte on FineWeb val, but at 24.3 MB, over the 16 MB capRead
- When a small model stops memorizing99.5% exact answers on 5,000 never-seen addition problems, from 10k worked examplesRead
- Speech from scratchA tiny text-to-speech model trained on one voice, first predicting raw audio directly, then spectrograms through a pretrained vocoder. Alongside it, tests of open voice models and a trick to make one of them generate faster.Read
- Quantum circuits, from a Bell pair to next-byte prediction5.38 bits per byte from an 8-qubit circuit, against 3.82 for a plain count tableRead
About
Within The Matrix is the independent research of Slavi Savanovic, working toward one goal: simulating living cells well enough to understand disease from the bottom up and find new ways to treat it.
That means learning how DNA is read and copied, how a single cell responds to it, and eventually how cells behave together. The earlier machine learning work is where the tools for that were built.
Compute used
- Data center
- NVIDIA B300, B200, H100, A100, A10, AMD MI300X
- Workstation
- RTX 6000 Ada, RTX 4000 Ada, RTX 2000 Ada
- Consumer
- RTX 5090, RTX 5080, RTX 4090
- Laptop
- MacBook Pro