Research – Foundations

Neurons as spinning dials

Language models where each neuron is a dial that turns, and neurons pull on each other like coupled clocks instead of passing numbers through weight matrices. Trained on raw FineWeb bytes, at 15k to 62k parameters.

StartedJun 6, 2026
Last updateNov 15, 2026
bits per byte on raw web text from a 52k-parameter dial model2.41

Question

Can memory live in angles instead of numbers? A dial that only rotates never fades, so a line of dials could hold context without decaying. The question is whether neurons built this way can learn to predict text at all, and how close they get to ordinary networks of the same size.

Approach

A family of small models, each named “something-line”, all trained to predict the next character or byte of FineWeb text.

Model Dates Idea
spinline Jun 6 A line of dials, each with its own clock speed. Every character turns every dial by a learned amount. Memory is the accumulated angle.
coupline Jun 7 – Nov 1 A memory line plus a “char neuron”: a fresh set of dials born from the current character. The two pull on each other’s angles for a few rounds (Kuramoto coupling, where each dial drifts toward or away from the others), then the char neuron’s pattern is matched to the nearest prototype to pick the next character.
scanline, coupgrid, coupens Jun 7 Variants: a read head that walks the line dial by dial; several lines side by side with neighbour-only links; an ensemble where each line makes its own prediction.
circline, fastline, vecline Jun 10 A detour to rings of ordinary neurons with a “notepad” memory. In fastline and vecline the notepad is a matrix written during the run, so cross-dimension wiring comes from the data, not from training.
keepline Oct 24 coupline, but the char neuron is carried from one character to the next instead of born fresh.
gructrl Oct 24 A plain GRU (a standard recurrent network) sized to match any coupline config by parameter count. The control.
sphereline Oct 25 Dials become needles on a sphere (3D unit vectors) instead of on a circle, so a pull has a direction, not just an amount. Adds raw-byte input, an 8-bit readout, and stacked layers.
attnline Nov 14–15 sphereline plus switches borrowed from attention and from quantum computing: selective reads, selective writes, a key–value store, slow layers, sparse writes, rigidly carried “twin” dials, and “entangled” lines. With every switch off it is sphereline exactly.

The sphere runs use 40.7M bytes of FineWeb, predicted byte by byte (256 possible values). Windows are 1,024 bytes. Early sphere runs were on an RTX 5090 or similar. The attnline sweeps ran on a 48 GB A6000, 3,000 steps each at batch 64, about 1.5 to 2.2 hours per run.

Bits per byte is how surprised the model is by each next byte; lower is better. Uniform guessing over 256 values is 8.

Results

Sphere dials on raw bytes (validation, 20-batch final eval)

Run Params Setup Steps Bits/byte
Byte table input 55,520 16 line dials, 64 char dials, 3 rounds 50,000 2.592
Bit readout 15,584 Same sizes, byte read in and out as 8 bits, ~16x smaller tables 30,000 (stopped) 2.754 (last val)
Batch ramp 64 to 256 52,224 48 line dials, 64 char dials, 5 rounds, readout neurons 10,000 2.411

attnline switches (same 52k base, same batches, 3,000 steps)

Variant Params Bits/byte vs base
Base (all switches off) 52,224 2.473 –
Wider line, 52 dials (param control) 55,824 2.459 −0.014
Selective read 52,228 2.463 −0.010
Selective write 56,448 2.462 −0.011
Key–value slots 56,321 2.470 −0.003
Twin dials (carried, never pulled) 52,804 2.473 0.000
Entangled lines (no new params) 52,224 2.441 −0.032
4D dials 54,208 2.482 +0.009
6D dials 58,176 2.529 +0.056
8D dials 62,144 2.560 +0.087
Slow layers (layer l ticks every K^l bytes), K = 2 / 4 / 8 52,224 2.730 / 2.774 / 2.783 +0.26 to +0.31
Sparse writes, 1 tick in 4 / 8 52,224 3.044 / 3.215 +0.57 / +0.74
Slow 4 + sparse 8 52,224 3.282 +0.81

Checks on trained models (load weights, evaluate only)

Check Base Entangled
No change 2.505 2.477
Memory wiped every 256 bytes 2.521 2.494
Memory wiped every 64 bytes 2.586 2.563
Entanglement switched off – 4.256
Entangled dial pairs shuffled – 4.875

These checks use a different eval draw than the training-end numbers above, so compare within the table only.

Where in the window the bits go (base model): 2.545 for bytes 0–128, 2.491 for 128–512, 2.481 for 512–1,024.

Findings

  • Dials can learn text. A 52k-parameter sphere model reaches 2.41 bits per byte on raw bytes, and it does use context: wiping its memory every 64 bytes costs about 0.08 bits per byte, and later bytes in a window are predicted better than early ones.
  • Entangled lines were the only switch to clearly beat the param-matched control (2.441 vs 2.459 for a wider line), with no extra parameters. The trained model leans on the links hard: cutting them or shuffling the pairing sends it to 4.3 and 4.9.
  • The attention-style switches (selective read, selective write, slots) each moved the score by about 0.01 at most, no better than just adding dials. Twin dials changed nothing.
  • More dimensions per dial made things worse: 3D beat 4D, 6D and 8D. A 2D (circle) run in the same sweep went to NaN.
  • Forcing structure in time hurt badly. Slow upper layers cost about 0.3 bits per byte and sparse writes cost 0.6 to 0.8. Every dial needs to be writable on every byte at this size.
  • In quantum-computing terms, a coupline dial is a qubit stuck on the equator: phase only, with the readout acting like an overlap between two states. The pulling that settles the dials has no quantum counterpart, since quantum gates only rotate and never settle.

Next

  • A precession switch (a turn around the pull direction, the way two real spins act on each other) ran out of GPU memory and needs a smaller run.
  • Run to the end: a protected 16-dial register, three-twin frames, entanglement passing downward, and a learned entanglement strength.
  • Run the GRU control and a small transformer on the same bytes, at the same parameter count, so the dial models have a fair baseline.