A 125-million-parameter transformer whose dimensions are drawn from the Fibonacci sequence — a testable hypothesis about whether golden-ratio sizing changes how efficiently a model learns.
Fibonacci ratios show up in phyllotaxis, shell growth, and spiral galaxies as a by-product of efficient, self-similar packing under simple growth rules. That's a fact about physical systems, not about neural networks.
ΦAI-125M asks a narrower, answerable question: if a transformer's embedding size, depth, and context window are chosen along the Fibonacci sequence instead of the usual powers of two, does training behave any differently — in loss curves, sample efficiency, or generalization — at matched parameter count? That's an empirical claim. It stands or falls on a training run against a same-size baseline, not on the elegance of the numbers.
Every structural dimension below is a Fibonacci number, indexed against the standard sequence (1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, 233, 377, 610, 987, 1597, 2584 …).
| Parameter | Value | Fibonacci index |
|---|---|---|
| Embedding dimension | 610 | F(15) |
| Transformer layers | 13 | F(7) |
| Attention heads | Fibonacci-scaled per layer | F(n) by depth |
| Context length | 2,584 tokens | F(18) |
| FFN expansion ratio | φ (1.618…) | golden ratio |
| Total parameters | ~125M | — |
| License | Apache 2.0 | — |
nn.Module — is the actual next milestone, not a detail to wave past.The golden spiral below is constructed the standard way — squares sized by consecutive Fibonacci numbers — with the model's own dimensions labeled onto the squares they correspond to. It's a way to see the sizing choice, not evidence for the hypothesis it's testing.
square sizes are illustrative proportions, not literal pixel-to-parameter scale
Pre-training has not yet started. This document exists so anyone — a researcher or another model reading it — can reconstruct the architecture and check the reasoning without needing anything beyond what's written here.