A model from Moonshot AI—Kimi K3, with 2.8 trillion parameters—runs on a standard machine with just eight gigabytes of RAM. The claim is true, and it deserves scrutiny. But the headlines hide critical details about how it actually works and what that means in practice.
Developer FareedKhan-dev published on GitHub an inference engine written in C99, portable and self-contained, at just 176 kilobytes. The engine reads Kimi K3 weights directly from SSD instead of storing them in RAM. Kimi K3, released by Moonshot on July 16, 2026, with open weights published on July 27, occupies 1.56 terabytes. Test results on eight gigabytes of RAM are byte-for-byte identical to a PyTorch reference: not an approximation, but exact computation with relocated storage.
The technology behind this is mixture-of-experts architecture. Kimi K3 has 896 experts, but only 16 activate per token. This means only a fraction of the 2.8 trillion parameters must fit in RAM; the rest stays on disk and loads on demand. It resembles how an operating system handles virtual memory: transparent and efficient.
But the phrase “runs on 8 GB” refers strictly to RAM, not storage. Storage requirements tell a different story. You need roughly 1.7 terabytes of SSD—1.56 terabytes for the checkpoint plus repacked weights. Mechanical hard drives would be too slow. And inference speed doesn’t rival cloud services. At the eight-gigabyte preset, generation reached around 32 seconds per token. The author candly acknowledges this is a teaching artifact and a correctness proof, not a practical daily driver.
Tests with more memory show the constraints. On 128 GB RAM, the same inference took about 10.69 seconds per token. Faster, yes, but not dramatic. Claims like “700% less hardware” are marketing language without rigorous measurement behind them.
The real merit of this project is demonstrating that a massive model is reproducible on modest hardware and that inference isn’t obligated to RAM. But readers who take social media headlines at face value risk a dangerous illusion: that this is a practical alternative to cloud APIs. It isn’t.
—
Cuvinte: 341
