When 1.58 Bits Beats 16: How Ternary Weights Are Rewriting the Mathematics of LLM Efficiency
The mathematics of neural networks has long been considered settled: gradients flow through continuous-valued weights, optimized via backpropagation through floating-point arithmetic. Yet in February 2024, Microsoft Research challenged this orthodoxy with a question that seemed absurd: what if every weight in a large language model could be expressed using only three values—{-1, 0, 1}? The answer, it turns out, rewrites everything we thought we knew about the efficiency-accuracy trade-off. BitNet b1.58, trained natively with ternary weights, matches full-precision LLaMA models in perplexity while consuming 90% less memory. QuEST demonstrates that LLMs can be trained stably at 1-bit precision. NanoQuant pushes further, achieving sub-1-bit compression that runs a 70B model on a consumer 8GB GPU. ...