How DeepSeek-R1 Learned to Think: The GRPO Algorithm Behind Open-Source Reasoning Models
On January 20, 2025, DeepSeek released R1—a 671B parameter Mixture-of-Experts model that achieved something remarkable: matching OpenAI’s o1 on reasoning benchmarks while being fully open-source. The breakthrough wasn’t just in scale or architecture, but in a fundamentally different approach to training reasoning capabilities: Group Relative Policy Optimization (GRPO), a reinforcement learning algorithm that eliminates the need for reward models while enabling sophisticated reasoning behaviors to emerge naturally. The Problem with Traditional LLM Training Standard large language models excel at pattern matching and next-token prediction, but struggle with tasks requiring multi-step logical deduction, self-correction, and complex problem decomposition. Chain-of-thought prompting helped, but it required extensive human-annotated demonstrations and still couldn’t match the systematic reasoning humans employ. ...