Aligning Large Language Models with Human Preferences: DPO, PPO, and GRPO in Practice
From Pre-training to Post-training
Large language models are usually built in two major stages:
Pre-training: The model learns to predict the next token given a prefix, acquiring broad linguistic competence.
Post-training: After the model can "speak", we teach it to "think". This stage splits into
Supervised Fine-Tuning (S ...
Posted on Wed, 16 Sep 2026 16:42:31 +0000 by tequila
Designing Reward Functions in Open R1: Code Execution Verification and Reasoning Process Evaluation
Architecture of the Reward System
The Open R1 project, a fully open-source reproduction of DeepSeek-R1, relies heavily on a sophisticated reinforcement learning pipeline. The core of its performance lies in the precise construction of its reward mechanisms. The project implements a multi-faceted scoring system within its training stack—specific ...
Posted on Wed, 05 Aug 2026 16:53:56 +0000 by hamishrock