NeurIPS 2026 Workshop
Transitioning from Pre-Training to Post-Training
A workshop on the interaction between pre-training and post-training for LLM training. Key themes: (i) what a base model must possess for post-training to succeed, (ii) what are the roles and properties for each stage of training, and (iii) how do we predict success or failure.
- Date
- December 11, 2026
- Location
- Sydney, Australia
- Venue
- NeurIPS 2026 · OpenReview
Overview
Modern foundation models are built in stages: large-scale pre-training, followed by increasingly complex post-training: supervised fine-tuning, preference optimization, reinforcement learning, self-improvement, and (on-policy) distillation. Yet we lack a principled understanding of how these stages relate: what each stage is for, what a base model must provide for the next to succeed, and which steps are actually necessary. As instruction and reasoning data increasingly enters the pre-training mix, the boundary between “pre” and “post” is itself becoming blurry.
This workshop brings together theory, empirical evidence, and benchmarks to turn folklore about this pipeline into science.
Central questions
- What must a pre-trained model possess for post-training to succeed — and how do we even define “success”?
- How do different post-training procedures reshape the base model, beyond targeted benchmark gains?
- When and why does post-training fail, and can pre-training-side signals predict it?
- How do data and optimization choices govern the transitions between stages?