Delta SyncHugging Face Blog
Shipping a Trillion Parameters With a Hub Bucket: Delta Weight Sync in TRL
Hugging Face Blog, May 27, 2026
Edward Beeching
I am a Machine Learning Research Scientist at Hugging Face. My recent work focuses on reinforcement learning and post-training for language models, open reasoning systems, and efficient training infrastructure.
Previously, I completed a PhD at INSA Lyon with the INRIA CHROMA team, where I studied deep reinforcement learning for planning, navigation, and embodied agents.
Research & Writing
My research spans language-model post-training, reasoning, embodied learning, and reinforcement learning. Publications and technical articles are listed together below.
2026
FinePhraseSynthetic pretraining data
How Can We Synthesize High-Quality Pretraining Data? A Systematic Study of Prompt Design, Generator Model, and Source Data
arXiv, 2026
A controlled study of design choices for generating synthetic language-model pretraining data.
QED-NanoOlympiad theorem proving
QED-Nano: Teaching a Tiny Model to Prove Hard Theorems
arXiv, 2026
An open 4B model and training pipeline for difficult Olympiad-level mathematical proofs.
Async RLHugging Face Blog
Keep the Tokens Flowing: Lessons from 16 Open-Source RL Libraries
Hugging Face Blog, March 10, 2026
2025
SmolLM3Hugging Face Blog
OlympicCoderHugging Face Blog
Open R1 #3Hugging Face Blog
Meta RLTest-time compute
Optimizing Test-Time Compute via Meta Reinforcement Fine-Tuning
ICML, 2025
A meta-reinforcement learning view of training models to use test-time compute efficiently.
Drug monitoringApplied machine learning
Applying Machine Learning to International Drug Monitoring: Classifying Cannabis Resin Collected in Europe Using Cannabinoid Concentrations
European Archives of Psychiatry and Clinical Neuroscience, 2025
Open R1 #1Hugging Face Blog
2024
ZephyrLanguage-model alignment
Zephyr: Direct Distillation of LM Alignment
Conference on Language Modeling (COLM), 2024
Distilling supervised and preference signals to align smaller open language models.
NuminaMathHugging Face Blog
JATHugging Face Blog
Jack of All Trades, Master of Some, a Multi-Purpose Transformer Agent
Hugging Face Blog, April 22, 2024
VLMsHugging Face Blog
JATA multi-purpose agent
Jack of All Trades, Master of Some, a Multi-Purpose Transformer Agent
arXiv, 2024
A single Transformer agent trained across text, vision, robotics, and gaming tasks.
Constitutional AIHugging Face Blog
Preference TuningHugging Face Blog
Preference Tuning LLMs with Direct Preference Optimization Methods
Hugging Face Blog, January 18, 2024
2023
LLM EvaluationHugging Face Blog
StarChatHugging Face Blog
StackLLaMAHugging Face Blog
TRL + PEFTHugging Face Blog
2022
Decision TransformerHugging Face Blog
Decision TransformerHugging Face Blog
2021
Godot Reinforcement Learning Agents
AAAI Workshop on Reinforcement Learning in Games, 2021
An open interface connecting the Godot game engine to common reinforcement-learning frameworks.
SmartGraphGameRLand3D
Graph Augmented Deep Reinforcement Learning in the GameRLand3D Environment
AAAI Workshop on Reinforcement Learning in Games, 2021
2020
Learning to Plan with Uncertain Topological Maps
ECCV, 2020 — Spotlight
Learning-based planning over uncertain graphs with visual node features.
EgoMap: Projective Mapping and Structured Egocentric Memory for Deep RL
ECML-PKDD, 2020
A spatially structured neural memory for embodied reinforcement-learning agents.
2014
3D SRMESeismic imaging
Enhancing 3D SRME to Stop Complex Continental Shelf Slope Topography Obscuring the Seismic Signal
76th EAGE Conference and Exhibition, 2014
Open Source
TRL
Post-training models with supervised fine-tuning, preference optimization, and reinforcement learning.
Open R1
An open reproduction of reasoning-model training recipes and datasets.
NuminaMath
Open models and 860,000 competition-mathematics problem-solution pairs.
JAT
A generalist Transformer agent for text, vision, robotics, and game environments.
Godot RL Agents
Tools for training reinforcement-learning agents inside the Godot game engine.
PhD Thesis
Large-Scale Automatic Learning of Autonomous Agent Behavior with Structured Deep Reinforcement Learning
University of Lyon / INSA Lyon, 2022. Supervised by Olivier Simonin, Christian Wolf, and Jilles Dibangoye.
Embodied Agent Demos
Behaviors learned with Godot RL Agents.
Image-goal navigation in a 3D scan without a provided map.
An agent learning to collect an ordered sequence of objects.