Kaili Huang
LLM post-training & reinforcement learning
-
Tsinghua2016 – 2020- CrossWOZ, KdConv, LCCC, CDial-GPT: dialogue datasets and models people still use, with Minlie Huang
- 3,000+ GitHub stars; NLPCC 2020 Best Student Paper
-
Stanford2019 · 2021 – 2023- ColBERT-Serve with Matei Zaharia (Databricks co-founder) and Christopher Potts
- RL for dialogue with Tengyu Ma; CS224N TA with Christopher Manning; wrote the Transformer notes
-
ByteDance AI Lab2020 – 2021- Fake-news and harmful-content detection at scale, with Hang Li
- 2M+ human labels with guidelines I iterated alongside the labeling team; automated retraining and rollout
-
Microsoft AI2023 – 2026- Evaluation and alignment loop for Copilot; co-authored DeepThink on preference alignment
- Post-training for multimodal foundation models
-
Apple2026 –- LLM post-training and RL: reward design and policy optimization
- CoGR: query- and item-side LLMs that co-evolve through SFT and alternating GRPO against a shared retrieval reward
I work on post-training and reinforcement learning for large language models. The parts I care about most are the ones that decide whether a model actually got better or only looks like it did: the reward that says what good means, the data the policy learns from, the evaluation that would catch it lying, and the training loop that holds the three together. At Apple I do this for LLM retrieval agents, from SFT through GRPO; I recently co-developed CoGR, which trains two LLMs to co-evolve against a single shared retrieval reward.
I got here by way of dialogue. At Tsinghua I co-created the Chinese dialogue datasets people still train on: CrossWOZ 
ideas I keep returning to
what it taught me about agency
Today's agents are good at pushing forward. What I want from the next generation is the judgment to stop: to step back and ask whether the whole approach is right, to change the architecture rather than the next line, and to reason from first principles: the goal is not to solve every problem it is handed, nor to grind on a particular one, but to keep the capacity to ask which problem is the right one. Most of my rule-setting in the experiment loop is an attempt to supply that judgment from the outside.
the longer bet
This is a long bet, measured in decades rather than product cycles. The hard parts are three: model capability, latency low enough that the system feels like part of you rather than a tool you wait on, and the interaction itself, how such a thing is built and wired into a life. It does the arithmetic, looks things up seamlessly as the conversation happens, remembers what you would otherwise have to memorize, helps you plan your life without the overhead, and clears away the tedious work that fills a day. You still do the thinking and make the decisions. It just makes sure nothing trivial is in the way. The measure of such a system is not how smart it sounds but how much clearer the person thinks, and how much better they decide, with it than without it. Underneath all of it is a simple belief: AI should serve people. Being free to do what you love and create what you want to create is one of the most valuable things a person could have. A system like this is worth building if it gives more people more of that freedom.
