I am a fourth-year Ph.D. candidate in the Department of Computer Science at Rutgers University, advised by Professor He Zhu. Prior to Rutgers, I received my B.E. degree in Artificial Intelligence at UESTC. In summer 2026 I was a Machine Learning Scientist Intern at TikTok in San Jose, CA.
My research asks a single question: how can an agent learn a task quickly when the reward is sparse? In a sparse-reward environment the agent sees almost no signal until the whole task is solved, so exploration, credit assignment and curriculum become the bottleneck rather than optimization. I attack this from two directions that turn out to be the same problem: goal-conditioned and model-based RL for embodied agents, and RL post-training for large models, where a long reasoning trajectory is likewise graded by one outcome-level reward at the very end.
Research Interests
- Sparse-Reward RL & Agent Learning: goal-conditioned RL, goal-directed exploration, model-based RL and world models, learning from demonstrations, automatic curriculum and subgoal discovery.
- RL Post-Training for LLMs: SFT and RL post-training with PPO, GRPO and preference-based objectives for reasoning; reward design and credit assignment under long-horizon, outcome-only supervision. I have post-trained open-source models with SFT / PPO / GRPO on mathematical reasoning and on agent benchmarks such as AppWorld and AndroidWorld.
- Vision-Language Models & Embodied Agents: VLMs for robotics, VLMs for RL, large world models, and program synthesis for interpretable robot policies.
- Machine Learning in Healthcare: survival and causal modeling (Cox regression, DID, ISM, etc.), and interpretability.
Publications
-
Preference-based Policy Optimization from Sparse-reward Offline Dataset
Wenjie Qiu, Guofeng Cui, Shicheng Liu, Yuanlin Duan, He Zhu.
International Conference on Learning Representations (ICLR), 2026
[Paper] [Code] -
Learning from Demonstrations via Capability-Aware Goal Sampling
Yuanlin Duan, Yuning Wang, Wenjie Qiu, He Zhu.
Neural Information Processing Systems (NeurIPS), 2025
[Paper] [Code] -
Abstraction Refinement-guided Program Synthesis for Robot Learning from Demonstrations
Guofeng Cui, Yuning Wang, Wensen Mao, Yuanlin Duan, He Zhu.
Proceedings of the ACM on Programming Languages (OOPSLA), 2025
[Paper] -
Learning World Models for Unconstrained Goal Navigation
Yuanlin Duan, Wensen Mao, He Zhu.
Neural Information Processing Systems (NeurIPS), 2024
[Paper] [Code] -
Exploring the Edges of Latent State Clusters for Goal-Conditioned Reinforcement Learning
Yuanlin Duan, Guofeng Cui, He Zhu.
Neural Information Processing Systems (NeurIPS), 2024
[Paper] [Code] -
MoE-I²: Compressing Mixture of Experts Models through Inter-Expert Pruning and Intra-Expert Low-Rank Decomposition
Cheng Yang, Yang Sui, Jinqi Xiao, Lingyi Huang, Yu Gong, Yuanlin Duan, Wenqi Jia, Miao Yin, Yu Cheng, Bo Yuan.
Findings of the Association for Computational Linguistics: EMNLP 2024
[Paper]
Experience
- TikTok, San Jose, CA, USA
Machine Learning Scientist Intern, May 2026 – Aug 2026 - General Medical Center of West China Hospital, Chengdu, China
Research Assistant, Jun 2022 – Aug 2023
Teaching
- Spring 2025 - Teaching Assistant, CS336: Principles of Information and Data Management
- Fall 2024 - Teaching Assistant, CS336: Principles of Information and Data Management
- Spring 2024 - Teaching Assistant, CS210: Data Management for Data Science
- Fall 2023 - Teaching Assistant, CS210: Data Management for Data Science
Service
- Reviewer: NeurIPS 2025, NeurIPS 2026.
Personal
- I am very happy to communicate with others about everything related to life, academics, and entrepreneurship.
- I am a follower of value investing and welcome discussions about perspectives on great businesses.
- I'm a badminton beginner and would love to find someone of a similar skill level to play with!