About
I am a Tenured Associate Professor in the School of Software at Tianjin University (TJU). During my Ph.D. studies, I was advised by Prof. Jianye Hao. I received my Bachelor's degree from Tianjin University as well.
My research centers on Reinforcement Learning, with applications expanding into Embodied Intelligence, 3D Modeling and Assembly Manufacturing, Video Generation, and Post-Training of Large Models. Although each domain presents its own distinct challenges, these areas share deep underlying connections.
I firmly believe in reinforcement learning and regard reinforcement as the core of intelligence.
Open Positions
I am always looking for motivated students and researchers to join the group. All positions involve an interview process. More information is available on our team website.
- M.S. students: self-motivated, eager to learn, and willing to communicate and collaborate.
- Ph.D. students: applicants with a solid research foundation, preferably with one or more publications in top-tier venues.
- Postdoctoral researchers and research assistants: opportunities are available and can be discussed in detail.
- Research interns, research assistants, and postdoctoral researchers: positions are available. Please feel free to contact me if you are interested.
Research Interest
Reinforcement Learning
Evolutionary RL
Multi-Agent RL
Multi-Objective RL
Continual RL
RL for Optimization
Reinforcement Learning for Embodied AI
Real-robot RL
Dexterous Hand Control
Sim2Real
Differentiable Simulation
Automated Simulation
World Models for Embodied AI
Automatic Reward
Agent
Agent for EDA
Agent for Video Generation
Agent for CAD
Agent for Embodied AI
Agent Self-Evolution
Agent RL Post-Training
News
Sep 24, 2026
Four papers accepted by NeurIPS 2026Conference
Jul, 2026
Invited talk: Research and Applications of Evolutionary Reinforcement Learning Algorithms @ National University of Defense Technology (NUDT)Talk
Jul, 2026
Invited talk: Research and Applications of Evolutionary Reinforcement Learning Algorithms @ Xiangtan UniversityTalk
Jun, 2026
Joined the School of Software at Tianjin University as an Associate ProfessorAppointment
May 8, 2026
Selected as the Outstanding PhD Graduate Representative of Tianjin UniversityAward
May 1, 2026
Three papers accepted by ICML 2026: one Spotlight and two PostersConference
Jan 27, 2026
Two papers accepted by ICLR 2026Conference
Jan 9, 2026
Invited to Didi Future Elite MeetupEvent
Jan, 2026
Received the JD TGT Talent Program award on Embodied AI InfrastructureAward
Dec 19, 2025
One paper accepted by AAMAS 2025 (Oral)Conference
Dec, 2025
Received the Meituan Beidou Talent Program award on LLM Agents and RL Post-trainingAward
Oct, 2025
Received the Didi Future Elite Talent Program award on Reinforcement Learning and Autonomous DrivingAward
Oct, 2025
Passed the ByteDance Jindouyun Talent Program on AIGCAward
Oct, 2025
Received the National Scholarship for Doctoral StudentsAward
Sep 26, 2025
Three papers accepted by NeurIPS 2025Conference
Jun 17, 2025
EmbodiedScene received the Best Paper Award at ICML 2025 World Model WorkshopAward
May 1, 2025
Two papers accepted by ICML 2025Conference
Jan 22, 2025
One paper accepted by ICLR 2025Conference
Nov 2, 2024
Invited talk: Population-based Multi-agent Collaboration @ DAI2024 ConferenceTalk
Sep 26, 2024
One paper accepted by NeurIPS 2024Conference
May 2, 2024
Two papers accepted by ICML 2024Conference
Jan 15, 2024
One paper accepted by ICLR 2024Conference
Dec 3, 2023
Invited talk: Mutual Information based Multi-agent Reinforcement Learning @ DAI2023 ConferenceTalk
May 10, 2023
One paper accepted by ICML 2023Conference
Jan 20, 2023
One paper accepted by ICLR 2023Conference
Apr, 2022
One paper accepted by ICML 2022Conference
Jan 20, 2022
One paper accepted by ICLR 2022Conference
Awards and Honors
Prestigious Forum, Scholarship & Projects
New Cornerstone 50 Square Forum (NSFC)
Only 25 PhD students nationwide selected; the only PhD student from Computer Science
NSFC Youth Project for Doctoral Researchers (First Batch at TJU)
Focus: Embodied Intelligence Research
National Scholarship for Doctoral Students
Two consecutive years (2024-2025)
Tianjin Municipal Government Award
Talent Programs
JD TGT Talent Program
Focus: Embodied AI Infrastructure
Didi Future Elite Talent Program
Focus: Reinforcement Learning and Autonomous Driving
Meituan Beidou Talent Program
Focus: LLM Agents and RL Post-training
ByteDance Jindouyun Talent Program
Focus: AIGC
Competitions
Silver Award, National "Internet+" Innovation Competition
Third Prize, Intel Cup
Industry Experience
Noah's Ark Lab (Internship)
NetEase Games (Internship)
Publications and Preprints
Papers are sorted by recency. Colors indicate research areas: reinforcement learning embodied AI agents
WarpSAC: Towards the Pinnacle of Scalable Off-policy RL by Rethinking Exploration and Exploitation
Zihao Wu, Hongyao Tang, Yi Ma, Huizhong Song, Pengyi Li, Yifu Yuan, Fei Ni, Jinyi Liu, Wei Wei, Jianrong Wang, Yan Zheng, Jianye Hao
arXiv preprint, 2026
Paper Project Code
Zihao Wu, Hongyao Tang, Yi Ma, Huizhong Song, Pengyi Li, Yifu Yuan, Fei Ni, Jinyi Liu, Wei Wei, Jianrong Wang, Yan Zheng, Jianye Hao
arXiv preprint, 2026
Paper Project Code
Guiding Team Objectives with Individual Policies: A Two-Stage Model Aggregation Framework for Partially Cooperative MARL
Wanting Liu, Baoxi Wang, Pengyi Li, Lu Jiang, Chengwei Zhang
International Joint Conference on Artificial Intelligence (IJCAI), 2026
Paper Code
Wanting Liu, Baoxi Wang, Pengyi Li, Lu Jiang, Chengwei Zhang
International Joint Conference on Artificial Intelligence (IJCAI), 2026
Paper Code
Neuro-evolutionary Continual Reinforcement Learning
Pengyi Li, Hongyao Tang, Yifu Yuan, Yan Zheng, Xin Xu, Jianye Hao
International Conference on Machine Learning (ICML), 2026 Spotlight
Link Code
Pengyi Li, Hongyao Tang, Yifu Yuan, Yan Zheng, Xin Xu, Jianye Hao
International Conference on Machine Learning (ICML), 2026 Spotlight
Link Code
TACTIC: Task-Aware Sparse Coordination Graphs for Multi-Task Multi-agent Reinforcement Learning
Kexing Peng, Pengyi Li, Tinghuai Ma, Jianye Hao
International Conference on Machine Learning (ICML), 2026 Poster
Link
Kexing Peng, Pengyi Li, Tinghuai Ma, Jianye Hao
International Conference on Machine Learning (ICML), 2026 Poster
Link
HPS: Hyperspherical Parameter Sharing for Efficient Multi-Agent Reinforcement Learning
Hu Fu, Pengyi Li, Hao Chen, Xuanyu Xiang, Biao Luo, Yihua Tan
International Conference on Machine Learning (ICML), 2026 Poster
Link
Hu Fu, Pengyi Li, Hao Chen, Xuanyu Xiang, Biao Luo, Yihua Tan
International Conference on Machine Learning (ICML), 2026 Poster
Link
Embodied-R1.5: Evolving Physical Intelligence via Embodied Foundation Models
Yifu Yuan, Yaoting Huang, Xianze Yao, Yutong Li, Shuoheng Zhang, Linqi Han, Pengyi Li, Jiangeng Sun, Wenting Jia, Zhao Zhang, Yuhao Liu, Ruihao Liao, Yucheng Hu, Qiyu Wu, Yuxiao Li, Zibin Dong, Fei Ni, Yan Zheng, Shuyang Gu, Yi Ma, Hongyao Tang, Han Hu, Jianye Hao
arXiv preprint, 2026
Paper Project Code
Yifu Yuan, Yaoting Huang, Xianze Yao, Yutong Li, Shuoheng Zhang, Linqi Han, Pengyi Li, Jiangeng Sun, Wenting Jia, Zhao Zhang, Yuhao Liu, Ruihao Liao, Yucheng Hu, Qiyu Wu, Yuxiao Li, Zibin Dong, Fei Ni, Yan Zheng, Shuyang Gu, Yi Ma, Hongyao Tang, Han Hu, Jianye Hao
arXiv preprint, 2026
Paper Project Code
Quality-Diversity for Multi-Agent Reinforcement Learning
Hao Chen*, Pengyi Li*, Bin Zhang, Hu Fu, Zhiwei Xu, Ce Zhang, Xinyue Lu, Guoliang Fan
International Conference on Autonomous Agents and Multiagent Systems (AAMAS), 2026
Paper
Hao Chen*, Pengyi Li*, Bin Zhang, Hu Fu, Zhiwei Xu, Ce Zhang, Xinyue Lu, Guoliang Fan
International Conference on Autonomous Agents and Multiagent Systems (AAMAS), 2026
Paper
ForceFlow: Learning to Feel and Act via Contact-Driven Flow Matching
Shuoheng Zhang, Yifu Yuan, Hongyao Tang, Yan Zheng, Qiaojun Yu, Pengyi Li, Guowei Huang, Helong Huang, Xingyue Quan, Jianye Hao
arXiv preprint, 2026
Paper Project Code
Shuoheng Zhang, Yifu Yuan, Hongyao Tang, Yan Zheng, Qiaojun Yu, Pengyi Li, Guowei Huang, Helong Huang, Xingyue Quan, Jianye Hao
arXiv preprint, 2026
Paper Project Code
Embodied-R1: Reinforced Embodied Reasoning for General Robotic Manipulation
Yifu Yuan, Hongyan Cui, Yihong Huang, Yidi Chen, Fei Ni, Zibin Dong, Pengyi Li, Yan Zheng, Jianye Hao
International Conference on Learning Representations (ICLR), 2026
ArXiv Code
Yifu Yuan, Hongyan Cui, Yihong Huang, Yidi Chen, Fei Ni, Zibin Dong, Pengyi Li, Yan Zheng, Jianye Hao
International Conference on Learning Representations (ICLR), 2026
ArXiv Code
From Seeing to Doing: Bridging Reasoning and Decision for Robotic Manipulation
Yifu Yuan, Hongyan Cui, Yidi Chen, Zibin Dong, Fei Ni, Lihua Kou, Jie Liu, Pengyi Li, Yan Zheng, Jianye Hao
International Conference on Learning Representations (ICLR), 2026
ArXiv Code
Yifu Yuan, Hongyan Cui, Yidi Chen, Zibin Dong, Fei Ni, Lihua Kou, Jie Liu, Pengyi Li, Yan Zheng, Jianye Hao
International Conference on Learning Representations (ICLR), 2026
ArXiv Code
LaRes: Evolutionary Reinforcement Learning with LLM-based Adaptive Reward Search
Pengyi Li, Hongyao Tang, Jinbin Qiao, Yan Zheng, Jianye Hao
Advances in Neural Information Processing Systems (NeurIPS), 2025
Link Code
Pengyi Li, Hongyao Tang, Jinbin Qiao, Yan Zheng, Jianye Hao
Advances in Neural Information Processing Systems (NeurIPS), 2025
Link Code
COLA: Towards Efficient Multi-Objective Reinforcement Learning with Conflict Objective Regularization in Latent Space
Pengyi Li, et al.
Advances in Neural Information Processing Systems (NeurIPS), 2025
Link Code
Pengyi Li, et al.
Advances in Neural Information Processing Systems (NeurIPS), 2025
Link Code
ReMAC: Large Language Model-Driven Reward Design for Multi-Agent Manipulation Collaboration
Pengyi Li, Hongyao Tang, Yifu Yuan, Jianye Hao
NeurIPS Workshop on Scaling Environments for Agents, 2025
Paper Project
Pengyi Li, Hongyao Tang, Yifu Yuan, Jianye Hao
NeurIPS Workshop on Scaling Environments for Agents, 2025
Paper Project
CORE: Collaborative Optimization with Reinforcement Learning and Evolutionary Algorithm for Floorplanning
Pengyi Li, Shixiong Kai, Jianye Hao, Ruizhe Zhong, Hongyao Tang, Zhentao Tang, Mingxuan Yuan, Junchi Yan
Advances in Neural Information Processing Systems (NeurIPS), 2025
Link Code
Pengyi Li, Shixiong Kai, Jianye Hao, Ruizhe Zhong, Hongyao Tang, Zhentao Tang, Mingxuan Yuan, Junchi Yan
Advances in Neural Information Processing Systems (NeurIPS), 2025
Link Code
EmbodiedScene: Towards Automated Generation of Diverse and Realistic Scenes for Embodied AI
Jinbin Qiao*, Pengyi Li*, Peilong Han, Yan Zheng, Jianye HAO
International Conference on Machine Learning (ICML) World Model Workshop Best Paper, 2025
Project Link
Jinbin Qiao*, Pengyi Li*, Peilong Han, Yan Zheng, Jianye HAO
International Conference on Machine Learning (ICML) World Model Workshop Best Paper, 2025
Project Link
R*: Efficient Reward Design via Reward Structure Evolution and Parameter Alignment Optimization with Large Language Models
Pengyi Li, Jianye Hao, Hongyao Tang, Yifu Yuan, Jinbin Qiao, Zibin Dong, Yan Zheng
International Conference on Machine Learning (ICML), 2025
Link
Pengyi Li, Jianye Hao, Hongyao Tang, Yifu Yuan, Jinbin Qiao, Zibin Dong, Yan Zheng
International Conference on Machine Learning (ICML), 2025
Link
Embodied Arena: A Comprehensive, Unified, and Evolving Evaluation Platform for Embodied AI
Fei Ni, Min Zhang, Pengyi Li, Yifu Yuan, et al.
arXiv preprint, 2025
ArXiv
Fei Ni, Min Zhang, Pengyi Li, Yifu Yuan, et al.
arXiv preprint, 2025
ArXiv
Enhancing Graph-based Coordination with Evolutionary Algorithms for Episodic Multi-agent Reinforcement Learning
Kexin Peng, Pengyi Li, Jianye Hao
International Conference on Autonomous Agents and Multiagent Systems (AAMAS), 2025 (Oral)
Link
Kexin Peng, Pengyi Li, Jianye Hao
International Conference on Autonomous Agents and Multiagent Systems (AAMAS), 2025 (Oral)
Link
Signaling-Driven Incentive Communication for Enhanced Multiagent Reinforcement Learning in Dynamic Environments
Kexin Peng, Pengyi Li, Jianye Hao
IEEE Transactions on Cybernetics, 2025
Link
Kexin Peng, Pengyi Li, Jianye Hao
IEEE Transactions on Cybernetics, 2025
Link
Tailoring Knowledge for Empowered Cooperative Actions in Multi-agent Reinforcement Learning
Hu Fu, Yihua Tan, Hao Chen, Pengyi Li
Neural Networks, 2025
Link
Hu Fu, Yihua Tan, Hao Chen, Pengyi Li
Neural Networks, 2025
Link
CleanDiffuser: An Easy-to-use Modularized Library for Diffusion Models in Decision Making
Zibin Dong, Yifu Yuan, Jianye Hao, Fei Ni, Yi Ma, Pengyi Li, Yan Zheng
Advances in Neural Information Processing Systems (NeurIPS), 2024
ArXiv Code
Zibin Dong, Yifu Yuan, Jianye Hao, Fei Ni, Yi Ma, Pengyi Li, Yan Zheng
Advances in Neural Information Processing Systems (NeurIPS), 2024
ArXiv Code
EvoRainbow: Combining Improvements in Evolutionary Reinforcement Learning for Policy Search
Pengyi Li, Yan Zheng, Hongyao Tang, Xian Fu, Jianye Hao
International Conference on Machine Learning (ICML), 2024
Link Code
Pengyi Li, Yan Zheng, Hongyao Tang, Xian Fu, Jianye Hao
International Conference on Machine Learning (ICML), 2024
Link Code
Value-Evolutionary-Based Reinforcement Learning
Pengyi Li, Jianye Hao, Hongyao Tang, Yan Zheng, Fazl Barez
International Conference on Machine Learning (ICML), 2024
Link Code
Pengyi Li, Jianye Hao, Hongyao Tang, Yan Zheng, Fazl Barez
International Conference on Machine Learning (ICML), 2024
Link Code
DiffuserLite: Towards Real-time Diffusion Planning
Zibin Dong, Jianye Hao, Yifu Yuan, Fei Ni, Yitian Wang, Pengyi Li, Yan Zheng
Advances in Neural Information Processing Systems (NeurIPS), 2024
ArXiv Code
Zibin Dong, Jianye Hao, Yifu Yuan, Fei Ni, Yitian Wang, Pengyi Li, Yan Zheng
Advances in Neural Information Processing Systems (NeurIPS), 2024
ArXiv Code
Bridging Evolutionary Algorithms and Reinforcement Learning: A Comprehensive Survey
Pengyi Li, Jianye Hao, Hongyao Tang, Xian Fu, Yan Zheng, Ke Tang
IEEE Transactions on Evolutionary Computation (IEEE TEVC), 2025
Github Repository ArXiv
Pengyi Li, Jianye Hao, Hongyao Tang, Xian Fu, Yan Zheng, Ke Tang
IEEE Transactions on Evolutionary Computation (IEEE TEVC), 2025
Github Repository ArXiv
Sample-Efficient Quality-Diversity by Cooperative Coevolution
Ke Xue, Ren-Jian Wang, Pengyi Li, Dong Li, Jianye Hao, Chao Qian
International Conference on Learning Representations (ICLR), 2024
OpenReview
Ke Xue, Ren-Jian Wang, Pengyi Li, Dong Li, Jianye Hao, Chao Qian
International Conference on Learning Representations (ICLR), 2024
OpenReview
RACE: Improve Multi-Agent Reinforcement Learning with Representation Asymmetry and Collaborative Evolution
Pengyi Li, Jianye Hao, Hongyao Tang, Yan Zheng, Xian Fu
International Conference on Machine Learning (ICML), 2023
Link Code
Pengyi Li, Jianye Hao, Hongyao Tang, Yan Zheng, Xian Fu
International Conference on Machine Learning (ICML), 2023
Link Code
ERL-Re^2: Efficient Evolutionary Reinforcement Learning with Shared State Representation and Individual Policy Representation
Jianye Hao, Pengyi Li (Student First Author), Hongyao Tang, Yan Zheng, Xian Fu, Zhaopeng Meng
International Conference on Learning Representations (ICLR), 2023
ArXiv Code
Jianye Hao, Pengyi Li (Student First Author), Hongyao Tang, Yan Zheng, Xian Fu, Zhaopeng Meng
International Conference on Learning Representations (ICLR), 2023
ArXiv Code
PMIC: Improving Multi-Agent Reinforcement Learning with Progressive Mutual Information Collaboration
Pengyi Li, Hongyao Tang, Tianpei Yang, Xiaotian Hao, Tong Sang, Yan Zheng, Jianye Hao, Matthew E. Taylor, Wenyuan Tao, Zhen Wang
International Conference on Machine Learning (ICML), 2022
Link Code
Pengyi Li, Hongyao Tang, Tianpei Yang, Xiaotian Hao, Tong Sang, Yan Zheng, Jianye Hao, Matthew E. Taylor, Wenyuan Tao, Zhen Wang
International Conference on Machine Learning (ICML), 2022
Link Code
HyAR: Addressing Discrete-Continuous Action Reinforcement Learning via Hybrid Action Representation
Boyan Li, Hongyao Tang, Yan Zheng, Jianye Hao, Pengyi Li, Zhen Wang, Zhaopeng Meng, Li Wang
International Conference on Learning Representations (ICLR), 2022
ArXiv
Boyan Li, Hongyao Tang, Yan Zheng, Jianye Hao, Pengyi Li, Zhen Wang, Zhaopeng Meng, Li Wang
International Conference on Learning Representations (ICLR), 2022
ArXiv
Open-source Repositories
Contact
You are very welcome to contact me regarding my research. I typically respond within a few days.
lipengyi [at] tju.edu.cn
Site Statistics
Visitor globe:
📊 Visitor badge:
New location data tracked by MapMyVisitors