Many roboticists dream of presenting a robot with a task in the evening and returning the next morning to find the robot capable of solving the task. What is preventing us from achieving this? Sim-to-real reinforcement learning (RL) has achieved impressive performance on challenging robotics tasks, but requires substantial human effort to set up the task in a way that is amenable to RL. It's our position that algorithmic improvements in policy optimization and other ideas should be guided towards resolving the primary bottleneck of shaping the training environment, i.e., designing observations, actions, rewards and simulation dynamics. Most practitioners don't tune the RL algorithm, but other environment parameters to obtain a desirable controller. We posit that scaling RL to diverse robotic tasks will only be achieved if the community focuses on automating environment shaping procedures.
EAGER: Asking and Answering Questions for Automatic Reward Shaping in Language-guided RL
Inria(法国国家信息与自动化研究所);Université de Bordeaux(波尔多大学);ISIR Sorbonne Université(索邦大学智能系统与机器人研究所);Univ Angers(昂热大学);Microsoft Research Montreal(微软研究院蒙特利尔分院)
Automatic Reward Shaping from Confounded Offline Data
Causal AI Lab, Columbia University, New York, USA(因果AI实验室,哥伦比亚大学,纽约,美国);Department of Electrical Engineering and Computer Science, Syracuse University, New York, USA(电气工程与计算机科学系, Syracuse大学,纽约,美国)
MVR: Multi-view Video Reward Shaping for Reinforcement Learning
School of Intelligence Science and Technology, Peking University(北京大学智能科学与技术学院);State Key Laboratory of General Artificial Intelligence, BIGAI(通用人工智能国家重点实验室)
FOM-Nav: Frontier-Object Maps for Object Goal Navigation
Inria(法国国家信息与自动化技术研究院);Department of Computer Science, École normale supérieure (ENS-PSL, CNRS, Inria)(高等师范学院计算机科学系);Courant Institute of Mathematical Sciences and Center for Data Science, New York University(纽约大学Courant数学科学研究所和数据科学中心)
Dependency-Aware Reward Shaping for Agentic Reinforcement Learning
University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);National University of Singapore(新加坡国立大学);Zhejiang University(浙江大学);University of Rochester(罗切斯特大学)
A Survey of Automatic Prompt Engineering: An Optimization Perspective
School of Computer Science and Technology, Tongji University(同济大学计算机科学与技术学院);East China Normal University(华东师范大学);Shanghai Research Institute for Intelligent Autonomous Systems, Tongji University(同济大学上海智能自主系统研究院)
Open-World Evaluations for Measuring Frontier AI Capabilities
Princeton University(普林斯顿大学);Cornflower Labs(Cornflower实验室);Meridian Labs(Meridian实验室);Stanford University(斯坦福大学);UK AI Security Institute(英国人工智能安全研究所);Johns Hopkins University(约翰霍普金斯大学);Adaption Labs(Adaption实验室);Australian National University(澳大利亚国立大学);Golden Gate Institute for AI(金门人工智能研究所);UW Madison(威斯康星大学麦迪逊分校);Microsoft Research(微软研究院);AI Digest(AI摘要);Georgetown University (CSET)(乔治城大学(CSET))