Natural language can offer a concise and human-interpretable means of specifying reinforcement learning (RL) tasks. The ability to extract rewards from a language instruction can enable the development of robotic systems that can learn from human guidance; however, it remains a challenging problem, especially in visual environments. Existing approaches that employ large, pretrained language models either rely on non-visual environment representations, require prohibitively large amounts of feedback, or generate noisy, ill-shaped reward functions. In this paper, we propose a novel method, GoalLadder, that leverages vision-language models (VLMs) to train RL agents from a single language instruction in visual environments. GoalLadder works by incrementally discovering states that bring the agent closer to completing a task specified in natural language. To do so, it queries a VLM to identify states that represent an improvement in agent's task progress and to rank them using pairwise comparisons. Unlike prior work, GoalLadder does not trust VLM's feedback completely; instead, it uses it to rank potential goal states using an ELO-based rating system, thus reducing the detrimental effects of noisy VLM feedback. Over the course of training, the agent is tasked with minimising the distance to the top-ranked goal in a learned embedding space, which is trained on unlabelled visual data. This key feature allows us to bypass the need for abundant and accurate feedback typically required to train a well-shaped reward function. We demonstrate that GoalLadder outperforms existing related methods on classic control and robotic manipulation environments with the average final success rate of $\sim$95% compared to only $\sim$45% of the best competitor.
Automatic Attack Discovery for Few-Shot Class-Incremental Learning via Large Language Models
School of Computer and Communication Engineering(计算机与通信工程学院);School of Information and Communication(信息与通信学院);University of Electronic Science and Technology of China(电子科学与技术大学);Department of Strategic and Advanced Interdisciplinary Research(战略与先进跨学科研究部);Pengcheng Laboratory(鹏城实验室)
Incremental Structure Discovery of Classification via Sequential Monte Carlo
Key Laboratory of High Confidence Software Technologies (Peking University)(高可信软件技术重点实验室(北京大学));School of Computer Science, Peking University(北京大学计算机学院)
GPT-Micro: A large language paradigm for accelerated, inexpensive, and thermodynamics-consistent discovery of constitutive models in manufacturing
Department of Mechanical and Aerospace Engineering, Rutgers University(罗格斯大学机械与航空航天工程系);Department of Mechanical, Aerospace & Manufacturing Engineering, University of Connecticut(康涅狄格大学机械、航空航天与制造工程系);Edison Welding Institute(埃迪森焊接研究所);Manufacturing Science Division, Oak Ridge National Laboratory(橡树岭国家实验室制造科学分会);Department of Aerospace and Mechanical Engineering, University of Texas at El Paso(德克萨斯州埃尔帕索大学航空航天与机械工程系)