As robotic systems become increasingly integrated into complex real-world environments, there is a growing need for approaches that enable robots to understand and act upon natural language instructions without relying on extensive pre-programmed knowledge of their surroundings. This paper presents PLATO, an innovative system that addresses this challenge by leveraging specialized large language model agents to process natural language inputs, understand the environment, predict tool affordances, and generate executable actions for robotic systems. Unlike traditional systems that depend on hard-coded environmental information, PLATO employs a modular architecture of specialized agents to operate without any initial knowledge of the environment. These agents identify objects and their locations within the scene, generate a comprehensive high-level plan, translate this plan into a series of low-level actions, and verify the completion of each step. The system is particularly tested on challenging tool-use tasks, which involve handling diverse objects and require long-horizon planning. PLATO's design allows it to adapt to dynamic and unstructured settings, significantly enhancing its flexibility and robustness. By evaluating the system across various complex scenarios, we demonstrate its capability to tackle a diverse range of tasks and offer a novel solution to integrate LLMs with robotic platforms, advancing the state-of-the-art in autonomous robotic task execution. For videos and prompt details, please see our project website: https://sites.google.com/andrew.cmu.edu/plato
Affordance2Action: Task-Conditioned Scene-level Affordance Grounding for Real-Time Manipulation
Department of Computer Science, Rutgers University-New Brunswick（罗格斯大学新布朗斯维尔回声分校计算机科学系）；The Hong Kong University of Science and Technology (GZ)（香港科学与技术大学（GZ））；Shanghai AI Laboratory（上海人工智能实验室）
DORA: Object Affordance-Guided Reinforcement Learning for Dexterous Robotic Manipulation
TAMS (Technical Aspects of Multimodal Systems), Department of Informatics, Universität Hamburg（技术多模态系统（TAMS），信息学院，汉堡大学）；Agile Robots AG（敏捷机器人公司）；Technical University of Munich（慕尼黑技术大学）
3D Affordance Keypoint Detection for Robotic Manipulation
Advanced Robotics Centre, National University of Singapore（新加坡国立大学先进机器人中心）；Department of Mechanical Engineering, National University of Singapore（新加坡国立大学机械工程系）
RoboAfford++: A Generative AI-Enhanced Dataset for Multimodal Affordance Learning in Robotic Manipulation and Navigation
Xiaomi EV（小米电动车）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Tsinghua Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）；Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学 Gallagher人工智能学院）；School of Computer Science and Information Engineering, Hefei University of Technology（合肥工业大学计算机科学与信息工程学院）
A4A: Cross-Embodiment Transfer of Action-Oriented 4D Affordances from Human Demonstrations
Shanghai Jiao Tong University（上海交通大学）；Rutgers University–New Brunswick（罗格斯大学新布朗斯维克分校）；Nanyang Technological University（南洋理工大学）；The Hong Kong University of Science and Technology (GZ)（香港科技大学（广州））；Shanghai AI Laboratory（上海人工智能实验室）
Articulated Object Manipulation with Coarse-to-fine Affordance for Mitigating the Effect of Point Cloud Noise
Hyperplane Lab, School of CS, Peking University（北京大学计算机学院超平面实验室）；National Key Laboratory for Multimedia Information Processing（多媒体信息处理国家重点实验室）；Huawei（华为）
Task-Aware Bimanual Affordance Prediction via VLM-Guided Semantic-Geometric Reasoning
Department of Computer Science, Technical University of Darmstadt（达姆施塔特工业大学计算机科学系）；German Research Center for AI (DFKI)（德国人工智能研究中心）；Centre for Cognitive Science, Technical University of Darmstadt（达姆施塔特工业大学认知科学中心）；Hessian Center for Artificial Intelligence (Hessian.AI), Darmstadt（黑森州人工智能中心）；Robotics Institute Germany (RIG)（德国机器人研究所）
UniAff: A Unified Representation of Affordances for Tool Usage and Articulation with Vision-Language Models
Shanghai Jiao Tong University（上海交通大学）；Hong Kong University of Science and Technology（香港科技大学）；National University of Singapore（新加坡国立大学）；Rutgers University（罗格斯大学）；Hefei University of Technology（合肥工业大学）；Shanghai AI Lab（上海人工智能实验室）
PALM: Progress-Aware Policy Learning via Affordance Reasoning for Long-Horizon Robotic Manipulation
University of Pennsylvania（宾夕法尼亚大学）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Nanyang Technological University（南洋理工大学）；University of Oxford（牛津大学）；Massachusetts Institute of Technology（麻省理工学院）
AffordGrasp: In-Context Affordance Reasoning for Open-Vocabulary Task-Oriented Grasping in Clutter
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Harbin Institute of Technology (Shenzhen)（哈尔滨工业大学（深圳））；Beijing Academy of Artificial Intelligence（北京人工智能研究院）；Peking University（北京大学）
AffordanceWAM: Affordance-Aware Joint World-Action Modeling for Robot Manipulation
The University of Hong Kong（香港大学）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Peking University（北京大学）；The Chinese University of Hong Kong（香港中文大学）；National University of Singapore（新加坡国立大学）；Knowin AI（考拉悠然科技）
