There has been a significant research interest in employing large language models to empower intelligent robots with complex reasoning. Existing work focuses on harnessing their abilities to reason about the histories of their actions and observations. In this paper, we explore a new dimension in which large language models may benefit robotics planning. In particular, we propose Statler, a framework in which large language models are prompted to maintain an estimate of the world state, which are often unobservable, and track its transition as new actions are taken. Our framework then conditions each action on the estimate of the current world state. Despite being conceptually simple, our Statler framework significantly outperforms strong competing methods (e.g., Code-as-Policies) on several robot planning tasks. Additionally, it has the potential advantage of scaling up to more challenging long-horizon planning tasks.
Embodied.cpp: A Portable Inference Runtime of Embodied AI Models on Heterogeneous Robots
Southeast University(东南大学);Nanjing University(南京大学);Microsoft Research(微软研究院);Institute for AI Industry Research (AIR), Tsinghua University(清华大学人工智能产业研究院(AIR))
Do Vision-Language-Action Models Mean What They Say? On the Role of Faithfulness in Embodied Reasoning
Stanford University(斯坦福大学);Politecnico di Milano(米兰理工大学);KTH Royal Institute of Technology(皇家理工学院);Italian Institute of Artificial Intelligence (AI4I)(意大利人工智能研究所);NVIDIA Research(英伟达研究院)
A Deployable Embodied Vision-Language Navigation System with Hierarchical Cognition and Context-Aware Exploration
Center for Advanced Robotics Technology Innovation (CARTIN), School of Electrical and Electronic Engineering, Nanyang Technological University(先进机器人技术与创新中心(CARTIN)、电子工程学院、南洋理工大学);Spatial AI & Robotics Lab, Department of Computer Science and Engineering(空间人工智能与机器人实验室、计算机科学与工程系)
V-CAGE: Context-Aware Generation and Verification for Scalable Long-Horizon Embodied Tasks
Department of Computer Science and Technology, University of Cambridge, Cambridge, England.(计算机科学与技术系,剑桥大学,剑桥,英格兰);Shanghai Jiao Tong University, Shanghai, China(上海交通大学,上海,中国);Wuhan University, Wuhan, Hubei, China(武汉大学,武汉,湖北,中国)
RoboSafe: Safeguarding Embodied Agents via Executable Safety Logic
Beihang University(北航大学);AI Security Lab(360AI安全实验室);The University of Sydney(悉尼大学);Nanyang Technological University(南洋理工大学);Peking University(北京大学);Beijing Academy of Artificial Intelligence(北京人工智能研究院)
DualVLA: Building a Generalizable Embodied Agent via Partial Decoupling of Reasoning and Action
MoE Key Laboratory of Brain-inspired Intelligent Perception and Cognition, University of Science and Technology of China(脑启发智能感知与认知国家重点实验室,中国科学技术大学);State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University(多媒体信息处理国家重点实验室,北京大学计算机学院);CUHK(香港大学)
Large Language Models as symbolic DNA of cultural dynamics
Human Energy(人类能源);San Francisco VA Medical Center(旧金山退伍军人医疗中心);University of California, San Francisco(加州大学旧金山分校);University of California, Berkeley(加州大学伯克利分校)
Concept-Based Dictionary Learning for Inference-Time Safety in Vision Language Action Models
Beijing Jiaotong University(北京交通大学);Provable Responsible AI and Data Analytics (PRADA) Lab(可证责任AI与数据分析实验室);King Abdullah University of Science and Technology(国王 Abdullah 科学技术大学);University of Auckland(奥克兰大学);RIKEN Center for Advanced Intelligence Project (AIP)(理化学研究所高级智能项目中心);Mohamed bin Zayed University of Artificial Intelligence (MBZUAI)(穆罕默德·本·扎耶德人工智能大学)
DaDu-E: Rethinking the Role of Large Language Model in Robotic Computing Pipeline
Institute of Computing Technology, Chinese Academy of Sciences (ICT, CAS)(中国科学院计算技术研究所);University of Chinese Academy of Sciences (UCAS)(中国科学院大学);Beijing Institute of Technology (BIT)(北京理工大学);Institute of Automation, Chinese Academy of Science (CASIA)(中国科学院自动化研究所);Shenzhen Institute of Artificial Intelligence and Robotics for Society (AIRS)(深圳市人工智能与机器人研究院)
On-Device Robotic Planning: Eliminating Inference Redundancy for Efficient Decision-Making
School of Integrated Technology, College of Computing, Yonsei University(延世大学整合技术学院,计算学院);Department of Hyperscale AI SoC Research Section, ETRI(ETRI超大规模AI SoC研究部);EECS - Electrical and Computer Engineering, University of Michigan(密歇根大学电气与计算机工程学院)
QVLA: Not All Channels Are Equal in Vision-Language-Action Model's Quantization
AutoLab, School of Artificial Intelligence, Shanghai Jiao Tong University(自动化实验室,人工智能学院,上海交通大学);Anyverse Dynamics;State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences(多模态人工智能系统国家重点实验室,自动化研究所,中国科学院);School of Artificial Intelligence, University of Chinese Academy of Sciences(人工智能学院,中国科学院大学);Terminal Technology Department, Alipay, Ant Group(终端技术部,蚂蚁集团)
UnderwaterVLA: Dual-brain Vision-Language-Action architecture for Autonomous Underwater Navigation
School of Engineering, Westlake University(西lake大学工程学院);College of Information Science & Electronic Engineering, Zhejiang University(浙江大学信息科学与电子工程学院);College of Environmental and Resource Sciences, Zhejiang University(浙江大学环境与资源科学学院);Australian National University(澳大利亚国立大学)