Cog-GA: A Large Language Models-based Generative Agent for Vision-Language Navigation in Continuous Environments
Institute of Automation, Chinese Academy of Science (CASIA)（中国科学院自动化研究所）；University of Chinese Academy of Sciences (UCAS)（中国科学院大学）；The University of Hong Kong（香港大学）
Vision Language Navigation in Continuous Environments (VLN-CE) represents a frontier in embodied AI, demanding agents to navigate freely in unbounded 3D spaces solely guided by natural language instructions. This task introduces distinct challenges in multimodal comprehension, spatial reasoning, and decision-making. To address these challenges, we introduce Cog-GA, a generative agent founded on large language models (LLMs) tailored for VLN-CE tasks. Cog-GA employs a dual-pronged strategy to emulate human-like cognitive processes. Firstly, it constructs a cognitive map, integrating temporal, spatial, and semantic elements, thereby facilitating the development of spatial memory within LLMs. Secondly, Cog-GA employs a predictive mechanism for waypoints, strategically optimizing the exploration trajectory to maximize navigational efficiency. Each waypoint is accompanied by a dual-channel scene description, categorizing environmental cues into 'what' and 'where' streams as the brain. This segregation enhances the agent's attentional focus, enabling it to discern pertinent spatial information for navigation. A reflective mechanism complements these strategies by capturing feedback from prior navigation experiences, facilitating continual learning and adaptive replanning. Extensive evaluations conducted on VLN-CE benchmarks validate Cog-GA's state-of-the-art performance and ability to simulate human-like navigation behaviors. This research significantly contributes to the development of strategic and interpretable VLN-CE agents.
Navigating Conceptual Space; A new take on Artificial General Intelligence
UiT - The Arctic University of Norway（特罗姆瑟大学-挪威北极圈大学）
Edward C.托尔曼发现，强化学习在解释智力方面并不令人满意，并提出了学习和行为之间的明确区别。托尔曼关于潜在学习和认知地图的想法最终导致了现在所知的概念空间，概念和想法可以形成点或形状的几何表示。想法之间的主动导航-推理-可以直接表达为概念空间中的目的导航。从现代神经科学中吸收概念空间理论，我们提出自主导航作为一种有效的方法，模拟认知。然而，在高维欧氏空间中实现自主导航在技术上并不简单。在这项工作中，我们探讨了neoRL导航是否适合这项任务;采用Kaelbling对高效机器人导航的关注，我们测试了neoRL方法是否在导航方式中通用，在经验考虑中组合，以及在多个欧几里德维度学习时是否有效。我们发现neoRL学习比AI中的RL更符合生物学习，并提出概念空间的neoRL导航作为模拟认知的一条可行的新途径。
WorldMAP: Bootstrapping Vision-Language Navigation Trajectory Prediction with Generative World Models
Harbin Institute of Technology（哈尔滨工业大学）；Shandong University（山东大学）；Tsinghua University（清华大学）；University of Science and Technology of China（中国科学技术大学）；Zhongguancun Academy（中关村学院）
Generative midtended cognition and Artificial Intelligence. Thinging with thinging things
IAS-Research Centre for Life, Mind and Society, Dept. Philosophy, UPV/EHU, University of the Basque Country, Donostia, Gipuzkoa (Spain)（生命、心智与社会研究中心，哲学系，巴斯克大学，Donostia, Gipuzkoa (西班牙)）
CitySeeker: How Do VLMS Explore Embodied Urban Navigation With Implicit Human Needs?
Department of Computing, The Hong Kong Polytechnic University（香港理工大学计算机系）；Tongji University（同济大学）；Nanjing Institute of Geography and Limnology, Chinese Academy of Sciences（中国科学院南京地理与湖泊研究所）；Research Centre for Data Science & Artificial Intelligence（数据科学与人工智能研究中心）
A Deployable Embodied Vision-Language Navigation System with Hierarchical Cognition and Context-Aware Exploration
Center for Advanced Robotics Technology Innovation (CARTIN), School of Electrical and Electronic Engineering, Nanyang Technological University（先进机器人技术与创新中心（CARTIN）、电子工程学院、南洋理工大学）；Spatial AI & Robotics Lab, Department of Computer Science and Engineering（空间人工智能与机器人实验室、计算机科学与工程系）
Mind over Space: Can Multimodal Large Language Models Mentally Navigate?
Institute of Artificial Intelligence, Beihang University（北京航空航天大学人工智能研究院）；Dept. of Comp. Sci. and Tech., Institute for AI, Tsinghua-Bosch Joint ML Center, THBI Lab, BNRist Center, Tsinghua University（清华大学-博世联合机器学习中心、THBI实验室、BNRist中心、清华大学计算机科学与技术系）；School of Automation Science and Electrical Engineering , Beihang University（北京航空航天大学自动化科学与电气工程学院）；college of AI, Tsinghua University（清华大学人工智能学院）；Department of Computer Science and Technology , Tsinghua University（清华大学计算机科学与技术系）
Faculty of Computer Science Dalhousie University（达尔豪斯大学计算机科学学院）；Department of Computer Science University of Waterloo（滑铁卢大学计算机科学系）；Computer Science and Artificial Intelligence Laboratory Massachussetts Institute of Technology（麻省理工学院计算机科学与人工智能实验室）；Department of Computer Science Cornell University（康奈尔大学计算机科学系）
Endowing Embodied Agents with Spatial Reasoning Capabilities for Vision-and-Language Navigation
Southwestern University of Finance（西南财经大学）；Department of Informatics,Universitat Hamburg, Hamburg, Germany（汉堡大学信息学院）；University of Electronic Science（电子科技大学）
Spatial Concept-Based Navigation with Human Speech Instructions via Probabilistic Inference on Bayesian Generative Model
College of Information Science and Engineering, Ritsumeikan University（立命馆大学信息科学与工程学院）；National Institute of Informatics（国立情报学研究所）；The Graduate University for Advanced Studies, SOKENDAI（综合研究大学院大学）
CogEvolution: A Human-like Generative Educational Agent to Simulate Student's Cognitive Evolution
Faculty of Artificial Intelligence in Education, Central China Normal University, Wuhan, Hubei, China（教育人工智能学院，中国中央师范大学，武汉，湖北，中国）；National Engineering Research Center for Educational Big Data, Central China Normal University, Wuhan, Hubei, China（教育大数据国家工程研究中心，中国中央师范大学，武汉，湖北，中国）；Central China Normal University Wollongong Joint Institute, Central China Normal University, Wuhan, Hubei, China（中国中央师范大学沃林顿联合学院，中国中央师范大学，武汉，湖北，中国）
EDU-MATRIX: A Society-Centric Generative Cognitive Digital Twin Architecture for Secondary Education
The High School Affiliated to Beijing Normal University（北京师范大学附属高中）；Department of Electronic and Communication Engineering, North China Electric Power University（华北电力大学电子与通信工程系）
From reactive to cognitive: brain-inspired spatial intelligence for embodied agents
Department of Computer Science and Technology, Institute for AI, BNRist Center, Tsinghua-Bosch Joint ML Center, THBI Lab（计算机科学与技术系、人工智能研究院、BNRist中心、清华-博世联合机器学习中心、THBI实验室）；Institute of Artificial Intelligence（人工智能研究院）；Department of Psychological and Cognitive Sciences（心理学与认知科学系）
