On the Sensory Commutativity of Action Sequences for Embodied Agents
ENSTA Paris（巴黎高等先进技术学院）；INRIA（法国国家信息与自动化研究所）；Softbank Robotics Europe（软银机器人欧洲公司）；City University of London（伦敦城市大学）
Hugo Caselles-Dupré, Michael Garcia-Ortiz, David Filliat
备注：
Accepted to RSS'20 Workshop on Self-Supervised Robot Learning & to the Workshop on Learning in Artificial Open Worlds at ICML20 & Extended abstract at AAMAS21
Perception of artificial agents is one the grand challenges of AI research. Deep Learning and data-driven approaches are successful on constrained problems where perception can be learned using supervision, but do not scale to open-worlds. In such case, for autonomous embodied agents with first-person sensors, perception can be learned end-to-end to solve particular tasks. However, literature shows that perception is not a purely passive compression mechanism, and that actions play an important role in the formulation of abstract representations. We propose to study perception for these embodied agents, under the mathematical formalism of group theory in order to make the link between perception and action. In particular, we consider the commutative properties of continuous action sequences with respect to sensory information perceived by such an embodied agent. We introduce the Sensory Commutativity Probability (SCP) criterion which measures how much an agent's degree of freedom affects the environment in embodied scenarios. We show how to compute this criterion in different environments, including realistic robotic setups. We empirically illustrate how SCP and the commutative properties of action sequences can be used to learn about objects in the environment and improve sample-efficiency in Reinforcement Learning.
Department of Computer Science, Stanford University（斯坦福大学计算机科学系）；Department of Applied Physics, Stanford University（斯坦福大学应用物理系）；Wu-Tsai Neurosciences Institute, Stanford University（斯坦福大学吴蔡神经科学研究所）；Stanford Institute for Human-Centered Artificial Intelligence, Stanford University（斯坦福大学以人为本人工智能研究院）
Agent AI: Surveying the Horizons of Multimodal Interaction
Microsoft Gaming（微软游戏）；University of California, Los Angeles（加州大学洛杉矶分校）；University of Washington（华盛顿大学）；Microsoft Research, Redmond（微软研究院（雷德蒙德））；Stanford University（斯坦福大学）
University of Hertfordshire, UK（赫特福德郡大学）；Earth-Life Science Institute (ELSI), Institute of Science Tokyo, Japan（地球生命科学研究所（ELSI））；Cross Labs, Cross Compass Ltd., Japan（Cross Labs）；Araya, Inc., Japan（Araya公司）
Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Georgia Institute of Technology（佐治亚理工学院）；University of Texas at Dallas（德克萨斯大学达拉斯分校）
What if Eye...? Computationally Recreating Vision Evolution
Camera Culture；MIT Media Lab（麻省理工学院媒体实验室）；Computational Imaging Lab（计算成像实验室）；Rice University（莱斯大学）；Center for Brains Minds and Machines（大脑、心智与机器中心）；MIT（麻省理工学院）；Lund Vision Group（隆德视觉组）；Lund University（隆德大学）；InfoLab（信息实验室）；MIT CSAIL（麻省理工学院计算机科学与人工智能实验室）；Charles Stark Draper Laboratory（查尔斯·斯塔克·德雷珀实验室）
Towards the Vision-Sound-Language-Action Paradigm: The HEAR Framework for Sound-Centric Manipulation
School of Automation and Intelligent Sensing, Shanghai Jiao Tong University and Shanghai Key Laboratory of Navigation and Location Based Services（自动化与智能感知学院，上海交通大学，导航与基于位置的服务重点实验室）；Department of Engineering, Cambridge University（工程系，剑桥大学）
Building Cooperative Embodied Agents Modularly with Large Language Models
University of Massachusetts Amherst（马萨诸塞大学阿默斯特分校）；Tsinghua University（清华大学）；Shanghai Jiao Tong University（上海交通大学）；MIT（麻省理工学院）；MIT-IBM Watson AI Lab（麻省理工学院-IBM沃森人工智能实验室）
Neural Foundations of Mental Simulation: Future Prediction of Latent Representations on Dynamic Scenes
McGovern Institute for Brain Research, MIT（麻省理工学院麦戈文脑研究所）；Reality Labs, Meta（Meta现实实验室）；Department of Brain and Cognitive Sciences, MIT（麻省理工学院脑与认知科学系）；Department of Electrical Engineering and Computer Science, MIT（麻省理工学院电气工程与计算机科学系）
Position Paper: Agent AI Towards a Holistic Intelligence
Stanford University（斯坦福大学）；University of California, Los Angeles（加利福尼亚大学洛杉矶分校）；Microsoft Gaming US（美国微软游戏部门）；MSR Accelerator（微软研究院加速器）；MSR AI Frontiers（微软研究院人工智能前沿部门）；Microsoft Research Core（微软研究院核心部门）；Microsoft Applied Robotics Research（微软应用机器人研究院）
本文倡导从还原论转向整体智能，提出整合大型基础模型与智能体行动的 Agent AI 及 Agent Foundation Model，并探讨其跨领域能力与认知意识意义。
Reactive, Proactive, and Inductive Agents: An evolutionary path for biological and artificial spiking networks
Sony Computer Science Laboratories, Inc.（索尼计算机科学研究所）；Earth-Life Science Institute, Tokyo Institute of Technology（东京工业大学地球生命研究所）；The University of Tokyo（东京大学）
Independent Researcher（独立研究者）；Department of Computing Science, University of Alberta, Canada（阿尔伯塔大学计算机科学系）；Alberta Machine Intelligence Institute (Amii)（阿尔伯塔机器智能研究所）
