多感官智能的愿景：感知、科学与协同
A Vision for Multisensory Intelligence: Sensing, Science, and Synergy
AI总结：
本文提出未来十年多感官人工智能的研究愿景，强调通过感知、科学和协同三大主题推动多感官技术发展，提升人与AI的交互体验。
摘要：
我们的世界体验是多感官的，涵盖了语言、视觉、听觉、触觉、味觉和嗅觉的综合。然而，人工智能主要在数字模态如文本、视觉和音频上取得了进展。本文提出了未来十年多感官人工智能的研究愿景。这一新的技术集合可以通过将AI连接到人类感官以及身体上的生理和触觉信号、家庭、城市和环境中的物理和社会信号，改变人类和AI的体验和互动方式。我们概述了这一领域必须通过感知、科学和协同三个相互关联的主题来推进。首先，感知研究应扩展AI以更丰富的方式捕捉世界，超越数字媒介。其次，发展一种原则性的科学，以量化多模态异质性和相互作用，开发统一的建模架构和表示，并理解跨模态转移。最后，我们提出了学习模态之间以及人与AI之间协同的新技术挑战，涵盖多感官整合、对齐、推理、生成、泛化和经验。本文还附有一系列项目、资源和演示，展示了麻省理工学院媒体实验室多感官智能小组的最新进展，见https://mit-mi.github.io/。
英文摘要：
Our experience of the world is multisensory, spanning a synthesis of language, sight, sound, touch, taste, and smell. Yet, artificial intelligence has primarily advanced in digital modalities like text, vision, and audio. This paper outlines a research vision for multisensory artificial intelligence over the next decade. This new set of technologies can change how humans and AI experience and interact with one another, by connecting AI to the human senses and a rich spectrum of signals from physiological and tactile cues on the body, to physical and social signals in homes, cities, and the environment. We outline how this field must advance through three interrelated themes of sensing, science, and synergy. Firstly, research in sensing should extend how AI captures the world in richer ways beyond the digital medium. Secondly, developing a principled science for quantifying multimodal heterogeneity and interactions, developing unified modeling architectures and representations, and understanding cross-modal transfer. Finally, we present new technical challenges to learn synergy between modalities and between humans and AI, covering multisensory integration, alignment, reasoning, generation, generalization, and experience. Accompanying this vision paper are a series of projects, resources, and demos of latest advances from the Multisensory Intelligence group at the MIT Media Lab, see https://mit-mi.github.io/.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
