arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2310.09605cs.AIcs.LG
收藏

穿透式人工智能:让大语言模型理解物理世界

Penetrative AI: Making LLMs Comprehend the Physical World

Hong Kong University of Science and Technology(香港科技大学);University of California Los Angeles(加州大学洛杉矶分校);Nanyang Technological University(南洋理工大学)

Huatao Xu, Liying Han, Qirui Yang, Mo Li, Mani Srivastava

AI总结:

本文提出“穿透式人工智能”概念,探索通过物联网传感器与执行器扩展大语言模型以理解和推理物理世界,并初步验证了以ChatGPT为代表的LLMs在解释传感数据和物理任务推理上的独特能力。

摘要:

大语言模型(LLMs)的最新进展展示了它们在一系列任务中的卓越能力。然而,关于LLMs的本质及其在执行涉及真实物理世界信息的任务时整合人类常识性知识的潜力,仍然存在疑问。本文通过探索如何借助物联网传感器和执行器扩展LLMs,使其与物理世界进行交互和推理,从而深入探讨这些问题,我们将这一概念称为“穿透式人工智能(Penetrative AI)”。本文从LLMs通过处理感知信号穿透物理世界的两个能力层面探索了这种扩展。我们的初步发现表明,以ChatGPT作为我们探索中的代表性示例,LLMs在运用内嵌的世界知识解释物联网传感器数据并据此对物理领域任务进行推理方面,具有相当显著且独特的熟练度。这不仅为LLMs开辟了超越传统文本任务的新应用,也为在信息物理系统中融入人类知识提供了新途径。

英文摘要:

Recent developments in Large Language Models (LLMs) have demonstrated their remarkable capabilities across a range of tasks. Questions, however, persist about the nature of LLMs and their potential to integrate common-sense human knowledge when performing tasks involving information about the real physical world. This paper delves into these questions by exploring how LLMs can be extended to interact with and reason about the physical world through IoT sensors and actuators, a concept that we term "Penetrative AI". The paper explores such an extension at two levels of LLMs' ability to penetrate into the physical world via the processing of sensory signals. Our preliminary findings indicate that LLMs, with ChatGPT being the representative example in our exploration, have considerable and unique proficiency in employing the embedded world knowledge for interpreting IoT sensor data and reasoning over them about tasks in the physical realm. Not only this opens up new applications for LLMs beyond traditional text-based tasks, but also enables new ways of incorporating human knowledge in cyber-physical systems.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑