Recent developments in Large Language Models (LLMs) have demonstrated their remarkable capabilities across a range of tasks. Questions, however, persist about the nature of LLMs and their potential to integrate common-sense human knowledge when performing tasks involving information about the real physical world. This paper delves into these questions by exploring how LLMs can be extended to interact with and reason about the physical world through IoT sensors and actuators, a concept that we term "Penetrative AI". The paper explores such an extension at two levels of LLMs' ability to penetrate into the physical world via the processing of sensory signals. Our preliminary findings indicate that LLMs, with ChatGPT being the representative example in our exploration, have considerable and unique proficiency in employing the embedded world knowledge for interpreting IoT sensor data and reasoning over them about tasks in the physical realm. Not only this opens up new applications for LLMs beyond traditional text-based tasks, but also enables new ways of incorporating human knowledge in cyber-physical systems.
Segment First or Comprehend First? Explore the Limit of Unsupervised Word Segmentation with Large Language Models
School of Artificial Intelligence, Wuhan University(武汉大学人工智能学院);School of Computer Science, Wuhan University(武汉大学计算机学院);School of Computer Science, Shanghai Jiao Tong University(上海交通大学计算机学院)
Enabling and Analyzing How to Efficiently Extract Information from Hybrid Long Documents with LLMs
School of Software & Microelectronics, Peking University(北京大学软件与微电子学院);Institute of Software Chinese Academy of Sciences(中国科学院软件研究所);Microsoft(微软公司);University of Technology Sydney(悉尼科技大学)
Multilingual Large Language Models do not comprehend all natural languages to equal degrees
Universitat Autònoma de Barcelona(巴塞罗那自治大学);Institució Catalana de Recerca i Estudis Avançats (ICREA)(加泰罗尼亚研究与高等教育机构);Radboud University Nijmegen(拉德堡德大学奈梅亨分校);Humboldt-Universität zu Berlin(柏林洪堡大学)
A Fine-tuning Dataset and Benchmark for Large Language Models for Protein Understanding
Toursun Synbio(图森合成生物);Johns Hopkins University(约翰斯·霍普金斯大学);University of Cambridge(剑桥大学);Shanghai Institute for Biomedical and Pharmaceutical Technologies(上海生物医药技术研究院);Shanghai AI Laboratory(上海人工智能实验室);Shanghai Jiao Tong University(上海交通大学);UNSW Syndey(新南威尔士大学悉尼分校)
Explore the Potential of LLMs in Misinformation Detection: An Empirical Study
Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);School of Cyber Security, University of Chinese Academy of Sciences(中国科学院大学网络空间安全学院)
Extract Information from Hybrid Long Documents Leveraging LLMs: A Framework and Dataset
Microsoft(微软公司);School of Software & Microelectronics, Peking University(北京大学软件与微电子学院);Institute of Software, Chinese Academy of Sciences(中国科学院软件研究所);Ant Group(蚂蚁集团)
Bridge-AI Lab(Bridge-AI实验室);University of Central Florida(中佛罗里达大学);Auburn University(奥本大学);Wichita State University(威奇托州立大学);Gustavus Adolphus College(古斯塔夫斯阿道夫学院)
Engineering Programme, NUS Graduate School, National University of Singapore(国立新加坡大学整合科学与工程计划);Institute of Data Science (IDS), National University of Singapore(国立新加坡大学数据科学研究所);Department of Computer Science, School of Computing, National University of Singapore(国立新加坡大学计算机科学系)
Walking the Tightrope of LLMs for Software Development: A Practitioners' Perspective
Faculty of Information Technology, Monash University(墨尔本大学信息科技学院);School of Computing and Information Systems, Singapore Management University(新加坡管理大学计算机与信息系统学院)
Integrating Time Series into LLMs via Multi-layer Steerable Embedding Fusion for Enhanced Forecasting
School of Software Engineering Sun Yat-Sen University(软件工程学院中山大学);Institute of Data Science(数据科学研究院);School of Computing(计算机学院);National University of Singapore(新加坡国立大学)