Task-Oriented Semantic Communication in Large Multimodal Models-based Vehicle Networks
Institute of Science and Engineering, Kanazawa University（金泽大学科学与工程研究所）；Department of Electrical and Electronic Engineering, University of Hong Kong（香港大学电子与电气工程系）；College of Computing and Data Science, Nanyang Technological University（南洋理工大学计算与数据科学学院）
Task-oriented semantic communication has emerged as a fundamental approach for enhancing performance in various communication scenarios. While recent advances in Generative Artificial Intelligence (GenAI), such as Large Language Models (LLMs), have been applied to semantic communication designs, the potential of Large Multimodal Models (LMMs) remains largely unexplored. In this paper, we investigate an LMM-based vehicle AI assistant using a Large Language and Vision Assistant (LLaVA) and propose a task-oriented semantic communication framework to facilitate efficient interaction between users and cloud servers. To reduce computational demands and shorten response time, we optimize LLaVA's image slicing to selectively focus on areas of utmost interest to users. Additionally, we assess the importance of image patches by combining objective and subjective user attention, adjusting energy usage for transmitting semantic information. This strategy optimizes resource utilization, ensuring precise transmission of critical information. We construct a Visual Question Answering (VQA) dataset for traffic scenarios to evaluate effectiveness. Experimental results show that our semantic communication framework significantly increases accuracy in answering questions under the same channel conditions, performing particularly well in environments with poor Signal-to-Noise Ratios (SNR). Accuracy can be improved by 13.4% at an SNR of 12dB and 33.1% at 10dB, respectively.
Visual-Oriented Fine-Grained Knowledge Editing for MultiModal Large Language Models
Hefei University of Technology（合肥工业大学）；University of Science and Technology of China（中国科学技术大学）；Hefei Comprehensive National Science Center（合肥综合性国家科学中心）
Specialty-Oriented Generalist Medical AI for Chest CT Screening
Rensselaer Polytechnic Institute（伦斯勒理工学院）；Wake Forest University School of Medicine（维克森林大学医学院）；Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）
Edge-Based Multimodal Sensor Data Fusion with Vision Language Models (VLMs) for Real-time Autonomous Vehicle Accident Avoidance
Department of Civil & Environmental Engineering University of Utah（土木与环境工程系 犹他大学）；Zachry Department of Civil and Environmental Engineering Texas A&M University（扎克里系 土木与环境工程系 德克萨斯农工大学）；Department of Computer Science & Engineering Texas A&M University（计算机科学与工程系 德克萨斯农工大学）
Advancing Autonomous Vehicle Intelligence: Deep Learning and Multimodal LLM for Traffic Sign Recognition and Robust Lane Detection
School of Computer Science and Engineering, Beihang University（北京航空航天大学计算机科学与工程学院）；School of Vehicle and Mobility, Tsinghua University（清华大学车辆与 Mobility 学院）
ReVision: A Dataset and Baseline VLM for Privacy-Preserving Task-Oriented Visual Instruction Rewriting
School of Information（信息学院）；The University of Texas at Austin（德克萨斯大学奥斯汀分校）；Department of Statistics and Data Science（统计与数据科学系）；Yale University（耶鲁大学）；School of Computing and Augmented Intelligence（计算与增强智能学院）
How Far Are Large Multimodal Models from Human-Level Spatial Action? A Benchmark for Goal-Oriented Embodied Navigation in Urban Airspace
Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）；Northeastern University（东北大学）；National University of Defense Technology（国防科技大学）；Shandong University（山东大学）；BNRist, Tsinghua University（清华大学北京信息科学与技术国家研究中心）
OmniFuser: Adaptive Multimodal Fusion for Service-Oriented Predictive Maintenance
School of Software Technology, Zhejiang University（浙江大学软件学院）；School of Computer Science and Technology, East China Normal University（华东师范大学计算机科学与技术学院）；Faculty of Computing, Harbin Institute of Technology（哈尔滨工业大学计算机学院）；Hangzhou School of Automation, Zhejiang Normal University（浙江师范大学杭州自动化学院）；Distributed Systems Group at the TU Wien and with ICREA at the UPF, Barcelona（维也纳大学分布式系统组和巴塞罗那大学ICREA）
Real-time Vehicle-to-Vehicle Communication Based Network Cooperative Control System through Distributed Database and Multimodal Perception: Demonstrated in Crossroads
Zhejiang University（浙江大学）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；University of Michigan（密歇根大学）
Exploring multimodal implicit behavior learning for vehicle navigation in simulated cities
Systems Engineering Department, Federal University of Santa Catarina, Florianopolis, Brazil Faculty of Science；Engineering, Information Technology Åbo Akademi University, Finland
Towards an Ontology for Scenario Definition for the Assessment of Automated Vehicles: An Object-Oriented Framework
TNO；Delft University of Technology（代尔夫特理工大学）；Radboud University（拉德堡德大学）；Symphony Co.（Symphony公司）；getthere（2getthere）；Eindhoven University of Technology（埃因霍温理工大学）
Vehicle-centric Perception via Multimodal Structured Pre-training
Information Materials and Intelligent Sensing Laboratory of Anhui Province（安徽省信息材料与智能感知实验室）；Anhui Provincial Key Laboratory of Multimodal Cognitive Computation（安徽省多模态认知计算重点实验室）；the School of Artificial Intelligence, Anhui University（安徽大学人工智能学院）；School of Computer Science and Technology, Anhui University（安徽大学计算机科学与技术学院）；Institute of Artificial Intelligence, Hefei Comprehensive National Science Center（合肥综合性国家科学中心人工智能研究院）
PhysDrive: A Multimodal Remote Physiological Measurement Dataset for In-vehicle Driver Monitoring
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；The Hong Kong University of Science and Technology（香港科技大学）；Tsinghua University（清华大学）；Sichuan Agricultural University（四川农业大学）
Target-oriented Multimodal Sentiment Classification with Counterfactual-enhanced Debiasing
School of Computer, Electronics and Information（计算机、电子与信息学院）；Guangxi University（广西大学）；Guangxi Key Laboratory of Multimedia Communications（广西多媒体通信与网络技术重点实验室）；School of Sociology and Anthropology（社会学与人类学学院）；Sun Yat-sen University（中山大学）
Multimodal Mathematical Reasoning Embedded in Aerial Vehicle Imagery: Benchmarking, Analysis, and Exploration
Department of Physics, J.K. Institute of Science（J.K.科学研究院物理系）；World Scientific University（世界科学大学）；University of Intelligent Studies（智能研究大学）；East China Normal University（华东师范大学）；Nanyang Technological University（南洋理工大学）；SenseTime Research（商汤科技研究院）；Shanghai Jiao Tong University（上海交通大学）；Harbin Institute of Technology（哈尔滨工业大学）
Towards Generalization-Oriented Models for Vehicle Routing Problems with Mixture-of-Experts
State Key Laboratory of Autonomous Intelligent Unmanned Systems, Beijing Institute of Technology（自主智能无人系统国家重点实验室，北京理工大学）；School of AI, Beijing Institute of Technology（北京理工大学人工智能学院）
Integration of Mixture of Experts and Multimodal Generative AI in Internet of Vehicles: A Survey
Nanyang Technological University（南洋理工大学）；Guangdong University of Technology（广东工业大学）；Singapore University of Technology and Design（新加坡科技设计大学）；University of Sydney（悉尼大学）；City University of Hong Kong（香港城市大学）；Sungkyunkwan University（成均馆大学）；University of Waterloo（滑铁卢大学）
Think Before You Drive: World Model-Inspired Multimodal Grounding for Autonomous Vehicles
University of Macau（澳门大学）；UESTC（电子科技大学）；Purdue University（普渡大学）；McGill University（麦吉尔大学）；Massachusetts Institute of Technology（麻省理工学院）；University of Washington（华盛顿大学）；The Hong Kong University of Science and Technology（香港科学与技术大学）
Looking and Listening Inside and Outside: Multimodal Artificial Intelligence Systems for Driver Safety Assessment and Intelligent Vehicle Decision-Making
Machine Intelligence, Interaction, and Imagination (Mi3) Laboratory（机器智能、交互与想象实验室）；Laboratory for Intelligent and Safe Automobiles (LISA)（智能与安全汽车实验室）；Johns Hopkins University（约翰霍普金斯大学）；Center for Medicinal Cannabis Research (CMCR)（医药大麻研究中心）
GUI-World: A Video Benchmark and Dataset for Multimodal GUI-oriented Understanding
Huazhong University of Science and Technology（华中科技大学）；University of Notre Dame（圣母大学）；Imperial College London（伦敦帝国学院）；Peking University（北京大学）；Microsoft Research（微软研究院）；Lehigh University（理海大学）
3DTTNet: Multimodal Fusion-Based 3D Traversable Terrain Modeling for Off-Road Environments
Shenzhen Automotive Research Institute, Beijing Institute of Technology, Shenzhen（深圳汽车研究院，北京理工大学，深圳）；School of Mechanical Engineering, Beijing Institute of Technology, Beijing（机械工程学院，北京理工大学，北京）；Research Center for Intelligent Computing Systems, Institute of Computing Technology, Chinese Academy of Sciences, Beijing（智能计算系统研究中心，计算技术研究所，中国科学院，北京）
