Today's advanced driver assistance systems (ADAS), like adaptive cruise control or rear collision warning, are finding broader adoption across vehicle classes. Integrating such advanced, multimodal Large Language Models (LLMs) on board a vehicle, which are capable of processing text, images, audio, and other data types, may have the potential to greatly enhance passenger comfort. Yet, an LLM's hallucinations are still a major challenge to be addressed. In this paper, we systematically assessed potential hallucination detection strategies for such LLMs in the context of object detection in vision-based data on the example of pedestrian detection and localization. We evaluate three hallucination detection strategies applied to two state-of-the-art LLMs, the proprietary GPT-4V and the open LLaVA, on two datasets (Waymo/US and PREPER CITY/Sweden). Our results show that these LLMs can describe a traffic situation to an impressive level of detail but are still challenged for further analysis activities such as object localization. We evaluate and extend hallucination detection approaches when applying these LLMs to video sequences in the example of pedestrian detection. Our experiments show that, at the moment, the state-of-the-art proprietary LLM performs much better than the open LLM. Furthermore, consistency enhancement techniques based on voting, such as the Best-of-Three (BO3) method, do not effectively reduce hallucinations in LLMs that tend to exhibit high false negatives in detecting pedestrians. However, extending the hallucination detection by including information from the past helps to improve results.
Enhancing Shape Perception and Segmentation Consistency for Industrial Image Inspection
School of Artificial Intelligence, Beijing University of Posts and Telecommunications（人工智能学院，北京邮电大学）；Ricoh Software Research Center (Beijing) Co., Ltd（Ricoh软件研究中心（北京）有限公司）
Enhancing Perception of Key Changes in Remote Sensing Image Change Captioning
National Engineering Research Center for Multimedia Software, School of Computer Science, Wuhan University（武汉大学计算机学院国家多媒体软件工程技术研究中心）；Hubei Luojia Laboratory（湖北珞珈实验室）；School of Urban Design, Wuhan University（武汉大学城市设计学院）；School of Remote Sensing and Information Engineering, Wuhan University（武汉大学遥感信息工程学院）
YOLO with Kolmogorov-Arnold networks and vision-language foundation models for interpretable object detection with trustworthy multimodal AI in computer vision perception
University of Bath（巴斯大学）；Zhejiang University（浙江大学）
PreSight: Enhancing Autonomous Vehicle Perception with City-Scale NeRF Priors
IIIS, Tsinghua University（清华大学信息科学技术学院（简称：姚期智讲座教授信息技术科学学院））；University of Science and Technology Beijing（北京科技大学）；University of Southern California（南加州大学）；Shanghai Qi Zhi Institute（上海期智研究院）
VideoChat-R1: Enhancing Spatio-Temporal Perception via Reinforcement Fine-Tuning
Shanghai AI Laboratory（上海人工智能实验室）；Nanjing University（南京大学）；Zhejiang University（浙江大学）；University of Science and Technology of China（中国科学技术大学）；Shanghai Innovation Institute（上海创新研究院）；Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究所）
VTPerception-R1: Enhancing Multimodal Reasoning via Explicit Visual and Textual Perceptual Grounding
Fudan University（复旦大学）；Shanghai AI Laboratory（上海人工智能实验室）；The University of Hong Kong（香港大学）；Shenzhen University（深圳大学）；University of Science and Technology of China（中国科学技术大学）
Imprinto: Enhancing Infrared Inkjet Watermarking for Human and Machine Perception
DFKI & Saarland University（DFKI与萨尔兰大学）；MIT CSAIL（MIT计算机科学与人工智能实验室）；Virginia Tech（弗吉尼亚理工大学）；Universidad Carlos III de Madrid（马德里第三大学）；Adobe Research（Adobe研究）
CoDS: Enhancing Collaborative Perception in Heterogeneous Scenarios via Domain Separation
Key Laboratory of Big Data & Artificial Intelligence in Transportation (Beijing Jiaotong University), Ministry of Education and School of Computer Science and Technology, Beijing Jiaotong University（大数据与人工智能交通实验室（北京交通大学）及计算机科学与技术学院，北京交通大学）
GUI-ARP: Enhancing Grounding with Adaptive Region Perception for GUI Agents
Wuhan University（武汉大学）；Sun Yat-sen University（中山大学）；Alibaba Group（阿里巴巴集团）；East China Normal University（华东师范大学）；University of Electronic Science and Technology of China（电子科技大学）
Enhancing Perception Capabilities of Multimodal LLMs with Training-Free Fusion
ZIP Lab, Zhejiang University（浙江大学 ZIP 实验室）；South China University of Technology（华南理工大学）；Pazhou Lab（琶洲实验室）；Key Laboratory of Big Data and Intelligent Robot, Ministry of Education（大数据与智能机器人教育部重点实验室）
SSR: Enhancing Depth Perception in Vision-Language Models via Rationale-Guided Spatial Reasoning
Westlake University（西湖大学）；Zhejiang University（浙江大学）；Harbin Institute of Technology（哈尔滨工业大学）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Shanghai Innovation Institute（上海创新研究院）
All Vehicles Can Lie: Efficient Adversarial Defense in Fully Untrusted-Vehicle Collaborative Perception via Pseudo-Random Bayesian Inference
School of Cyber Science and Engineering, Wuhan University（武汉大学网络科学与工程学院）；Cyberspace Institute of Advanced Technology, Guangzhou University（广州大学先进技术网络研究院）
Enhancing Online Road Network Perception and Reasoning with Standard Definition Maps
Bosch North America（博世北美）；Bosch Center for AI (BCAI)（博世人工智能中心）；Robert Bosch GmbH（罗伯特·博世有限公司）；Contextual Robotics Institute（情境机器人学研究所）；UC San Diego（加州大学圣迭戈分校）
SoPE: Spherical Coordinate-Based Positional Embedding for Enhancing Spatial Perception of 3D LVLMs
University of Macau（澳门大学）；University of Science and Technology of China（中国科学技术大学）；Shanghai Jiaotong University（上海交通大学）；Hefei University of Technology（合肥工业大学）；National University of Singapore（新加坡国立大学）
VisDoT : Enhancing Visual Reasoning through Human-Like Interpretation Grounding and Decomposition of Thought
Department of Computer Science and Artificial Intelligence, Dongguk University（东国大学计算机科学与人工智能系）；Department of Electronics and Electrical Engineering, Dongguk University（东国大学电子与电气工程系）
Enhancing Predictability of Multi-Tenant DNN Inference for Autonomous Vehicles' Perception
Department of Electrical Engineering and Computer Science, University of Michigan（电气工程与计算机科学系，密歇根大学）；Department of Computer Science and Engineering, Yonsei University（计算机科学与工程系，延世大学）；Department of Computer and Information Sciences, University of Delaware（计算机与信息科学系，特拉华大学）
