Existing semi-supervised video anomaly detection (VAD) methods often struggle with detecting complex anomalies involving object interactions and generally lack explainability. To overcome these limitations, we propose a novel VAD framework leveraging Multimodal Large Language Models (MLLMs). Unlike previous MLLM-based approaches that make direct anomaly judgments at the frame level, our method focuses on extracting and interpreting object activity and interactions over time. By querying an MLLM with visual inputs of object pairs at different moments, we generate textual descriptions of the activity and interactions from nominal videos. These textual descriptions serve as a high-level representation of the activity and interactions of objects in a video. They are used to detect anomalies during test time by comparing them to textual descriptions found in nominal training videos. Our approach inherently provides explainability and can be combined with many traditional VAD methods to further enhance their interpretability. Extensive experiments on benchmark datasets demonstrate that our method not only detects complex interaction-based anomalies effectively but also achieves state-of-the-art performance on datasets without interaction anomalies.
Towards Explainable Industrial Anomaly Detection via Knowledge-Guided Latent Reasoning
School of Cyber Science and Technology, Shenzhen Campus of Sun Yat-sen University（中山大学深圳校区计算机科学与技术学院）；School of Artificial Intelligence and Robotics, Hunan University（湖南大学人工智能与机器人学院）；Department of Computer and Information Science, University of Macau（澳门大学计算机与信息科学系）
The aim of this work is to detect and automatically generate high-level explanations of anomalous events in video. Understanding the cause of an anomalous event is crucial as the required response is dependant on its nature and severity. Recent works typically use object or action classifier to detect and provide labels for anomalous events. However, this constrains detection systems to a finite set of known classes and prevents generalisation to unknown objects or behaviours. Here we show how to robustly detect anomalies without the use of object or action classifiers yet still recover the hi…
VADTree: Explainable Training-Free Video Anomaly Detection via Hierarchical Granularity-Aware Tree
School of Software, Xi’an Jiaotong University（西安交通大学软件学院）；China Railway Xi’an Group（中国铁路西安集团）；College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）
School of Computer Science & Software Engineering, Shenzhen University（深圳大学计算机科学与软件工程学院）；School of Computing and Information Technology, Great Bay University（海湾大学计算机与信息学院）
Multimodal Industrial Anomaly Detection via Geometric Prior
Faculty of Data Science, City University of Macau（澳门城市大学数据科学学院）；Key Laboratory of Computing Power Network and Information Security, Ministry of Education, Shandong Computer Science Center (National Supercomputer Center in Jinan), Qilu University of Technology (Shandong Academy of Sciences)（教育部计算功率网络与信息安全重点实验室，山东计算机科学中心（济南国家超算中心），齐鲁工业大学（山东省科学院））；Shandong Provincial Key Laboratory of Computing Power Internet and Service Computing, Shandong Fundamental Research Center for Computer Science（山东省计算功率互联网与服务计算重点实验室，山东省计算机科学基础研究中心）
Memoryless Multimodal Anomaly Detection via Student-Teacher Network and Signed Distance Learning
Mine Digitization Engineering Research Center of the Ministry of Education（教育部矿山数字化工程研究中心）；School of Computer Science and Technology, China University of Mining and Technology（中国矿业大学计算机科学与技术学院）；Sun Yueqi Honors College, China University of Mining and Technology（中国矿业大学孙越崎学院）
BridgeNet: A Unified Multimodal Framework for Bridging 2D and 3D Industrial Anomaly Detection
Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（深圳先进技术研究院，中国科学院）；University of Chinese Academy of Sciences（中国科学院大学）；Shenzhen University of Advanced Technology（深圳先进技术大学）；State Key Lab of IOTSC, Department of CIS, University of Macau（物联网科学国家重点实验室，澳门大学）
SGANet: Semantic and Geometric Alignment for Multimodal Multi-view Anomaly Detection
Smart Manufacturing Thrust, The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州）智能制造学域）；College of Mechanical and Vehicle Engineering, Hunan University（湖南大学机械与运载工程学院）；The Hong Kong University of Science and Technology（香港科技大学）
An Explainable Anomaly Detection Framework for Monitoring Depression and Anxiety Using Consumer Wearable Devices
University of Sheffield（谢菲尔德大学）；University College London（伦敦大学学院）；South London and Maudsley NHS Foundation Trust（南伦敦和莫兹利国民健康服务体系基金会信托）；NHS Foundation Trust（国民健康服务体系基金会信托）；Health Data Research UK London（英国健康数据研究伦敦中心）
Existing anomaly detection paradigms overwhelmingly focus on training detection models using exclusively normal data or unlabeled data (mostly normal samples). One notorious issue with these approaches is that they are weak in discriminating anomalies from normal samples due to the lack of the knowledge about the anomalies. Here, we study the problem of few-shot anomaly detection, in which we aim at using a few labeled anomaly examples to train sample-efficient discriminative detection models. To address this problem, we introduce a novel weakly-supervised anomaly detection framework to train …
Enhancing Weakly Supervised Multimodal Video Anomaly Detection through Text Guidance
School of Computer Science and Technology, Hangzhou Dianzi University（杭州电子科技大学计算机科学与技术学院）；Alibaba Cloud（阿里云）；College of Information Science and Electronic Engineering, Zhejiang University（浙江大学信息科学与电子工程学院）
University of Massachusetts Amherst（马萨诸塞大学阿默斯特分校）；Mitsubishi Electric Research Laboratories（三菱电机研究实验室）
We develop a novel framework for single-scene video anomaly localization that allows for human-understandable reasons for the decisions the system makes. We first learn general representations of objects and their motions (using deep networks) and then use these representations to build a high-level, location-dependent model of any particular scene. This model can be used to detect anomalies in new videos of the same scene. Importantly, our approach is explainable - our high-level appearance and motion features can provide human-understandable reasons for why any part of a video is classified …
EXACT: an explainable anomaly-aware vision foundation model for analysis of 3D chest CT
School of Biomedical Engineering, Tsinghua University（清华大学生物医学工程学院）；Department of Radiology, Mianyang Central Hospital（绵阳市中心医院放射科）；Center for Biomedical Imaging Research, Tsinghua University（清华大学生物医学成像研究中心）
KAIST（韩国科学技术院）；Technical University of Munich（慕尼黑工业大学）
The ambiguity of the decision-making process has been pointed out as the main obstacle to applying the deep learning-based method in a practical way in spite of its outstanding performance. Interpretability could guarantee the confidence of deep learning system, therefore it is particularly important in the medical field. In this study, a novel deep network is proposed to explain the diagnostic decision with visual pointing map and diagnostic sentence justifying result simultaneously. For the purpose of increasing the accuracy of sentence generation, a visual word constraint model is devised i…
Bridging the gap between Performance and Interpretability: An Explainable Disentangled Multimodal Framework for Cancer Survival Prediction
AI Technology for Life（人工智能技术与生命科学）；Department of Information and Computing Sciences（信息与计算科学系）；Department of Biology（生物学系）；Utrecht University（乌得勒支大学）；Department of Metabolic Diseases（代谢疾病部门）；Wilhelmina Children’s Hospital（维廉明娜儿童医院）；University Medical Center Utrecht（乌得勒支大学医学中心）；Regenerative Medicine Center Utrecht（乌得勒支再生医学中心）；Computational Pathology（计算病理学）；Department of Pathology（病理学系）；The Netherlands Cancer Institute（荷兰癌症研究所）；Visualization and Graphics（可视化与图形学）；The Netherlands Cancer Institute, Amsterdam（荷兰癌症研究所，阿姆斯特丹）
MARE: Multimodal Alignment and Reinforcement for Explainable Deepfake Detection via Vision-Language Models
School of Computer Science and Engineering, MoE Key Laboratory of Information Technology, Guangdong Province Key Laboratory of Information Security Technology, Sun Yat-sen University, Guangzhou 510006, China（计算机科学与工程学院，信息技术MOE实验室，广东省信息安全技术重点实验室，中山大学，广州510006，中国）；State Key Laboratory of Mathematical Engineering and Advanced Computing（数学工程与先进计算国家重点实验室）；Department of Computer and Information Science, University of Macau.（计算机与信息科学系，澳门大学）
School of Computer Science and Engineering, Beihang University（北京航空航天大学计算机科学与工程学院）；Key Laboratory of Intelligent Bionic Unmanned Systems, Ministry of Education（教育部智能仿生无人系统重点实验室）；School of Artificial Intelligence, University of Science and Technology Beijing（北京科技大学人工智能学院）；School of Computer Science and Technology, Beijing Jiaotong University（北京交通大学计算机科学与技术学院）
Towards Explainable Bilingual Multimodal Misinformation Detection and Localization
University of Liverpool, United Kingdom（利物浦大学）；National University of Singapore, Singapore（新加坡国立大学）；The Chinese University of Hong Kong, Shenzhen, China（香港中文大学（深圳））
Multimodal Doctor-in-the-Loop: A Clinically-Guided Explainable Framework for Predicting Pathological Response in Non-Small Cell Lung Cancer
Research Unit of Computer Systems and Bioinformatics, Department of Engineering（计算机系统与生物信息学研究单位，工程系）；Research Unit of Radiation Oncology, Department of Medicine and Surgery（放射肿瘤学研究单位，医学与外科系）；Research Unit of Anatomical Pathology, Department of Medicine and Surgery（病理学研究单位，医学与外科系）
