For stability and reliability of real-world applications, the robustness of DNNs in unimodal tasks has been evaluated. However, few studies consider abnormal situations that a visual question answering (VQA) model might encounter at test time after deployment in the real-world. In this study, we evaluate the robustness of state-of-the-art VQA models to five different anomalies, including worst-case scenarios, the most frequent scenarios, and the current limitation of VQA models. Different from the results in unimodal tasks, the maximum confidence of answers in VQA models cannot detect anomalous inputs, and post-training of the outputs, such as outlier exposure, is ineffective for VQA models. Thus, we propose an attention-based method, which uses confidence of reasoning between input images and questions and shows much more promising results than the previous methods in unimodal tasks. In addition, we show that a maximum entropy regularization of attention networks can significantly improve the attention-based anomaly detection of the VQA models. Thanks to the simplicity, attention-based anomaly detection and the regularization are model-agnostic methods, which can be used for various cross-modal attentions in the state-of-the-art VQA models. The results imply that cross-modal attention in VQA is important to improve not only VQA accuracy, but also the robustness to various anomalies.
From Time Series Analysis to Question Answering: A Survey in the LLM Era
Renmin University of China(中国人民大学);Tsinghua University(清华大学);Hong Kong University of Science and Technology (Guangzhou)(香港科学与技术大学(广州));Aalborg University(奥胡斯大学)
Hawk: Learning to Understand Open-World Video Anomalies
The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));The Hong Kong University of Science and Technology(香港科技大学);SmartMore Corporation(思谋科技);The Chinese University of Hong Kong(香港中文大学);Zhejiang University(浙江大学);Northwestern Polytechnical University(西北工业大学)
Language Models Meet Anomaly Detection for Better Interpretability and Generalizability
Technical University of Munich(慕尼黑工业大学);Munich Center for Machine Learning(慕尼黑机器学习中心);Klinikum Rechts der Isar(Rechts der Isar 医院);Imperial College London(伦敦帝国学院);Helmholtz AI(亥姆霍兹人工智能研究所);Helmholtz Munich(亥姆霍兹慕尼黑中心);King’s College London(伦敦国王学院)
Time-MQA: Time Series Multi-Task Question Answering with Context Enhancement
University of Oxford(牛津大学);PyPOTS Research(PyPOTS研究);University of Texas at Austin(德克萨斯大学奥斯汀分校);Griffith University(格里菲斯大学);Squirrel Ai Learning(Squirrel Ai学习)
OptIForest: Optimal Isolation Forest for Anomaly Detection
Macquarie University(麦考瑞大学);CSIRO’s Data61(联邦科学与工业研究组织数据61研究所);Qufu Normal University(曲阜师范大学);Nanjing University(南京大学);Nanjing University of Information Science and Technology(南京信息工程大学)
MedAD-R1: Consistency-Reinforced Policy Optimization for Interpretable Medical Anomaly Detection
School of Informatics, Xiamen University(厦门大学信息学院);Institute of Artificial Intelligence, Xiamen University(厦门大学人工智能研究院);Zhejiang Expressway Co., Ltd.(浙江高速公路有限公司);School of Science and Engineering, Chinese University of Hong Kong(香港中文大学科学与工程学院);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室)
3D-RAD: A Comprehensive 3D Radiology Med-VQA Dataset with Multi-Temporal Analysis and Diverse Diagnostic Tasks
ZJU-Angelalign R&D Center for Intelligence Healthcare(浙江大学智能医疗研发中心);Zhejiang University(浙江大学);Zhejiang Key Laboratory of Medical Imaging Artificial Intelligence(浙江省医学影像人工智能重点实验室);Guangdong Institute of Intelligence Science and Technology(广东省智能科学与技术研究院)
Effectiveness Assessment of Recent Large Vision-Language Models
Mohamed Bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学);Sichuan University(四川大学);Tianjin University(天津大学);Australian National University(澳大利亚国立大学);Harbin Institute of Technology(哈尔滨工业大学);Nankai University(南开大学)
AssistPDA: An Online Video Surveillance Assistant for Video Anomaly Prediction, Detection, and Analysis
Xidian University(西安电子科技大学);National University of Singapore(新加坡国立大学);Northwestern Polytechnical University(西北工业大学);Singapore Management University(新加坡管理大学)
CNN Attention Guidance for Improved Orthopedics Radiographic Fracture Classification
Australian Institute for Machine Learning(澳大利亚机器学习研究所);University of Adelaide(阿德莱德大学);Peter Faber Business School, Australian Catholic University(澳大利亚天主教大学彼得·法伯商学院);Australian Catholic University(澳大利亚天主教大学);Amsterdam UMC(阿姆斯特丹大学医学中心);Flinders Medical Centre(弗林德斯医学中心);Flinders University(弗林德斯大学);Groningen University Medical Centre(格罗宁根大学医学中心)
Geometry-Aware Semantic Reasoning for Training Free Video Anomaly Detection
School of Computing, Engineering \& Mathematical Sciences, La Trobe University, Melbourne, Australia School of Engineering;Applied Sciences, GIFT University, Gujranwala 52250, Pakistan
VANE-Bench: Video Anomaly Evaluation Benchmark for Conversational LMMs
Mohamed Bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学);University of Edinburgh(爱丁堡大学);Khalifa University(哈利法大学);Linköping University(林雪平大学);Australian National University(澳大利亚国立大学)