GEAR-Seg: A Grounded Explainable Agent for Reasoning Segmentation and Data Engine
College of Biosystems Engineering and Food Science, Zhejiang University（浙江大学生物系统工程与食品科学学院）；ZJU-Hangzhou Global Scientific and Technological Innovation Center, Zhejiang University（浙江大学杭州国际科创中心）
Reasoning segmentation requires localizing targets based on complex, implicit queries. Current end-to-end models typically entangle perception and deduction into an opaque black box, severely limiting interpretability and scalability. To address this, we propose GEAR-Seg (Grounded Explainable Agent for Reasoning Segmentation), an explicitly decoupled agent that shifts the paradigm by translating visual pixels into dense, attribute-rich text. By decoupling class-agnostic segmentation, semantic description, and Large Language Model (LLM) deduction, GEAR-Seg transforms implicit reasoning into an explicit, trackable logic chain. As a zero-shot inference framework, it achieves highly competitive performance across diverse reasoning and fine-grained referring segmentation benchmarks. Furthermore, GEAR-Seg inherently functions as a highly scalable data engine. Utilizing this engine, we construct GEAR-131K, a massive benchmark (over 38k images, 656k QA-mask pairs) introducing a multifaceted taxonomy tailored for complex real-world manipulation-oriented reasoning. Finally, distillation experiments demonstrate that lightweight models supervised exclusively by our automated pipeline closely match the upper-bound performance of costly human-annotated baselines.
No Masks Needed: Explainable AI for Deriving Segmentation from Classification
Dept. Electrical and Electronic Engineering, Imperial College London（帝国理工学院伦敦分校电子与电气工程系）；Centre for Vision, Speech and Signal Processing, University of Surrey（萨里大学视觉、语音与信号处理中心）
Fraunhofer Institute for Telecommunications, Heinrich Hertz Institute, HHI（弗劳恩霍夫电信研究所，海因里希·赫兹研究所，HHI）；Humboldt University Berlin（柏林洪堡大学）
The new era of image segmentation leveraging the power of Deep Neural Nets (DNNs) comes with a price tag: to train a neural network for pixel-wise segmentation, a large amount of training samples has to be manually labeled on pixel-precision. In this work, we address this by following an indirect solution. We build upon the advances of the Explainable AI (XAI) community and extract a pixel-wise binary segmentation from the output of the Layer-wise Relevance Propagation (LRP) explaining the decision of a classification network. We show that we achieve similar results compared to an established …
James Cook University（詹姆斯库克大学）；La Trobe University（乐卓博大学）；Beijing University of Technology（北京工业大学）
Recently vision transformer models have become prominent models for a range of vision tasks. These models, however, are usually opaque with weak feature interpretability. Moreover, there is no method currently built for an intrinsically interpretable transformer, which is able to explain its reasoning process and provide a faithful explanation. To close these crucial gaps, we propose a novel vision transformer dubbed the eXplainable Vision Transformer (eX-ViT), an intrinsically interpretable transformer model that is able to jointly discover robust interpretable features and perform the predic…
Explainable Continuous-Time Mask Refinement with Local Self-Similarity Priors for Medical Image Segmentation
School of Computer Engineering, Kalinga Institute of Industrial Technology, Bhubaneswar, India（计算机工程学院，印度克里希纳加尔工业技术学院，布巴内斯瓦尔，印度）；AmygdalaAI-India Lab, Bhubaneswar, India（AmygdalaAI-印度实验室，布巴内斯瓦尔，印度）
Entropy-Centric Explainable AI for Remote Sensing Image Segmentation
Faculty of Engineering, Lebanese University（黎巴嫩大学工程学院）；University of Paris-Est Créteil (UPEC)（巴黎东部克雷泰伊大学）；EFREI（EFREI学院）；National Center for Remote Sensing - CNRS（国家遥感中心 - 法国国家科学研究中心）
Semantic medical image segmentation using deep learning has recently achieved high accuracy, making it appealing to clinical problems such as radiation therapy. However, the lack of high-quality semantically labelled data remains a challenge leading to model brittleness to small shifts to input data. Most works require extra data for semi-supervised learning and lack the interpretability of the boundaries of the training data distribution during training, which is essential for model deployment in clinical practice. We propose a fully supervised generative framework that can achieve generalisa…
Understanding intermediate layers of a deep learning model and discovering the driving features of stimuli have attracted much interest, recently. Explainable artificial intelligence (XAI) provides a new way to open an AI black box and makes a transparent and interpretable decision. This paper proposes a new explainable convolutional neural network (XCNN) which represents important and driving visual features of stimuli in an end-to-end model architecture. This network employs encoder-decoder neural networks in a CNN architecture to represent regions of interest in an image based on its catego…
Explainable Binary Classification of Separable Shape Ensembles
Applied & Computational Mathematics Div.（应用与计算数学部）；National Institute of Standards & Technology（国家标准与技术研究院）；Dept. of Mathematics and Statistics（数学与统计学系）；Portland State University（波特兰州立大学）；Computational Science Center（计算科学中心）；National Laboratory of the Rockies（落基山脉国家实验室）
The complex driving environment brings great challenges to the visual perception of autonomous vehicles. It's essential to extract clear and explainable information from the complex road and traffic scenarios and offer clues to decision and control. However, the previous scene explanation had been implemented as a separate model. The black box model makes it difficult to interpret the driving environment. It cannot detect comprehensive textual information and requires a high computational load and time consumption. Thus, this study proposed a comprehensive and efficient textual explanation mod…
Explainable Quantum Machine Learning for Multispectral Images Segmentation: Case Study
Systems Research Institute Polish Academy of Sciences（波兰科学院系统研究所）；Nicolaus Copernicus Astronomical Center, Polish Academy of Sciences（波兰科学院尼古拉·哥白尼天文中心）；Center of Excellence in Artificial Intelligence, AGH University（AGH大学人工智能卓越中心）
We propose an explainable model to generate semantic color labels for person search. In this context, persons are described from their semantic parts, such as hat, shirt, etc. Person search consists in looking for people based on these descriptions. In this work, we aim to improve the accuracy of color labels for people. Our goal is to handle the high variability of human perception. Existing solutions are based on hand-crafted features or learnt features that are not explainable. Moreover most of them only focus on a limited set of colors. We propose a method based on binary search trees and …
Indian Institute of Technology Jodhpur（印度理工学院焦特布尔分校）
Machine learning (ML) in general and deep learning (DL) in particular has become an extremely popular tool in several vision applications (like object detection, super resolution, segmentation, object tracking etc.). Almost in parallel, the issue of explainability in ML (i.e. the ability to explain/elaborate the way a trained ML model arrived at its decision) in vision has also received fairly significant attention from various quarters. However, we argue that the current philosophy behind explainable ML suffers from certain limitations, and the resulting explanations may not meaningfully unco…
Universitat Oberta de Catalunya（加泰罗尼亚开放大学）；Universitat Rovira i Virgili（罗维拉-威尔吉利大学）；Massachusetts Institute of Technology（麻省理工学院）
At the moment, urban mobility research and governmental initiatives are mostly focused on motor-related issues, e.g. the problems of congestion and pollution. And yet, we can not disregard the most vulnerable elements in the urban landscape: pedestrians, exposed to higher risks than other road users. Indeed, safe, accessible, and sustainable transport systems in cities are a core target of the UN's 2030 Agenda. Thus, there is an opportunity to apply advanced computational tools to the problem of traffic safety, in regards especially to pedestrians, who have been often overlooked in the past. T…
Muroran Institute of Technology（室兰工业大学）；Niigata University of Healthcare and Welfare（新泻医疗福祉大学）
Nuclear segmentation, classification and quantification within Haematoxylin & Eosin stained histology images enables the extraction of interpretable cell-based features that can be used in downstream explainable models in computational pathology. The Colon Nuclei Identification and Counting (CoNIC) Challenge is held to help drive forward research and innovation for automatic nuclei recognition in computational pathology. This report describes our proposed method submitted to the CoNIC challenge. Our method employs a multi-task learning framework, which performs a panoptic segmentation task and…
Kartezio: Evolutionary Design of Explainable Pipelines for Biomedical Image Analysis
L’Institut National de la Santé et de la Recherche Médicale (INSERM)（法国国家健康与医学研究院）；Centre National de la Recherche Scientifique (CNRS)（法国国家科学研究中心）；University of Toulouse III - Paul Sabatier（图卢兹第三大学 - 保罗·萨巴蒂埃大学）；Institut Universitaire du Cancer-Oncopole de Toulouse (IUCT)（图卢兹大学癌症研究所 - 昂科波勒）；Centre Hospitalier Universitaire de Bordeaux（波尔多大学医院中心）；Université de Bordeaux（波尔多大学）；University of Toulouse - Institut de Recherche en Informatique de Toulouse (IRIT)（图卢兹大学 - 图卢兹计算机科学研究所）；University of Oxford（牛津大学）；Kennedy Institute of Rheumatology (KIR)（肯尼迪风湿病研究所）；Leibniz Institute for New Materials（莱布尼茨新材料研究所）
Due to the cross-domain distribution shift aroused from diverse medical imaging systems, many deep learning segmentation methods fail to perform well on unseen data, which limits their real-world applicability. Recent works have shown the benefits of extracting domain-invariant representations on domain generalization. However, the interpretability of domain-invariant features remains a great challenge. To address this problem, we propose an interpretable Bayesian framework (BayeSeg) through Bayesian modeling of image and label statistics to enhance model generalizability for medical image seg…
GEAR: GEometry-motion Alternating Refinement for Articulated Object Modeling with Gaussian Splatting
Key Laboratory of AI Safety of Chinese Academy of Sciences (CAS), Institute of Computing Technology, CAS（中国科学院人工智能安全重点实验室，中国科学院计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）
National Taiwan University（台湾大学）；Chang Gung Memorial Hospital（长庚纪念医院）；Chang Gung University（长庚大学）
Automated vascular segmentation on optical coherence tomography angiography (OCTA) is important for the quantitative analyses of retinal microvasculature in neuroretinal and systemic diseases. Despite recent improvements, artifacts continue to pose challenges in segmentation. Our study focused on removing the speckle noise artifact from OCTA images when performing segmentation. Speckle noise is common in OCTA and is particularly prominent over large non-perfusion areas. It may interfere with the proper assessment of retinal vasculature. In this study, we proposed a novel Supervision Vessel Seg…
Explainable AI for Biodiversity Monitoring and Ecological Image Analysis
Pratt School of Engineering, Duke University（杜克大学工程学院）；British Antarctic Survey（英国南极调查局）；Department of Applied Mathematics and Theoretical Physics, University of Cambridge（剑桥大学应用数学与理论物理系）；Duke University Division of Marine Science and Conservation, Nicholas School of the Environment, Duke University Marine Laboratory（杜克大学海洋科学与保护分会，尼古拉斯环境学院，杜克大学海洋实验室）
PLAS-Net: Pixel-Level Area Segmentation for UAV-Based Beach Litter Monitoring
Graduate School of Frontier Sciences, The University of Tokyo, Japan（前沿科学研究生院，东京大学，日本）；Department of Urban Informatics, Shenzhen University, China（城市信息化系，深圳大学，中国）
Image Segmentation with Large Language Models: A Survey with Perspectives for Intelligent Transportation Systems
Department of Computer Science, Iowa State University, Ames, IA, USA（Iowa State University）；Department of Civil, Construction and Environmental Engineering, Iowa State University, Ames, IA, USA（Iowa State University）
Exploiting XAI maps to improve MS lesion segmentation and detection in MRI
University Hospital Basel（巴塞尔大学医院）；University of Basel（巴塞尔大学）；Research Center for Clinical Neuroimmunology and Neuroscience Basel (RC2NB)（巴塞尔临床神经免疫学与神经科学研究中心）；HES-SO Valais-Wallis University of Applied Sciences and Arts Western Switzerland（瑞士西部高等专业学院瓦莱州瓦利斯分校）；CIBM Center for Biomedical Imaging（生物医学成像中心（CIBM））
本文提出从 MS 病灶显著性映射中提取放射组学特征，并训练逻辑回归区分 TP/FP，显著提升 3D U-Net 的 MRI 病灶分割检测性能。
This paper contributes to automating medical image segmentation by proposing generative adversarial network-based models to segment both polyps and instruments in endoscopy images. A major contribution of this work is to provide explanations for the predictions using a layer-wise relevance propagation approach designating which input image pixels are relevant to the predictions and to what extent. On the polyp segmentation task, the models achieved 0.84 of accuracy and 0.46 on Jaccard index. On the instrument segmentation task, the models achieved 0.96 of accuracy and 0.70 on Jaccard index. Th…
Demystifying Deep Learning-based Brain Tumor Segmentation with 3D UNets and Explainable AI (XAI): A Comparative Analysis
School of Artificial Intelligence and Robotics, Xiamen University Malaysia（人工智能与机器人学院，马来西亚厦门大学）；School of Computing and Data Science, Xiamen University Malaysia（计算与数据科学学院，马来西亚厦门大学）；Georgia Institute of Technology（佐治亚理工学院）；School of Social and Health Sciences, James Cook University（社会科学与健康科学学院，詹姆斯库克大学）；College of Healthcare Sciences, James Cook University（健康科学学院，詹姆斯库克大学）
Explainable LiDAR 3D Point Cloud Segmentation and Clustering for Detecting Airplane-Generated Wind Turbulence
Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）；ScaDS.AI, TU Dresden（德累斯顿工业大学ScaDS.AI）；Institute of Atmospheric Physics, German Aerospace Center（德国航空航天中心大气物理研究所）；RWTH Aachen University（亚琛工业大学）
From Classification to Segmentation with Explainable AI: A Study on Crack Detection and Growth Monitoring
Intelligent Maintenance and Operations Systems (IMOS), EPFL（智能维护与运营系统研究所（IMOS），洛桑联邦理工学院）；Environmental Computational Science and Earth Observation (ECEO), EPFL（环境计算科学与地球观测研究所（ECEO），洛桑联邦理工学院）
IIIT Hyderabad（印度理工学院海得拉巴分校（IIIT Hyderabad））；India（印度（注意：根据提取规则，国家“India”不属于机构，此处可能是地址信息，原文本中“Affiliation: IIIT Hyderabad, India, 500032”，主要机构为IIIT Hyderabad，“India”和“500032”为地址相关，按要求不提取，因此仅保留IIIT Hyderabad）}]} 修正：根据规则，机构粒度到大学/学院级别即可，且不要提取国家、城市等。原文本中“Affiliation: IIIT Hyderabad, India, 500032”，核心机构是“IIIT Hyderabad”，印度（India）是国家，不属于机构，因此正确输出应为：{）
Deep learning based approaches to Computer Aided Diagnosis (CAD) typically pose the problem as an image classification (Normal or Abnormal) problem. These systems achieve high to very high accuracy in specific disease detection for which they are trained but lack in terms of an explanation for the provided decision/classification result. The activation maps which correspond to decisions do not correlate well with regions of interest for specific diseases. This paper examines this problem and proposes an approach which mimics the clinical practice of looking for an evidence prior to diagnosis. …
Counterfactuals and Uncertainty-Based Explainable Paradigm for the Automated Detection and Segmentation of Renal Cysts in Computed Tomography Images: A Multi-Center Study
Maastricht University（马斯特里赫特大学）；Memorial Sloan Kettering Cancer Center（纪念斯隆凯特琳癌症中心）；Maastricht University Medical Center+（马斯特里赫特大学医学中心+）；University of California-San Francisco（加州大学旧金山分校）；University of Amsterdam（阿姆斯特丹大学）；Amsterdam UMC（阿姆斯特丹大学医学中心）；GROW Research Institute for Oncology and Reproduction（GROW肿瘤学与生殖学研究所）
University of Electronic Science and Technology of China（电子科技大学）；SenseTime Research（商汤科技研究院）；University Hospitals Leuven（鲁汶大学医院）；King’s College London（伦敦国王学院）；University College London（伦敦大学学院）
Accurate medical image segmentation is essential for diagnosis and treatment planning of diseases. Convolutional Neural Networks (CNNs) have achieved state-of-the-art performance for automatic medical image segmentation. However, they are still challenged by complicated conditions where the segmentation target has large variations of position, shape and scale, and existing CNNs have a poor explainability that limits their application to clinical decisions. In this work, we make extensive use of multiple attentions in a CNN architecture and propose a comprehensive attention-based CNN (CA-Net) f…
XAI-CLIP: ROI-Guided Perturbation Framework for Explainable Medical Image Segmentation in Multimodal Vision-Language Models
King Fahd University of Petroleum and Minerals（国王法赫德石油和矿物大学）；Massachusetts Institute of Technology（麻省理工学院）；Imperial College London（伦敦帝国学院）；KFUPM-SDAIA Joint Research Centre for Artificial Intelligence（KFUPM-SDAIA联合人工智能研究中心）
PEFT-MedSAM: Efficient Fine-Tuning of Medical Foundation Models for Explainable Skin Lesion Segmentation
Department of Computer Science, Quaid-e-Awam University of Engineering, Sciences & Technology（计算机科学系，卡迪尔-阿瓦姆工程、科学与技术大学）；Department of Artificial Intelligence, Quaid-e-Awam University of Engineering, Sciences & Technology（人工智能系，卡迪尔-阿瓦姆工程、科学与技术大学）；Department of Computer Science, Sindh Madressatul Islam University, City Campus, Karachi（计算机科学系， Sind 阿里斯坦伊斯兰大学，卡拉奇城校区）；Department of Computer Science and Digital Technologies, School of Architecture, Computing and Engineering, University of East London（计算机科学与数字技术系，建筑、计算与工程学院，东伦敦大学）
Recently, the coronavirus disease 2019 (COVID-19) has caused a pandemic disease in over 200 countries, influencing billions of humans. To control the infection, identifying and separating the infected people is the most crucial step. The main diagnostic tool is the Reverse Transcription Polymerase Chain Reaction (RT-PCR) test. Still, the sensitivity of the RT-PCR test is not high enough to effectively prevent the pandemic. The chest CT scan test provides a valuable complementary tool to the RT-PCR test, and it can identify the patients in the early-stage with high sensitivity. However, the che…
PotatoGANs: Utilizing Generative Adversarial Networks, Instance Segmentation, and Explainable AI for Enhanced Potato Disease Identification and Classification
Ahsanullah University of Science and Technology（阿沙努拉科学与技术大学）；The University of Western Australia（西澳大学）
Weakly-supervised segmentation using inherently-explainable classification models and their application to brain tumour classification
Knowledge Engineering Group, Faculty of Computer Science, Otto von Guericke University Magdeburg, Germany；Biomedical Magnetic Resonance, Faculty of Nature Sciences, Otto von Guericke University Magdeburg, Germany；Genomics Research Centre, Human Technopole, Milan, Italy；Institute for Medical Engineering, Faculty of Electrical Engineering；Information Technology, Otto von Guericke University Magdeburg, Germany；Digital Engineering Faculty, University of Potsdam, Germany；Machine Learning, Hasso-Plattner-Institute, Potsdam, Germany；Department of Biomedical Data Science, Stanford University, Stanford, CA, United States；Centre for Behavioural Brain Sciences, Magdeburg, Germany；German Centre for Neurodegenerative Disease, Magdeburg, Germany
Benchmarking Class Activation Map Methods for Explainable Brain Hemorrhage Classification on Hemorica Dataset
Faculty of Computer Engineering, K. N. Toosi University of Technology（计算机工程学院，K.N.托菲大学）；Faculty of Electrical Engineering, K. N. Toosi University of Technology（电气工程学院，K.N.托菲大学）；Faculty of Mechanical Engineering, Tarbiat Modares University（机械工程学院，塔里比亚特莫达res大学）
