Segmentation Vision-Language Models (VLMs) have significantly advanced grounded visual understanding, yet they remain prone to pixel-grounding hallucinations, producing masks for incorrect objects or for objects that are entirely absent. Existing evaluations rely almost entirely on text- or label-based perturbations, which check only whether the predicted mask matches the queried label. Such evaluations overlook the spatial footprint and severity of hallucination and therefore fail to reveal vision-driven hallucinations, which are more challenging and more prevalent. To address this gap, we formalize the task of Counterfactual Segmentation Reasoning (CSR), where a model must segment the referenced object in the factual image and abstain in its counterfactual counterpart. To support this task, we curate HalluSegBench, the first large-scale benchmark to diagnose referring and reasoning expression segmentation hallucinations using controlled visual counterfactuals, alongside new evaluation metrics that measure hallucination severity and disentangle vision- and language-driven failure modes. We further introduce RobustSeg, a segmentation VLM trained with counterfactual fine-tuning (CFT) to learn when to segment and when to abstain. Experimental results confirm RobustSeg reduces hallucinations by 30%, while improving segmentation performance on FP-RefCOCO(+/g).
ClinHallu: A Benchmark for Diagnosing Stage-Wise Hallucinations in Medical MLLM Reasoning
The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));DAMO Academy, Alibaba Group(阿里巴巴达摩院);Hupan Lab(湖畔实验室);Zhejiang University(浙江大学)
Detecting Clinical Hallucinations in LVLMs via Counterfactual Visual Grounding Uncertainty
School of Intelligent Science and Technology, Nanjing University(南京大学智能科学与技术学院);National Institute of Healthcare Data Science at Nanjing University(南京大学医疗数据科学国家研究院);School of Biomedical Engineering, Nanjing University(南京大学生物医学工程学院)
HalluClear: Diagnosing, Evaluating and Mitigating Hallucinations in GUI Agents
MAIS&NLPR, Institute of Automation, Chinese Academy of Sciences(自动化研究所人工智能与自然语言处理实验室,中国科学院);School of Artificial Intelligence, UCAS(中国科学院大学人工智能学院);Meituan(美团)
TwinICL: Diagnosing Multimodal In-Context Learning through Paired Counterfactuals
University of California, Los Angeles(加州大学洛杉矶分校);National Taiwan University(国立台湾大学);Texas A&M University(德克萨斯A&M大学);NVIDIA AI Technology Center(英伟达AI技术中心);Arena Intelligence Inc(Arena Intelligence公司)
AHA: Aligning Large Audio-Language Models for Reasoning Hallucinations via Counterfactual Hard Negatives
Arizona State University(亚利桑那州立大学);Clemson University(克莱姆斯大学);Washington University in St.Louis(圣路易斯华盛顿大学);Rice University(Rice 大学);Morgan Stanley(摩根大通)
Diagnosing Hallucination Risk in AI Surgical Decision-Support: A Sequential Framework for Sequential Validation
Orthopaedic Centre, The University of Hong Kong - Shenzhen Hospital(香港大学深圳医院骨科中心);Translational Medicine Centre, The University of Hong Kong - Shenzhen Hospital(香港大学深圳医院转化医学中心);Department of Orthopaedics & Traumatology, Li Ka Shing Faculty of Medicine, The University of Hong Kong(香港大学李嘉诚医学院骨科与创伤外科部)
CDH-Bench: A Commonsense-Driven Hallucination Benchmark for Evaluating Visual Fidelity in Vision-Language Models
Guangdong Provincial Key Laboratory of Novel Security Intelligence Technologies, Institute of Cyberspace Security, School of Computer Science and Technology, Harbin Institute of Technology, Shenzhen(广东省新型安全智能技术重点实验室,网络空间安全研究院,计算机科学与技术学院,哈尔滨工业大学(深圳))
CheXPO: Preference Optimization for Chest X-ray VLMs with Counterfactual Rationale
The Key Laboratory of Smart Human-Computer Interaction(智能人机交互关键实验室);Wearable Technology of Shaanxi Province, Xidian University, Xi'an, China(陕西 wearable 技术, 西安电子科技大学, 中国);Nanjing University of Science and Technology(南京理工大学)