Evaluating LLM-generated interactive software requires execution in addition to static analysis. The key difficulty is that correctness is a graph-level reachable property over latent UI state-transition graphs, whereas a GUI evaluator observes only a single execution trajectory. A failed rollout therefore rules out only one realized path, leaving failure attribution ambiguous between evaluator-side execution error and genuine software defect. We present DiagEval, a trajectory-conditioned diagnostic evaluation protocol for post-failure GUI-agent evaluation of interactive software. Rather than blindly retrying from scratch, DiagEval reuses the failed trajectory to choose targeted diagnostic probes and aggregates their outcomes into an internal attribution signal. The latent-graph view motivates the diagnostic problem; DiagEval does not reconstruct the graph or estimate calibrated posterior probabilities. We evaluate DiagEval on WebDevJudge-Unit and RealDevBench across multiple GUI-agent evaluators and LLM backbones. On false-negative cases, DiagEval recovers 45.6-62.1% of failures that were initially misattributed to software defects, outperforming retry-based baselines with 34.4-160.6% relative gains. On the full evaluation sets, this recovery improves accuracy from 69.9% to 78.3% on WebDevJudge-Unit and from 65.0% to 81.6% on RealDevBench. These results suggest that reliable GUI-agent evaluation requires not only stronger execution, but also active failure diagnosis to disambiguate evaluator-side errors from genuine software defects. Our code is available at https://github.com/scutGit/DiagEval.
G-SHARE: A Guideline-Based Structured Reasoning Framework for Human-Factor Event Diagnosis
Institute of Nuclear and New Energy Technology, Tsinghua University(清华大学核能与新能源技术研究院);National Key Laboratory of Human Factors Engineering(人因工程重点实验室);Fujian Fuqing Nuclear Power Co., Ltd.(福建福清核电有限公司)
CDS: Knowledge Component-Driven Data Synthesis Guided by Cognitive Diagnosis Theory
College of Computer Science and Artificial Intelligence, Fudan University(复旦大学计算机科学与人工智能学院);Shanghai Institute of Artificial Intelligence for Education, East China Normal University(华东师范大学教育人工智能研究所);School of Data Science, Fudan University(复旦大学数据科学学院);Huawei Noah’s Ark Lab(华为诺亚实验室)
Lung-R1: A Knowledge Graph-Guided LLM for Pulmonary Diagnostic Reasoning
School of Computer Science, Chongqing University(重庆大学计算机学院);AI Research Institution, Mashang Financial Institution(马上金融人工智能研究院);Department of Information, Third Military Medical University(陆军军医大学信息系)
Prompting Large Language Models for Supporting the Differential Diagnosis of Anemia
Inria Paris(法国国家信息与自动化研究所巴黎分部);Centre de Recherche des Cordeliers(科尔德里耶研究中心);Inserm(法国国家健康与医学研究院);Université Paris Cité(巴黎城市大学);Sorbonne Université(索邦大学)
Enhancing Explainable Cardiac Diagnosis with Guide-Grounded Multimodal LLMs
Business AI Lab, College of Technology, National Economics University(商业人工智能实验室,技术学院,越南国家经济大学);A2I Lab, Phenikaa School of Computing, Phenikaa University(A2I实验室,费尼卡计算机学院,费尼卡大学);VNPT AI, VNPT Group(VNPT人工智能,VNPT集团);FPT University(FPT大学);Department of Pediatrics, Hospital of University Medicine and Pharmacy, Vietnam National University Hanoi(越南河内国家大学医学与药学院儿科学系)
Look Before You Leap: A GUI-Critic-R1 Model for Pre-Operative Error Diagnosis in GUI Automation
MAIS, Institute of Automation, Chinese Academy of Sciences, China(自动化研究所,中国科学院,中国);School of Artificial Intelligence, University of Chinese Academy of Sciences, China(中国科学院大学人工智能学院,中国);Alibaba Group(阿里巴巴集团)
Unifying VLM-Guided Flow Matching and Spectral Anomaly Detection for Interpretable Veterinary Diagnosis
School of Mathematics, Shandong University(山东大学数学学院);Shenzhen Loop Area Institute(深圳河套学院);Yeshiva University(叶史瓦大学);Qilu University of Technology(齐鲁工业大学);Shandong Normal University(山东师范大学);Chuzhou University(滁州学院)
Enhancing Depression-Diagnosis-Oriented Chat with Psychological State Tracking
School of Computer Science and Technology, East China Normal University(东华大学计算机科学与技术学院);Shanghai Institute of Al for Education, East China Normal University(东华大学上海人工智能教育研究所);School of Psychology and Cognitive Science, East China Normal University(东华大学心理学与认知科学学院)
RetiBridge: Bridging Quantitative Retinal Biomarkers and Qualitative Diagnosis with a Knowledge-Guided Multimodal Large Language Model
University of Liverpool(利物浦大学);Institute of Life Course & Medical Sciences(生命课程与医学科学研究院);Department of Eye and Vision Sciences(眼科与视觉科学系);Computer Science Department(计算机科学系);Cardiovascular & Metabolic Medicine(心血管与代谢医学);Liverpool Centre for Cardiovascular Science(利物浦心血管科学中心)
AD-CARE: A Guideline-grounded, Modality-agnostic LLM Agent for Real-world Alzheimer's Disease Diagnosis with Multi-cohort Assessment, Fairness Analysis, and Reader Study
The Hong Kong Polytechnic University(香港理工大学);Xuanwu Hospital, Capital Medical University(首都医科大学宣武医院);Amazon(亚马逊);Zhejiang University(浙江大学);Sany AI(三一人工智能);Yantai Yuhuangding Hospital, Qingdao University(青岛大学烟台毓璜顶医院);The University of Hong Kong(香港大学);The Third Affiliated Hospital of Sun Yat-sen University(中山大学附属第三医院)
Interpretable Few-Shot Retinal Disease Diagnosis with Concept-Guided Prompting of Vision-Language Models
Monash University(莫纳什大学);Centre for Eye Research Australia(澳大利亚眼科研究中心);Royal Victorian Eye and Ear Hospital(皇家维多利亚眼耳医院);The University of Melbourne(墨尔本大学);The Hong Kong Polytechnic University(香港理工大学);Centre for Eye and Vision Research(眼与视觉研究中心);Indian Institute of Technology Bombay(印度理工学院孟买分校);Airdoc-Monash Research Lab(Airdoc-莫纳什研究实验室)
Beyond Uncertainty Reduction: Calibrated Planning for Interactive Diagnosis
Lanzhou University(兰州大学);Hunan University(湖南大学);National University of Singapore(新加坡国立大学);City University of Hong Kong(香港城市大学);The Ohio State University(俄亥俄州立大学)
Dep-LLM: Training-Free Depression Diagnosis via Evidence-Guided Structured Multi-factor with Reliable LLM Reasoning
School of Computer Science and Technology, Harbin Institute of Technology, Shenzhen, Guangdong, China(哈尔滨工业大学(深圳)计算机科学与技术学院);School of Artificial Intelligence and Computer Science, Jiangnan University, Wuxi, China(江南大学人工智能与计算机学院);Guangdong Provincial Key Laboratory of Intelligent Information Processing(广东省智能信息处理重点实验室);Pengcheng Laboratory(鹏城实验室);Chinese People’s Liberation Army General Hospital, Beijing, China(中国人民解放军总医院)
Green Shielding: A User-Centric Approach Towards Trustworthy AI
University of California, Berkeley(加州大学伯克利分校);University of Melbourne(墨尔本大学);University of California, San Francisco(加州大学旧金山分校);University of Georgia(佐治亚大学);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校)