Large language models generate outputs stochastically and may produce fluent but invalid responses, including factual hallucinations. Existing mitigation strategies reduce average error rates but do not provide explicit control over the \emph{frequency} of such failures under repeated use. We formulate inference as a deployment-time risk control problem and introduce \emph{chance-constrained inference}, which directly bounds the probability of hallucinations among accepted generations. Hallucinations are modeled as stochastic constraint violations, and we show that confidence-based selective prediction does not, in general, imply probabilistic risk guarantees. To enforce chance constraints efficiently, we propose a sequential, anytime-valid inference procedure that adaptively certifies feasibility or infeasibility using finite samples, avoiding conservative fixed-sample bounds. Experiments on questions inspired by NaturalQuestions and controlled multi-hop question answering demonstrate reliable risk control, early detection of intrinsically infeasible inputs, and safe composition under repeated use, while confidence-based baselines fail to provide consistent guarantees.
ReliableNet: A Chance-Constrained Approach to Trustworthy Classification in Deep Learning
School of Agriculture and Science, University of KwaZulu-Natal(夸祖鲁-纳塔尔大学农业与科学学院);AIMS Research and Innovation Centre, African Institute for Mathematical Sciences(非洲数学科学研究所AIMS研究与创新中心);African Institute for Mathematical Sciences(非洲数学科学研究所)
HalluGuard: Demystifying Data-Driven and Reasoning-Driven Hallucinations in LLMs
CS Department Virginia Tech(弗吉尼亚理工学院计算机科学系);EECS Department MIT(麻省理工学院电子工程与计算机科学系);CS Department Dartmouth College(达特茅斯学院计算机科学系);Statistics Department Virginia Tech(弗吉尼亚理工学院统计学系);CS Department Michigan State University(密歇根州立大学计算机科学系)
Diagnosing Hallucination Risk in AI Surgical Decision-Support: A Sequential Framework for Sequential Validation
Orthopaedic Centre, The University of Hong Kong - Shenzhen Hospital(香港大学深圳医院骨科中心);Translational Medicine Centre, The University of Hong Kong - Shenzhen Hospital(香港大学深圳医院转化医学中心);Department of Orthopaedics & Traumatology, Li Ka Shing Faculty of Medicine, The University of Hong Kong(香港大学李嘉诚医学院骨科与创伤外科部)
Evaluation and Analysis of Hallucination in Large Vision-Language Models
Beijing Jiaotong University(北京交通大学);Xi’an Jiaotong University(西安交通大学);DAMO Academy, Alibaba Group(阿里巴巴集团达摩院);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);Shandong University(山东大学)