How much do large language models actually hallucinate when answering questions grounded in provided documents? Despite the critical importance of this question for enterprise AI deployments, reliable measurement has been hampered by benchmarks that rely on static datasets vulnerable to contamination, LLM-based judges with documented biases, or evaluation scales too small for statistical confidence. We address this gap using RIKER, a ground-truth-first evaluation methodology that enables deterministic scoring without human annotation. Across 35 open-weight models, three context lengths (32K, 128K, and 200K tokens), four temperature settings, and three hardware platforms (NVIDIA H200, AMD MI300X, and Intel Gaudi 3), we conducted over 172 billion tokens of evaluation - an order of magnitude beyond prior work. Our findings reveal that: (1) even the best-performing models fabricate answers at a non-trivial rate - 1.19% at best at 32K, with top-tier models at 5 - 7% - and fabrication rises steeply with context length, nearly tripling at 128K and exceeding 10% for all models at 200K; (2) model selection dominates all other factors, with overall accuracy spanning a 72-percentage-point range and model family predicting fabrication resistance better than model size; (3) temperature effects are nuanced - T=0.0 yields the best overall accuracy in roughly 60% of cases, but higher temperatures reduce fabrication for the majority of models and dramatically reduce coherence loss (infinite generation loops), which can reach 48x higher rates at T=0.0 versus T=1.0; (4) grounding ability and fabrication resistance are distinct capabilities - models that excel at finding facts may still fabricate facts that do not exist; and (5) results are consistent across hardware platforms, confirming that deployment decisions need not be hardware-dependent.
Enhancing Large Language Models with Domain-specific Retrieval Augment Generation: A Case Study on Long-form Consumer Health Question Answering in Ophthalmology
Yale University(耶鲁大学);National Eye Institute, National Institutes of Health(美国国立卫生研究院国家眼科研究所);Singapore Eye Research Institute(新加坡眼科研究所);Singapore National Eye Centre(新加坡国家眼科中心);National Center for Biotechnology Information, National Library of Medicine, National Institutes of Health(美国国立卫生研究院国家医学图书馆国家生物技术信息中心);Rice University(莱斯大学);National University of Singapore(新加坡国立大学)
Fine PT-PT Web: A High-Quality 41 Billion Tokens Data Collection of the European Portuguese Web
NOVA School of Science and Technology(NOVA科学与技术学院);NOVA LINCS;Fundação para a Ciência e Tecnologia(科学技术基金会);Instituto Superior Técnico, Universidade de Lisboa(里斯本大学高等技术学院);Instituto de Telecomunicações(电信研究院)
Tackling the Inherent Difficulty of Noise Filtering in RAG
Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学人工智能学院);Beijing Key Laboratory of Big Data Management and Analysis Methods(北京大数据管理与分析方法重点实验室);School of Software Tsinghua University(清华大学软件学院)
OmniCorpus: A Unified Multimodal Corpus of 10 Billion-Level Images Interleaved with Text
Harbin Institute of Technology(哈尔滨工业大学);Nanjing University(南京大学);Fudan University(复旦大学);The Chinese University of Hong Kong(香港中文大学);Shanghai AI Laboratory(上海人工智能实验室);SenseTime Research(商汤科技研究院);Tsinghua University(清华大学)
ClueWeb22: 10 Billion Web Documents with Visual and Semantic Information
Microsoft(微软);Carnegie Mellon University(卡内基梅隆大学)
作为最新版本的数据集,CNET Web 22提供了100亿个包含丰富信息的网页。它的设计受到了对高质量,大规模网络语料库的需求的影响,以支持一系列学术和工业研究,例如,在信息系统,检索增强AI系统和模型预训练中。与早期的语料库相比,CCRYSTWeb 22语料库更大,更多样,质量更高,并且符合商业网络搜索的文档分布。除了原始HTML之外,Cocktail Web 22还包含有关行业标准文档理解系统提供的网页的丰富信息,包括Web浏览器呈现的页面的视觉表示,来自神经网络解析器的解析HTML结构信息,以及预处理的清洁文档文本,以降低进入门槛。这些信号中有许多已在工业中广泛使用,但在这种规模下首次提供给研究界。
SaulLM-7B: A pioneering Large Language Model for Law
CentraleSupélec(中央高等电力学院);Université Paris-Saclay(巴黎萨克雷大学);Sorbonne Université(索邦大学);Instituto Superior Técnico(里斯本高等理工学院);Universidade de Lisboa(里斯本大学);NOVA School of Law(NOVA法学院)
Less Data, More Security: Advancing Cybersecurity LLMs Specialization via Resource-Efficient Domain-Adaptive Continuous Pre-training with Minimal Tokens
SSH Communications Security(SSH通讯安全公司);KTH Royal Institute of Technology(皇家理工学院);Aalto University(阿尔托大学)
Small Encoders Can Rival Large Decoders in Detecting Groundedness
Chandar Research Lab(昌达尔研究实验室);Mila – Quebec AI Institute(魁北克人工智能研究院);Université de Montréal(蒙特利尔大学);Polytechnique Montréal(蒙特利尔理工学院);Aily Labs(Aily实验室);Canada CIFAR AI Chair(加拿大CIFAR人工智能主席)