We introduce GISTBench, a benchmark for evaluating Large Language Models' (LLMs) ability to understand users from their interaction histories in recommendation systems. Unlike traditional RecSys benchmarks that focus on item prediction accuracy, our benchmark evaluates how well LLMs can extract and verify user interests from engagement data. We propose two novel metric families: Interest Groundedness (IG), decomposed into precision and recall components to separately penalize hallucinated interest categories and reward coverage, and Interest Specificity (IS), which assesses the distinctiveness of verified LLM-predicted user profiles. We release a synthetic dataset constructed on real user interactions on a global short-form video platform. Our dataset contains both implicit and explicit engagement signals and rich textual descriptions. We validate our dataset fidelity against user surveys, and evaluate eight open-weight LLMs spanning 7B to 235B parameters, together with three proprietary frontier models (GPT-5, Claude 4.6, and Gemini 3.5 Flash). Our findings reveal performance bottlenecks in current LLMs, particularly their limited ability to accurately count and attribute engagement signals across heterogeneous interaction types.
GPE: Evaluating Robust Evidence Aggregation for Fact Verification under Controllable GEO-Style Poisoning
School of Cyberspace Science and Technology, Beijing Institute of Technology(航天信息科学技术学院,北京理工大学);School of Computer Science, The University of Auckland(计算机科学学院,奥克兰大学)
Evaluating LLM Personalization via Semantic Constraint Verification
University of New South Wales(新南威尔士大学);Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学);The Technology Innovation Institute(技术创新研究所);The Hong Kong University of Science and Technology(香港科技大学)
AgentPerturb: Evaluating LLM Agent Robustness under Controlled Interaction Perturbations
The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));Tencent(腾讯);East China Normal University(华东师范大学);University College London(伦敦大学学院)
FactEHR: A Dataset for Evaluating Factuality in Clinical Notes Using LLMs
Khoury College of Computer Sciences, Northeastern University(东北大学克劳尔计算机科学学院);Center for Biomedical Informatics Research, Stanford University(斯坦福大学生物医学信息学研究中心);Department of Biomedical Data Science, Stanford School of Medicine(斯坦福医学院生物医学数据科学系);Stanford Health Care(斯坦福健康系统);Department of Medicine, Stanford School of Medicine(斯坦福医学院医学系);Clinical Excellence Research Center, Stanford School of Medicine(斯坦福医学院临床卓越研究中心);Department of Anesthesiology, Perioperative & Pain Medicine, Stanford School of Medicine(斯坦福医学院麻醉学、手术及疼痛医学系);Department of Dermatology, Stanford School of Medicine(斯坦福医学院皮肤病学系);Technology and Digital Solutions, Stanford Health Care(斯坦福健康系统技术与数字解决方案)
VCIFBench: Evaluating Complex Instruction Following for Video Understanding
School of Artificial Intelligence, Jilin University(吉林大学人工智能学院);Engineering Research Center of Knowledge-Driven Human-Machine Intelligence, Jilin University(知识驱动人机智能工程研究中心,吉林大学);International Center of Future Science, Jilin University(未来科学国际中心,吉林大学)
FVEL: Interactive Formal Verification Environment with Large Language Models via Theorem Proving
Shenzhen Campus of Sun Yat-sen University(中山大学深圳校区);City Univeristy of Hong Kong(香港城市大学);Sun Yat-sen University(中山大学);The University of Hong Kong(香港大学);Huawei Noah’s Ark Lab(华为诺亚方舟实验室);MBZUAI(Mohamed bin Zayed 人工智能大学);DarkMatter AI Research(DarkMatter AI 研究院)
Generating Zero-shot Abstractive Explanations for Rumour Verification
University of Warwick(华威大学);Mohammed Bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学);The Alan Turing Institute(阿兰·图灵研究所);Queen Mary University of London(伦敦大学玛丽皇后学院)
FaStfact: Faster, Stronger Long-Form Factuality Evaluations in LLMs
UCLA(美国大学洛杉矶分校);CUHK(香港中文大学);HKUST (GZ)(香港科技大学(广州));HKUST(香港科技大学);Tsinghua University(清华大学);ECNU(华东师范大学);NVIDIA(英伟达);UCL(伦敦大学学院);HKU(香港大学);University of Cambridge(剑桥大学)
Verification and Refinement of Natural Language Explanations through LLM-Symbolic Theorem Proving
University of Manchester(曼彻斯特大学);Idiap Research Institute(Idiap研究所);National Biomarker Centre, CRUK-MI, University of Manchester(英国曼彻斯特大学英国癌症研究中心曼彻斯特研究所国家生物标志物中心)
VEX-Bench: Benchmarking Verification Complexity of LLM-Generated Misinformation
The University of Melbourne(墨尔本大学);Deakin University(迪肯大学);Singapore Management University(新加坡管理大学);City University of Hong Kong(香港城市大学);Queensland University of Technology(昆士兰理工大学);Fudan University(复旦大学)