We present HEAD-QA, a multi-choice question answering testbed to encourage research on complex reasoning. The questions come from exams to access a specialized position in the Spanish healthcare system, and are challenging even for highly specialized humans. We then consider monolingual (Spanish) and cross-lingual (to English) experiments with information retrieval and neural techniques. We show that: (i) HEAD-QA challenges current methods, and (ii) the results lag well behind human performance, demonstrating its usefulness as a benchmark for future work.
ViHERMES: A Graph-Grounded Multihop Question Answering Benchmark and System for Vietnamese Healthcare Regulations
URA Research Group, Faculty of Computer Science and Engineering, Ho Chi Minh City University of Technology (HCMUT), VNU-HCM, Vietnam(URA研究组,计算机科学与工程学院,胡志明市技术大学(HCMUT),VNU-HCM,越南)
Think Parallax: Solving Multi-Hop Problems via Multi-View Knowledge-Graph-Based Retrieval-Augmented Generation
Yangtze Delta Region Institute (Huzhou), University of Electronic Science and Technology of China(长江三角洲地区研究院(湖州),电子科学与技术大学);School of Information and Software Engineering, University of Electronic Science and Technology of China(信息与软件工程学院,电子科学与技术大学)
Which Heads Matter for Reasoning? RL-Guided KV Cache Compression
Westlake University(西华大学);McGill University(麦吉尔大学);Mila - Quebec AI Institute(魁北克AI研究院);Zhejiang University(浙江大学);Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德智能大学)
Pruning via Causal Attribution Preserves Reasoning Performance in Large Language Models
Edison Academy Magnet School(爱迪生学院磁石学校);Massachusetts Institute of Technology(麻省理工学院);State University of New York College at Plattsburgh(纽约州立大学普拉茨堡学院);The University of Texas at Austin(德克萨斯大学奥斯汀分校);Independent Researcher(独立研究员)
Attention Heads of Large Language Models: A Survey
Institute for Advanced Algorithms Research (IAAR)(高级算法研究院(IAAR));Institute for AI Industry Research (AIR), Tsinghua University(清华大学人工智能产业研究院);Research Institute of China Telecom(中国电信研究院)
College of William and Mary(威廉与玛丽学院);University of Toronto(多伦多大学);University Health Network(大学健康网络);KU Leuven(鲁汶大学);Bucknell University(巴克内尔大学);University of Cincinnati(辛辛那提大学);University of Alabama at Birmingham(伯明翰大学)
Scaling Reasoning Hop Exposes Weaknesses: Demystifying and Improving Hop Generalization in Large Language Models
University of Science and Technology of China(中国科学技术大学);City University of Hong Kong(香港城市大学);Zhejiang University(浙江大学);City University of Hong Kong, Shenzhen Research Institute(香港城市大学深圳研究院)
Two Heads Are Better Than One: Dual-Model Verbal Reflection at Inference-Time
King’s College London(伦敦国王学院);AQA;The Alan Turing Institute(艾伦·图灵研究所);Tencent YouTu Lab(腾讯YouTu实验室);University of Cambridge(剑桥大学);Queen Mary University of London(伦敦大学玛丽女王学院)
针对 ASAS 中 LLM 评分缺乏可解释性且难以及早发现细微推理错误的问题,本文提出基于结构推理图差异的对比反思合成流程与双模型反思评分框架 DARS,并在全部指标上超越基线。
Benchmarking Next-Generation Reasoning-Focused Large Language Models in Ophthalmology: A Head-to-Head Evaluation on 5,888 Items
National University of Singapore(新加坡国立大学);Singapore Eye Research Institute(新加坡眼科研究所);Singapore National Eye Centre(新加坡国家眼科中心);The Chinese University of Hong Kong(香港中文大学);Yale University(耶鲁大学);Cleveland Clinic(克利夫兰诊所);Montreal(蒙特利尔);Tan Tock Seng Hospital(陈笃生医院);Stanford University(斯坦福大学)
Can OpenAI o1 Reason Well in Ophthalmology? A 6,990-Question Head-to-Head Evaluation Study
National University of Singapore(新加坡国立大学);Singapore Eye Research Institute(新加坡眼科研究所);Yale University(耶鲁大学);Cleveland Clinic(克利夫兰诊所);Zhejiang University(浙江大学)
Iteration Head: A Mechanistic Study of Chain-of-Thought
FAIR, Meta AI(FAIR,Meta人工智能公司);Datashape, INRIA(Datashape,法国国家信息与自动化研究所);Courant University and Center for Data Science, NYU(纽约大学柯朗数学科学研究所与数据科学中心);Center for Data Science, NYU(纽约大学数据科学中心);NYU(纽约大学);Courant University(柯朗大学)
DeepSeek in Healthcare: A Survey of Capabilities, Risks, and Clinical Applications of Open-Source Large Language Models
Weill Cornell Medicine, Cornell University(威尔康奈尔医学院,康奈尔大学);Johns Hopkins University(约翰斯·霍普金斯大学);Touro University College of Osteopathic Medicine(图罗大学整骨医学院)
Large Language Models Illuminate a Progressive Pathway to Artificial Healthcare Assistant: A Review
Peking University(北京大学);Peking University Cancer Hospital and Institute(北京大学肿瘤医院暨北京市肿瘤防治研究所);National Engineering Laboratory for Big Data Analysis and Applications, Peking University(北京大学大数据分析与应用技术国家工程实验室);Peking University Changsha Institute for Computing and Digital Economy(北京大学长沙计算与数字经济研究院)