MedHopQA: A Disease-Centered Multi-Hop Reasoning Benchmark and Evaluation Framework for LLM-Based Biomedical Question Answering
MedHopQA: 一种以疾病为中心的多跳推理基准和评估框架,用于基于大语言模型的生物医学问答
Rezarta Islamaj, Robert Leaman, Joey Chan, Nicholas Wan, Qiao Jin, Natalie Xie, John Wilbur, Shubo Tian, Lana Yeganova, Po-Ting Lai, Chih-Hsuan Wei, Yifan Yang, Yao Ge, Qingqing Zhu, Zhizheng Wang, Zhiyong Lu
机构
*
National Library of Medicine, Division of Intramural Research(国家医学图书馆,院内研究部)
;
University of Illinois at Urbana-Champaign, Department of Computer Science(伊利诺伊大学厄巴纳-香槟分校计算机科学系)
;
University of Michigan Medical School(密歇根大学医学院)
PhyGround: Benchmarking Physical Reasoning in Generative World Models
PhyGround:用于生成世界模型中物理推理的基准测试
Juyi Lin, Arash Akbari, Yumei He, Lin Zhao, Haichao Zhang, Arman Akbari, Xingchen Xu, Zoe Y. Lu, Enfu Nan, Hokin Deng, Edmund Yeh, Sarah Ostadabbas, Yun Fu, Jennifer Dy, Pu Zhao, Yanzhi Wang
机构
*
Northeastern University(东北大学)
;
Tulane University(路易斯安那州立大学)
;
University of Washington(华盛顿大学)
;
Carnegie Mellon University(卡内基梅隆大学)
LINGOLY-TOO: Disentangling Reasoning from Knowledge with Templatised Orthographic Obfuscation
LINGOLY-TOO:通过模板化正交混淆分离推理与知识
Jude Khouja, Lingyi Yang, Karolina Korgul, Simeon Hellsten, Vlad A. Neacsu, Harry Mayne, Ryan Othniel Kearns, Andrew M. Bean, Adam Mahdi
机构
*
University of Oxford(牛津大学)
;
University of Nottingham(诺丁汉大学)
;
University of Glasgow(格拉斯哥大学)
;
United Kingdom Linguistics Olympiad(英国语言学奥林匹克)
;
Asia-Pacific Linguistics Olympiad(亚太语言学奥林匹克)
Probing the Critical Point (CritPt) of AI Reasoning: a Frontier Physics Research Benchmark
探测人工智能推理的临界点(CritPt):一个前沿物理研究基准
Minhui Zhu, Minyang Tian, Xiaocheng Yang, Tianci Zhou, Lifan Yuan, Penghao Zhu, Eli Chertkov, Shengyan Liu, Yufeng Du, Ziming Ji, Indranil Das, Qingzhi Chen, Junyi Cao, Yufeng Du, Jiabin Yu, Peixue Wu, Jinchen He, Yifan Su, Yikun Jiang, Yujie Zhang, Chang Liu, Ze-Min Huang, Weizhen Jia, Yunkai Wang, Farshid Jafarpour, Yong Zhao, Xinan Chen, Jessie Shelton, Aaron W. Young, John Bartolotta, Wenchao Xu, Yue Sun, Anjun Chu, Victor Colussi, Chris Akers, Nathan Brooks, Wenbo Fu, Jinchao Zhao, Marvin Qi, Anqi Mu, Yubo Yang, Allen Zang, Yang Lyu, Peizhi Mai, Christopher Wilson, Xuefei Guo, Juntai Zhou, Daniel Inafuku, Chi Xue, Luyu Gao, Ze Yang, Yaïr Hein, Yonatan Kahn, Kevin Zhou, Di Luo, John Drew Wilson, Jarrod T. Reilly, Dmytro Bandak, Ofir Press, Liang Yang, Xueying Wang, Hao Tong, Nicolas Chia, Eliu Huerta, Hao Peng
机构
*
Argonne National Laboratory(阿贡国家实验室)
;
University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校)
;
Virginia Tech(弗吉尼亚理工大学)
;
Ohio State University(俄亥俄州立大学)
;
Independent(独立)
;
Northeastern University(东北大学)
;
Caltech(加州理工学院)
;
University of Florida(佛罗里达大学)
;
University of Waterloo(滑铁卢大学)
;
University of Maryland, College Park(马里兰大学学院公园分校)
;
Columbia University(哥伦比亚大学)
;
Perimeter Institute for Theoretical Physics(理论物理研究所)
;
University of Connecticut(康涅狄格大学)
;
University of Cologne(科隆大学)
;
The Chinese University of Hong Kong(香港中文大学)
;
Utrecht University(乌得勒支大学)
;
Harvard University(哈佛大学)
;
ETH Zürich(苏黎世联邦理工学院)
;
Paul Scherrer Institute(保罗·谢尔研究所)
Structured Role-Aware Policy Optimization for Multimodal Reasoning
结构化角色感知策略优化用于多模态推理
Bingqing Jiang, Difan Zou
机构
*
School of Computing & Data Science, The University of Hong Kong(计算与数据科学学院,香港大学)
;
School of Computing & Data Science and Institute of Data Science, The University of Hong Kong(计算与数据科学学院和数据科学研究所,香港大学)
机构
*
Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学)
;
University of Illinois at Urbana–Champaign(伊利诺伊大学厄巴纳-香槟分校)
;
University of Maryland, College Park(马里兰大学学院公园分校)
;
University of California, San Diego(加州大学圣地亚哥分校)
机构
*
Department of Statistics and Data Science, Southern University of Science and Technology(统计与数据科学系,南方科技大学)
;
Department of Computer Sciences, University of Wisconsin–Madison(计算机科学系,威斯康星大学麦迪逊分校)
GR-Ben: A General Reasoning Benchmark for Evaluating Process Reward Models
GR-Ben:一种通用推理基准,用于评估过程奖励模型
Zhouhao Sun, Xuan Zhang, Xiao Ding, Bibo Cai, Li Du, Kai Xiong, Xinran Dai, Fei Zhang, weidi tang, Zhiyuan Kan, Yang Zhao, Bing Qin, Ting Liu
机构
*
Research Center for Social Computing(社会计算研究中心)
;
Interactive Robotics, Harbin Institute of Technology, China(交互机器人学,哈尔滨工业大学,中国)
;
Beijing Academy of Artificial Intelligence, Beijing, China(北京人工智能研究院,北京,中国)
Comments22 pages, 8 figures, 14 tables. Accepted at IJCNLP-AACL 2025
Journal refProceedings of the 14th International Joint Conference on Natural Language Processing and the 4th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics 2025, 2046-2067
CommentsAccepted to The 15th edition of the Workshop on Cognitive Modeling and Computational Linguistics, co-located with the Language Resources and Evaluation Conference
Agentic clinical reasoning over longitudinal myeloma records: a retrospective evaluation against expert consensus
基于纵向骨髓瘤记录的代理临床推理:一项回顾性评估与专家共识的对比
Johannes Moll, Jannik Lübberstedt, Christoph Nuernbergk, Jacob Stroh, Luisa Mertens, Anna Purcarea, Christopher Zirn, Zeineb Benchaaben, Fabian Drexel, Hartmut Häntze, Anirudh Narayanan, Friedrich Puttkammer, Andrei Zhukov, Jacqueline Lammert, Sebastian Ziegelmayer, Markus Graf, Marion Högner, Marcus Makowski, Florian Bassermann, Lisa C. Adams, Jiazhen Pan, Daniel Rueckert, Krischan Braitsch, Keno K. Bressem
机构
*
Chair for AI in Healthcare and Medicine, Technical University of Munich (TUM) and TUM University Hospital(人工智能在医疗与健康中的研究所,慕尼黑技术大学(TUM)及慕尼黑技术大学医院)
;
Department of Diagnostic and Interventional Radiology, Klinikum rechts der Isar, TUM University Hospital, School of Medicine and Health, Technical University of Munich(诊断与介入放射科,莱茵河右岸医院,慕尼黑技术大学医院,医学与健康学院,慕尼黑技术大学)
;
Department of Cardiovascular Radiology and Nuclear Medicine, German Heart Center, TUM University Hospital, School of Medicine and Health, Technical University of Munich(心血管放射学与核医学科,德国心脏中心,慕尼黑技术大学医院,医学与健康学院,慕尼黑技术大学)
;
Department of Medicine III, Klinikum rechts der Isar, TUM University Hospital, School of Medicine and Health, Technical University of Munich(第三医学部,莱茵河右岸医院,慕尼黑技术大学医院,医学与健康学院,慕尼黑技术大学)