Distorted science communication harms individuals and society as it can lead to unhealthy behavior change and decrease trust in scientific institutions. Given the rapidly increasing volume of science communication in recent years, a fine-grained understanding of how findings from scientific publications are reported to the general public, and methods to detect distortions from the original work automatically, are crucial. Prior work focused on individual aspects of distortions or worked with unpaired data. In this work, we make three foundational contributions towards addressing this problem: (1) annotating 1,600 instances of scientific findings from academic papers paired with corresponding findings as reported in news articles and tweets wrt. four characteristics: causality, certainty, generality and sensationalism; (2) establishing baselines for automatically detecting these characteristics; and (3) analyzing the prevalence of changes in these characteristics in both human-annotated and large-scale unlabeled data. Our results show that scientific findings frequently undergo subtle distortions when reported. Tweets distort findings more often than science news reports. Detecting fine-grained distortions automatically poses a challenging task. In our experiments, fine-tuned task-specific models consistently outperform few-shot LLM prompting.
MOOSE-Chem2: Exploring LLM Limits in Fine-Grained Scientific Hypothesis Discovery via Hierarchical Search
Nanyang Technological University（南洋理工大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；University of Science and Technology of China（中国科学技术大学）；Wuhan University（武汉大学）；National University of Singapore（新加坡国立大学）；University of New South Wales（新南威尔士大学）；MiroMind；The Chinese University of Hong Kong（香港中文大学）
MeasHalu: Mitigation of Scientific Measurement Hallucinations for Large Language Models with Enhanced Reasoning
Shenzhen Key Laboratory for High Performance Data Mining（深圳高性能数据挖掘重点实验室）；Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（深圳先进技术研究院，中国科学院）；Artificial Intelligence Research Institute, Shenzhen University of Advanced Technology（深圳先进技术大学人工智能研究院）
SciCUEval: A Comprehensive Dataset for Evaluating Scientific Context Understanding in Large Language Models
ZJU-Hangzhou Global Scientific and Technological Innovation Center, Zhejiang University（浙江大学杭州国际科创中心）；The Polytechnic Institute, Zhejiang University（浙江大学理工学院）；ZJU-UIUC, Zhejiang University（浙江大学伊利诺伊大学厄巴纳-香槟联合学院）；College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；AntGroup（蚂蚁集团）
SciCustom: A Framework for Custom Evaluation of Scientific Capabilities in Large Language Models
State Key Laboratory for Multimedia Information Processing, School of Computer Science, PKU-Anker LLM Lab, Peking University（多媒体信息处理国家重点实验室，计算机学院，PKU-Anker LLM实验室，北京大学）；Zhongguancun Academy（中关村学院）；IDEA；Xidian University（西安电子科技大学）；Peking University（北京大学）；University of Washington（华盛顿大学）；University of Wisconsin–Madison（威斯康星大学麦迪逊分校）；University of Illinois Chicago（伊利诺伊大学芝加哥分校）
RPC-Bench: A Fine-grained Benchmark for Research Paper Comprehension
Xinjiang University（新疆大学）；Renmin University of China（中国人民大学）；Anhui University（安徽大学）；Sun Yat-sen University（中山大学）；Tsinghua University（清华大学）；University of Southampton（南安普顿大学）
When Reviews Disagree: Fine-Grained Contradiction Analysis in Scientific Peer Reviews
Department of Computer Science and Engineering, Indian Institute of Technology Patna, India（印度理工学院帕纳瓦分校计算机科学与工程系）；School of Computer Engineering, KIIT Deemed to be University, Bhubaneswar, India（比哈尔邦布尔萨大学计算机工程学院）
SciQAG: A Framework for Auto-Generated Science Question Answering Dataset with Fine-grained Evaluation
GreenDynamics Pty. Ltd（格林动力私人有限公司）；City University of Hong Kong（香港城市大学）；University of Chicago（芝加哥大学）；University of Sydney（悉尼大学）；University of New South Wales（新南威尔士大学）
Scaling Creative Inspiration with Fine-Grained Functional Aspects of Ideas
Allen Institute for AI（艾伦人工智能研究所）；The University of Washington（华盛顿大学）；Hebrew University of Jerusalem（耶路撒冷希伯来大学）；Carnegie Mellon University（卡内基梅隆大学）；University of Copenhagen（哥本哈根大学）；University of Maryland（马里兰大学）
OpenMedReason: Scientific Reasoning Supervision for Medical Vision-Language Models
York University（约克大学）；Vector Institute（向量研究所）；University of British Columbia（不列颠哥伦比亚大学）；University of Toronto（多伦多大学）；Unity Health Toronto / St. Michael’s Hospital（多伦多联合健康/圣迈克尔医院）；University Health Network（大学健康网络）；Arc Institute（弧研究所）；Queen's University（女王大学）
