Large language models (LLMs) are increasingly used as decision-support tools in data-constrained scientific workflows, where correctness and validity are critical. However, evaluation practices often emphasize stability or reproducibility across repeated runs. While these properties are desirable, stability alone does not guar- antee agreement with statistical ground truth when such references are available. We introduce a controlled behavioral evaluation framework that explicitly sep- arates four dimensions of LLM decision-making: stability, correctness, prompt sensitivity, and output validity under fixed statistical inputs. We evaluate multi- ple LLMs using a statistical gene prioritization task derived from differential ex- pression analysis across prompt regimes involving strict and relaxed significance thresholds, borderline ranking scenarios, and minor wording variations. Our ex- periments show that LLMs can exhibit near-perfect run-to-run stability while sys- tematically diverging from statistical ground truth, over-selecting under relaxed thresholds, responding sharply to minor prompt wording changes, or producing syntactically plausible gene identifiers absent from the input table. Although sta- bility reflects robustness across repeated runs, it does not guarantee agreement with statistical ground truth in structured scientific decision tasks. These findings highlight the importance of explicit ground-truth validation and output validity checks when deploying LLMs in automated or semi-automated scientific work- flows.
Induced Numerical Instability: Hidden Costs in Multimodal Large Language Models
School of Computing（计算学院）；Information Systems, Singapore Management University, Singapore（信息系统，新加坡管理大学，新加坡）；Information Systems Department, Rutgers Business School, New Jersey, USA（信息系统系，罗格斯商学院，新泽西，美国）
CausalGame: Benchmarking Causal Thinking of LLM Agents in Games
MBZUAI（穆罕默德·本·扎耶德人工智能大学）；Carnegie Mellon University（卡内基梅隆大学）；Hong Kong Baptist University（香港浸会大学）；University of Oxford（牛津大学）；New York University, Abu Dhabi（纽约大学阿布扎比分校）
Benchmark Illusion: Disagreement among LLMs and Its Scientific Consequences
Department of Political Science, Purdue University（政治学系，普渡大学）；Kellogg School of Management, Center for Science of Science and Innovation, Northwestern Innovation Institute, Ryan Institute on Complexity, and McCormick School of Engineering（凯洛格管理学院、科学与创新中心、西北创新研究所、复杂性研究院以及麦科马克工程学院）
Systematic Failures in Collective Reasoning under Distributed Information in Multi-Agent LLMs
School of Computer Science, Carnegie Mellon University, Pittsburgh, USA（计算机科学学院，卡内基梅隆大学，匹兹堡，美国）；School of Environment and Society, Institute of Science Tokyo, Tokyo, Japan（环境与社会学院，东京科学研究所，东京，日本）
From Weak Task Specifications to Scientific Extraction Agents: Optimizing Task Construction
University of Science and Technology of China（中国科学技术大学）；State Key Laboratory of Cognitive Intelligence（认知智能全国重点实验室）；CCCC Second Highway Consultants Co., Ltd.（中交第二公路勘察设计研究院有限公司）
Encoded but Not Routed: Explaining the Table-Chart Gap in Scientific Claim Verification
The University of Tokyo（东京大学）；NII LLMC（日本信息处理学会LLMC）；National Institute of Informatics（日本信息处理研究所）；University of Göttingen（哥廷根大学）；Inria, LS2N, Nantes Université（法国国家信息与自动化技术研究院（Inria）、LS2N实验室、南特大学）
Demystifying Scientific Problem-Solving in LLMs by Probing Knowledge and Reasoning
Department of Computer Science, Yale University；Department of Molecular \& Cellular Biology, Harvard University；Allen Institute for AI；Department of Computer Science, Northwestern University
SciIF: Benchmarking Scientific Instruction Following Towards Rigorous Scientific Intelligence
Shanghai AI Laboratory（上海人工智能实验室）；University of Science and Technology of China（中国科学技术大学）；Shanghai Jiao Tong University（上海交通大学）；The Chinese University of Hong Kong（香港中文大学）；University of Sydney（悉尼大学）；Fudan University（复旦大学）；Tsinghua University（清华大学）；Beihang University（北航大学）
A quantum learning approach based on Hidden Markov Models for failure scenarios generation
EDF Lab Saclay（法国电力公司萨克莱实验室）；LIPN - CNRS UMR 7030（法国国家科学研究中心LIPN实验室（UMR 7030））；Université Sorbonne Paris Nord（索邦巴黎北大学）；La Maison des Sciences Numériques（数字科学之家）
Scientifically-Interpretable Reasoning Network (ScIReN): Discovering Hidden Relationships in the Carbon Cycle and Beyond
Department of Informatics and Intelligent Systems, Institute of Energy and the Environment, The Pennsylvania State University（信息学与智能系统系，能源与环境研究所，宾夕法尼亚州立大学）
FLAT: Revealing Hidden Latent-Conditioned Backdoor Failures in Federated Learning
College of Engineering and Computer Science, VinUniversity（VinUniversity工程与计算机科学学院）；VinUni-Illinois Smart Health Center, VinUniversity（VinUniversity-VinUni-Illinois智能健康中心）；Siebel School of Computing and Data Science, University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校Siebel计算与数据科学学院）；Faculty of Computer Science and Information Technology, Universiti Malaya（马来亚大学计算机科学与信息技术学院）
Uncertainties in Physics-informed Inverse Problems: The Hidden Risk in Scientific AI
Graduate School of Social and Data Sciences（社会科学与数据科学研究生院）；Hitotsubashi University（立命馆大学）；Department of Electrical and Electronic Engineering（电子工程系）；Nihon University（日立大学）
When Molecular Similarity Works: Property Cliffs Reveal Hidden Errors
School of Economics and Management（经济管理学院）；School of Computer Science（计算机科学学院）；Department of Data Science and Artificial Intelligence（数据科学与人工智能系）；College of Computer Science and Technology（计算机科学与技术学院）；School of Life Sciences and Technology（生命科学与技术学院）
