Large Language Models (LLMs) have recently shown promise in molecular discovery, yet a gap remains between their probabilistic nature over discrete sequential tokens and the rigid topological constraints of chemical space. This raises the question of whether molecular LLMs can generalize beyond the local neighborhoods induced by their sequence-based representations. To systematically investigate this question, we introduce a Molecular Perturbation framework that generates syntax-valid structural variants of training molecules under controlled Graph Edit Distance (GED) to probe the manifold regularity of molecular LLMs. Our analysis shows that even a single edit can cause substantial performance drops on common molecular tasks, revealing a narrow local trust region and fragile sensitivity to structural changes. Since similar molecules tend to exhibit similar properties, In-Context Tuning (ICT), which anchors predictions on structurally similar molecules, offers a natural way to mitigate such fragility. Our experiments also examine whether ICT confers robustness under controlled structural perturbations, and the results suggest that it can partially expand the local trust region and offer a promising direction for stabilizing molecular LLMs against structural variation.
Relevant or Random: Can LLMs Truly Perform Analogical Reasoning?
The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));Princeton University(普林斯顿大学);Nanyang Technological University(南洋理工大学);Salesforce Research(Salesforce 研究);Institute of High Performance Computing (IHPC), Agency for Science, Technology and Research (A*STAR), Singapore(高性能计算研究所,新加坡科技研究局)
Do Large Language Models Truly Grasp Mathematics? An Empirical Exploration From Cognitive Psychology
College of Computer Science and Technology, National University of Defense Technology(国防科技大学计算机科学与技术学院);Institute of High Performance Computing, Agency for Science, Technology and Research (A*STAR)(科学、技术和研究局高性能计算研究所);Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所)
The LLM Effect: Are Humans Truly Using LLMs, or Are They Being Influenced By Them Instead?
George Mason University(乔治梅森大学);The Schar School of Policy and Government(沙尔政策与政府学院);Department of Computer Science, George Mason University(乔治梅森大学计算机科学系);Archimedes AI Unit, Athena Research Center, Greece(希腊雅典娜研究中心阿基米德人工智能单元)
GAOKAO-Eval: Does high scores truly reflect strong capabilities in LLMs?
Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);East China Normal University(华东师范大学);Harbin Institute of Technology(哈尔滨工业大学);The Chinese University of Hong Kong(香港中文大学)
LMU Munich(慕尼黑大学);Munich Center for Machine Learning (MCML)(慕尼黑机器学习中心);Technical University of Munich(慕尼黑技术大学);Siemens AG(西门子股份公司);University of Oxford(牛津大学)
Don't Tell the Answer, Truly Guide the Reasoning During RL Rollouts
School of Data Science, Fudan University(复旦大学数据科学学院);Shanghai Institute of Artificial Intelligence for Education, East China Normal University(华东师范大学教育人工智能研究所);College of Computer Science and Artificial Intelligence, Fudan University(复旦大学计算机科学与人工智能学院);Ant Group(蚂蚁集团)
Are LLM-generated plain language summaries truly understandable? A large-scale crowdsourced evaluation
School of Information Sciences, University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校信息科学学院);Department of Radiology and Biomedical Imaging, University of California, San Francisco(加州大学旧金山分校放射科与生物医学成像系);Eller College of Management, University of Arizona(亚利桑那大学埃勒管理学院);Biomedical Informatics and Medical Education, University of Washington(华盛顿大学生物医学信息学与医学教育)
ValueDCG: Measuring Comprehensive Human Value Understanding Ability of Language Models
Institute for Artificial Intelligence, Peking University(北京大学人工智能研究院);State Key Laboratory of General Artificial Intelligence, BIGAI(北京通用人工智能研究院通用人工智能国家重点实验室);School of Artificial Intelligence, Beijing University of Posts and Telecommunications(北京邮电大学人工智能学院)
Institute of Computational Life Sciences, Zurich University of Applied Sciences(计算生命科学学院,应用科学大学);Institute of History and Philosophy of Science and Technology (IHPST), Université Paris 1 - Panthéon-Sorbonne(科学与技术历史与哲学研究所,巴黎第一大学)