SciTS: Scientific Time Series Understanding and Generation with LLMs
Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);Tsinghua University(清华大学);Harbin Institute of Technology(哈尔滨工业大学);University of Science and Technology of China(中国科学技术大学)
The scientific reasoning ability of large language models (LLMs) has recently attracted significant attention. Time series, as a fundamental modality in scientific data, presents unique challenges that are often overlooked in current multimodal LLMs, which either encode numerical sequences as text or convert them into images. Such approaches may be insufficient for comprehensive scientific time series understanding and generation. Existing unified time series models typically specialise in either forecasting or analysis, and their effectiveness on non-periodic, heterogeneous scientific signals remains unclear. To address these gaps, we introduce SciTS, a benchmark spanning 12 scientific domains and 43 tasks, with over 50k+ instances, both univariate and multivariate signals ranging from $10^0$ to $10^7$ in length and up to 10~MHz in frequency. We benchmark 17 models, including text-only LLMs, multimodal LLMs, and unified time series models, and find that general-purpose LLMs exhibit stronger generalisability than specialised time series models, while representing time series as text or images limits their performance due to excessively long sequences and loss of numerical precision, respectively. We then introduce TimeOmni, a framework that equips LLMs with the ability to understand and generate time series while remaining compatible with general-purpose LLM training. This work fills a gap in both dedicated benchmarks and modelling frameworks for scientific time series, paving the way for LLMs to understand and generate complex temporal scientific data.
SciAgent: Tool-augmented Language Models for Scientific Reasoning
Microsoft(微软公司);University of California, Santa Barbara(加利福尼亚大学圣巴巴拉分校);Tsinghua University(清华大学);Singapore Management University(新加坡管理大学);Nanyang Technological University(南洋理工大学)
Can LLMs Discover Scientific Laws in Real and Parallel Worlds?
University of California San Diego(加州大学圣地亚哥分校);University of California, Irvine(加州大学欧文分校);Cushing Academy(卡欣学院);University of California, Santa Barbara(加州大学圣巴巴拉分校)
LLMs can Realize Combinatorial Creativity: Generating Creative Ideas via LLMs for Scientific Research
Boston University(波士顿大学);University of Illinois at Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);University of Delaware(特拉华大学);Acaciawood Preparatory Academy(阿卡西亚伍德预备学院)
Project Alexandria: Towards Freeing Scientific Knowledge from Copyright Burdens via LLMs
LAION;Open-Ψ (Open-Sci) Collective(Open-Ψ(Open-Sci)集体);L3S Research Center, Leibniz University Hannover(汉诺威莱布尼茨大学L3S研究中心);Stanford University(斯坦福大学);TIB—Leibniz Information Centre for Science and Technology(莱布尼茨科技信息中心);Juelich Supercomputing Center(于利希超级计算中心);Technical University of Munich(慕尼黑工业大学)
SciR: A Controllable Benchmark for Scientific Reasoning in LLMs
Idiap Research Institute(Idiap研究 institute);EPFL(瑞士联邦理工学院);School of Computer Science, University of Sheffield(谢菲尔德大学计算机科学学院);University of Manchester(曼彻斯特大学);National Biomarker Centre, CRUK Manchester Institute(国家生物标志物中心,CRUK曼彻斯特研究所)
Advancing the Scientific Method with Large Language Models: From Hypothesis to Discovery
Tufts University(塔夫茨大学);KAUST(阿卜杜拉国王科技大学);Network Rail(英国铁路网公司);Zuse Institute Berlin(柏林祖泽研究所);NVIDIA Research(英伟达研究院);King Abdullah University of Science and Technology(阿卜杜拉国王科技大学);Pasteur Labs(巴斯德实验室);Harvard University(哈佛大学);University of Southampton(南安普顿大学);Stanford University(斯坦福大学);University of Chicago(芝加哥大学);Santa Fe Institute(圣塔菲研究所);The University of Edinburgh(爱丁堡大学);Jožef Stefan Institute(约热夫·斯特凡研究所);Karolinska Institutet(卡罗林斯卡学院);The Alan Turing Institute(艾伦·图灵研究所);Oxford Immune Algorithmics(牛津免疫算法公司);Francis Crick Institute(弗朗西斯·克里克研究所)
EarthSE: A Benchmark for Evaluating Earth Scientific Exploration Capability of LLMs
Shanghai Jiao Tong University(上海交通大学);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);The Hong Kong Polytechnic University(香港理工大学);The Chinese University of Hong Kong(香港中文大学)
SciRerankBench: Benchmarking Rerankers Towards Scientific Retrieval-Augmented Generation with LLMs
Computer Network Information Center, Chinese Academy of Sciences(中国科学院计算机网络信息中心);University of California, Los Angeles(加州大学洛杉矶分校);Peking University(北京大学)
Evaluating LLMs' Divergent Thinking Capabilities for Scientific Idea Generation with Minimal Context
Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学人工智能学院);College of Computer Science and Technology, Zhejiang University(浙江大学计算机科学与技术学院);Bank of China(中国银行);School of Engineering Science, University of Chinese Academy of Sciences(中国科学院大学工程科学学院);State Key Laboratory of Nonlinear Mechanics, Institute of Mechanics, Chinese Academy of Sciences(中国科学院力学研究所非线性力学重点实验室)
SciSafeEval: A Comprehensive Benchmark for Safety Alignment of Large Language Models in Scientific Tasks
Duke University(杜克大学);Zhejiang University(浙江大学);Australian National University(澳大利亚国立大学);Institute of Software Chinese Academy of Sciences(中国科学院软件研究所);Beijing University of Posts and Telecommunications(北京邮电大学);University College London(伦敦大学学院);University of Glasgow(格拉斯哥大学);China Mobile Communications Group Co.,Ltd(中国移动通信集团有限公司);Hong Kong Polytechnic University(香港理工大学);North China University of Technology(北方工业大学)
How Do LLMs Encode Scientific Quality? An Empirical Study Using Monosemantic Features from Sparse Autoencoders
Knowledge Media Institute, The Open University, Walton Hall, Milton Keynes, MK7 6AA, United Kingdom(开放大学知识媒体学院);Department of Business and Law, University of Milano Bicocca, Milan, IT(米兰比可卡大学商学院)
Lost in Tokenization: Context as the Key to Unlocking Biomolecular Understanding in Scientific LLMs
Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);Westlake University(西交大学);Shanghai Innovation Institute(上海创新研究院);Shanghai Jiaotong University(上海交通大学);The Chinese University of Hong Kong(香港中文大学);Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所)
On LLM-Based Scientific Inductive Reasoning Beyond Equations
Dept. of Comp. Sci. & Tech., Institute for AI, BNRist Center, Tsinghua University(计算机科学与技术系,人工智能研究院,BNRist中心,清华大学);Jiangsu Collaborative Innovation Center for Language Ability, Jiangsu Normal University(江苏语言能力协同创新中心,江苏师范大学);Beijing Language and Culture University(北京语言文化大学);Xiamen University(厦门大学);Harbin Institute of Technology(哈尔滨工业大学)