Can Large Language Models Unlock Novel Scientific Research Ideas?
Department of Computer Science and Engineering, Indian Institute of Technology Patna（计算机科学与工程系，印度理工学院帕纳布分校）；National Center for Computational Sciences, Oak Ridge National Laboratory（计算科学国家中心，橡树岭国家实验室）
The widespread adoption of Large Language Models (LLMs) and publicly available ChatGPT have marked a significant turning point in the integration of Artificial Intelligence (AI) into people's everyday lives. This study examines the ability of Large Language Models (LLMs) to generate future research ideas from scientific papers. Unlike tasks such as summarization or translation, idea generation lacks a clearly defined reference set or structure, making manual evaluation the default standard. However, human evaluation in this setting is extremely challenging ie: it requires substantial domain expertise, contextual understanding of the paper, and awareness of the current research landscape. This makes it time-consuming, costly, and fundamentally non-scalable, particularly as new LLMs are being released at a rapid pace. Currently, there is no automated evaluation metric specifically designed for this task. To address this gap, we propose two automated evaluation metrics: Idea Alignment Score (IAScore) and Idea Distinctness Index. We further conducted human evaluation to assess the novelty, relevance, and feasibility of the generated future research ideas. This investigation offers insights into the evolving role of LLMs in idea generation, highlighting both its capability and limitations. Our work contributes to the ongoing efforts in evaluating and utilizing language models for generating future research ideas. We make our datasets and codes publicly available
Lost in Tokenization: Context as the Key to Unlocking Biomolecular Understanding in Scientific LLMs
Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；Westlake University（西交大学）；Shanghai Innovation Institute（上海创新研究院）；Shanghai Jiaotong University（上海交通大学）；The Chinese University of Hong Kong（香港中文大学）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）
Evaluating LLMs' Divergent Thinking Capabilities for Scientific Idea Generation with Minimal Context
Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学人工智能学院）；College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；Bank of China（中国银行）；School of Engineering Science, University of Chinese Academy of Sciences（中国科学院大学工程科学学院）；State Key Laboratory of Nonlinear Mechanics, Institute of Mechanics, Chinese Academy of Sciences（中国科学院力学研究所非线性力学重点实验室）
Many Heads Are Better Than One: Improved Scientific Idea Generation by A LLM-Based Multi-Agent System
Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；Shanghai Innovation Institute（上海创新研究院）；Department of Engineering Science, University of Oxford（牛津大学工程科学系）；Shanghai Institute for Science of Science（上海科学研究院）；Department of Information Engineering, Chinese University of Hong Kong（香港中文大学信息工程系）；Department of Electronic Engineering, Tsinghua University（清华大学电子工程系）
FlowPIE: Test-Time Scientific Idea Evolution with Flow-Guided Literature Exploration
University of Chinese Academy of Sciences（中国科学院大学）；Dalian University of Technology（大连理工大学）；UNSW Sydney（新南威尔士大学悉尼分校）；Shenzhen University of Advanced Technology（深圳理工大学）；Xiamen University（厦门大学）
Graph Your Way to Inspiration: Integrating Co-Author Graphs with Retrieval-Augmented Generation for Large Language Model Based Scientific Idea Generation
School of Computing Newcastle University（计算学院新castle大学）
SciAgents: Automating scientific discovery through multi-agent intelligent graph reasoning
Massachusetts Institute of Technology（麻省理工学院）；Schwarzman College of Computing（计算学院）；Center for Computational Science and Engineering（计算科学与工程中心）；Laboratory for Atomistic and Molecular Mechanics (LAMM)（原子与分子力学实验室）
Harnessing Large Language Models for Scientific Novelty Detection
Nanyang Technological University（南洋理工大学）；Singapore University of Technology and Design（新加坡科技设计大学）；Agency for Science, Technology and Research (A*STAR)（新加坡科技研究局）
LLMs can Realize Combinatorial Creativity: Generating Creative Ideas via LLMs for Scientific Research
Boston University（波士顿大学）；University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；University of Delaware（特拉华大学）；Acaciawood Preparatory Academy（阿卡西亚伍德预备学院）
Large Language Models as Master Key: Unlocking the Secrets of Materials Science with GPT
University of New South Wales（新南威尔士大学）；GreenDynamics Pty. Ltd（绿动力私人有限公司）；City University of Hong Kong（香港城市大学）；University of Sydney（悉尼大学）；DARE ARC Training Centre in Data Analytics for Resources and Environments（DARE ARC资源与环境数据分析培训中心）
MSEarth: A Multimodal Scientific Dataset and Benchmark for Phenomena Uncovering in Earth Science
The Hong Kong Polytechnic University（香港理工大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；Shanghai Jiao Tong University（上海交通大学）；The Chinese University of Hong Kong（香港中文大学）
