arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.03849cs.HCcs.AIcs.CLcs.LG
收藏

HybridQuestion: 人机协作识别高影响力研究问题

HybridQuestion: Human-AI Collaboration for Identifying High-Impact Research Questions

Department of Electronic Engineering, BNRist, Tsinghua University(电子工程系、北京理工大学、清华大学)

Keyu Zhao, Fengli Xu, Yong Li, Tie-Yan Liu

AI总结:

HybridQuestion通过人机协作识别高影响力研究问题,利用AI处理数据与人类判断结合,验证了在科学突破和未来问题预测中的有效性。

摘要:

"AI科学家"范式通过自动化研究过程的关键阶段,从想法生成到学术写作,正在改变科学研究。这种转变预计会加速发现并扩大科学探究的范围。然而,一个关键问题仍然不明确:AI科学家能否识别有意义的研究问题?尽管大型语言模型(LLMs)已在特定任务的创意生成中得到成功应用,但它们在评估过去突破和未来问题的战略、长期潜力方面仍 largely 未被探索。为解决这一差距,我们探索了一种人机混合解决方案,将AI的可扩展数据处理能力与人类专家的价值判断相结合。我们的方法分为三个阶段。第一阶段,AI加速信息收集,利用AI在处理大量文献方面的优势生成混合信息库。第二阶段,候选问题提出,利用合成数据促使六个不同的LLMs提出初始候选池,通过跨模型投票机制进行过滤。第三阶段,混合问题选择,通过多阶段过滤过程逐步增加人类监督。为了验证该系统,我们进行了一项实验,旨在识别2025年的十大科学突破和2026年的十大科学问题,涵盖五个主要学科。我们的分析表明,尽管AI代理在识别已确立的突破方面与人类专家高度一致,但在预测前瞻性问题方面表现出更大的分歧,这表明人类判断在评估主观、前瞻性挑战方面仍然至关重要。

英文摘要:

The "AI Scientist" paradigm is transforming scientific research by automating key stages of the research process, from idea generation to scholarly writing. This shift is expected to accelerate discovery and expand the scope of scientific inquiry. However, a key question remains unclear: can AI scientists identify meaningful research questions? While Large Language Models (LLMs) have been applied successfully to task-specific ideation, their potential to conduct strategic, long-term assessments of past breakthroughs and future questions remains largely unexplored. To address this gap, we explore a human-AI hybrid solution that integrates the scalable data processing capabilities of AI with the value judgment of human experts. Our methodology is structured in three phases. The first phase, AI-Accelerated Information Gathering, leverages AI's advantage in processing vast amounts of literature to generate a hybrid information base. The second phase, Candidate Question Proposing, utilizes this synthesized data to prompt an ensemble of six diverse LLMs to propose an initial candidate pool, filtered via a cross-model voting mechanism. The third phase, Hybrid Question Selection, refines this pool through a multi-stage filtering process that progressively increases human oversight. To validate this system, we conducted an experiment aiming to identify the Top 10 Scientific Breakthroughs of 2025 and the Top 10 Scientific Questions for 2026 across five major disciplines. Our analysis reveals that while AI agents demonstrate high alignment with human experts in recognizing established breakthroughs, they exhibit greater divergence in forecasting prospective questions, suggesting that human judgment remains crucial for evaluating subjective, forward-looking challenges.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑