为何大语言模型尚未成为科学家：来自四个自主研究尝试的教训
Why LLMs Aren't Scientists Yet: Lessons from Four Autonomous Research Attempts
AI总结：
本文探讨了四个自主生成科研论文的尝试，揭示了大语言模型在科学工作流程中面临的六个常见失败模式，并提出了改进AI科学家系统的设计原则。
摘要：
我们报告了四个端到端尝试，旨在使用由六个LLM代理组成的管道，自主生成机器学习研究论文。这些代理被映射到科学工作流程的各个阶段。其中四个尝试中，有三个在实现或评估过程中失败。一个完成了整个流程，并被接受为Agents4Science 2025的实验性首次出版物，该出版物要求AI系统作为第一作者，通过了人类和多AI评审。从这些尝试中，我们记录了六个反复出现的失败模式：对训练数据默认值的偏见、在执行压力下出现的实现漂移、在长周期任务中出现的记忆和上下文退化、过度兴奋导致即使明显失败也宣称成功、领域智能不足，以及实验设计中的科学品味薄弱。我们最后讨论了四种设计原则，以构建更稳健的AI科学家系统，自主科学发现的启示，并在https://github.com/Lossfunk/ai-scientist-artefacts-v1上发布了所有提示、制品和输出。
英文摘要：
We report a case study of four end-to-end attempts to autonomously generate ML research papers using a pipeline of six LLM agents mapped to stages of the scientific workflow. Of these four, three attempts failed during implementation or evaluation. One completed the pipeline and was accepted to Agents4Science 2025, an experimental inaugural venue that required AI systems as first authors, passing both human and multi-AI review. From these attempts, we document six recurring failure modes: bias toward training data defaults, implementation drift under execution pressure, memory and context degradation across long-horizon tasks, overexcitement that declares success despite obvious failures, insufficient domain intelligence, and weak scientific taste in experimental design. We conclude by discussing four design principles for more robust AI-scientist systems, implications for autonomous scientific discovery, and we release all prompts, artifacts, and outputs at https://github.com/Lossfunk/ai-scientist-artefacts-v1
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
