发表机构
University of Science and Technology of China; State Key Laboratory of Cognitive Intelligence; CCCC Second Highway Consultants Co., Ltd.（中国科学技术大学; 认知智能全国重点实验室; 中交第二公路勘察设计研究院有限公司）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
针对科学提取智能体，提出从弱任务规范自动构建任务配置并持续优化的框架，在异质催化文献上验证了联合优化模式与指令的有效性。
AI 中文摘要
大多数优化LLM提示和智能体工作流的方法都假设任务特定的输出模式、提取指令和评估标准是预定义的。然而，对于科学提取智能体，一个简短的任务目标可能无法完全确定这些组件，而手动指定这些组件成本高昂。我们研究了从仅包含简短目标和未标注参考文档的弱规范中构建任务特定配置的上游问题。我们的框架并未将自动构建视为固定的预处理步骤，而是构建任务特定的模式、提取指令和基础训练评分标准，然后在优化过程中保持模式构建和提取指令的可编辑性。失败聚焦更新将文本梯度反馈集中在得分较低的文档上，而训练时评估标准则适应反复出现的失败。在异质催化文献语料库上，自动构建仍然可以改进，并且同时优化模式构建和提取指令在所有四种评判者-评分标准设置中表现最佳，消融实验和盲法人工评估支持所提出的方案。
英文摘要
Most methods that optimize LLM prompts and agent workflows assume that task-specific output schemas, extraction instructions, and evaluation criteria are predefined. For scientific extraction agents, however, a short task goal may not fully determine these components, while specifying them manually is costly. We study the upstream problem of constructing the task-specific configuration from a weak specification containing only a short goal and unannotated reference documents. Rather than treating automatic construction as a fixed preprocessing step, our framework constructs a task-specific schema, extraction instructions, and base training rubrics, then keeps schema construction and extraction instructions editable during optimization. Failure-focused updates concentrate textual-gradient feedback on lower-scoring documents, while training-time evaluation criteria adapt to recurring failures. On a heterogeneous-catalysis literature corpus, automatic construction remains improvable, and optimizing both schema construction and extraction instructions performs best across all four judge-rubric settings, with ablations and blinded human evaluation supporting the proposed formulation.
