发表机构
Amazon Web Services（亚马逊云科技）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
IDP AutoOpt是自主LLM智能体，通过闭环流程优化IDP流水线配置，在多领域任务上性能优于人类专家且成本更低，还可扩展至RAG等其他企业AI系统。
AI 中文摘要
我们提出了IDP AutoOpt，一种自主大语言模型（LLM）智能体，用于发现智能文档处理（IDP）流水线的高性能配置。当前，领域专家联合调优IDP提示词、模型、光学字符识别（OCR）设置和模式，每种文档类型需耗费20至80+人时，且随着企业增加文档类别，该过程无法扩展。IDP AutoOpt运行闭环流程：在小型标注集上对配置评分，诊断字段级错误，生成针对性编辑，再重新评估，其由人类编写的领域技能引导，这些技能编码了生产专业知识。在医疗、营销智能和金融服务场景中部署的抽取、分类及数据包拆分任务上，IDP AutoOpt在成本相当或更低的情况下达到或超过人类专家的准确率（在某抽取基准上，准确率为90.2%，而人类专家为81.6%，单页成本降低4.6倍），将配置时间从数周缩短至两小时以内。我们进一步发现，智能体LLM能力存在一个硬阈值，低于该阈值优化会失败，且精心设计的领域技能优于原始源代码访问，后者在无结构提供时会降低性能。我们还分享了上下文管理和方差缓解的实用经验。该方法仅需可配置的流水线、评分函数和小型标注集，可从IDP扩展到其他企业AI系统，如检索增强生成（RAG）和多智能体工作流，这些系统的配置会阻碍部署。
英文摘要
We present IDP AutoOpt, an autonomous LLM agent that discovers high-performing configurations for intelligent document processing (IDP) pipelines. Tuning IDP prompts, models, OCR settings, and schemas jointly currently costs domain specialists 20 to 80+ person-hours per document type and does not scale as enterprises add document classes. IDP AutoOpt runs a closed loop: it scores a configuration on a small labeled set, diagnoses field-level errors, generates targeted edits, and re-evaluates, guided by human-authored domain skills that encode production expertise. Across extraction, classification, and packet-splitting tasks deployed in healthcare, marketing-intelligence, and financial-services settings, IDP AutoOpt matches or exceeds human-expert accuracy at equal or lower cost (on an extraction benchmark, 90.2% vs 81.6% at 4.6 x lower per-page cost), cutting configuration time from weeks to under two hours. We further show that agent LLM capability has a hard threshold below which optimization fails, and that curated domain skills outperform raw source-code access, which can degrade performance when provided without structure. We also share practical lessons on context management and variance mitigation. Requiring only a configurable pipeline, a scoring function, and a small labeled set, the approach extends beyond IDP to other enterprise AI systems, such as RAG and multi-agent workflows, where configuration bottlenecks deployment.
