发表机构
Johns Hopkins University; Optum（约翰斯·霍普金斯大学; 奥普特姆）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
针对结构化预测中上下文学习难以捕捉词元级模式及任务特定标注的问题，提出Verb-ICL选择性标注框架，通过词元级覆盖选择示例并生成错误反馈，在六个数据集上优于基线。
AI 中文摘要
结构化预测任务对上下文学习（ICL）提出了独特挑战：其组合性输出需要建模细粒度、词元级别的模式，而句子级方法无法捕捉这些模式；此外，任务特定的标注约定是人类定义的产物，无法仅通过预训练获得。我们提出Verb-ICL，一种用于基于ICL的结构化预测的选择性标注框架，以应对这两个挑战。Verb-ICL首先使用词元级覆盖策略选择代表性示例，该策略捕捉对结构化预测至关重要的局部语义模式，然后生成可操作的错误反馈，将任务特定的标注指南编码，并将此反馈纳入ICL演示中。我们在涵盖信息抽取和语义解析的六个结构化预测数据集上评估了Verb-ICL。使用近期大语言模型的实验表明，在低资源设置下，Verb-ICL始终优于强选择性标注基线，并且随着标注预算的增加继续带来收益。扩展分析表明，生成的反馈在四类别质量分类法中主要是有用的，作为任务级指导超越实例特定修正进行泛化，并且无论底层选择策略如何都能提升性能。
英文摘要
Structured prediction tasks pose unique challenges for in-context learning (ICL): their compositional outputs require modeling fine-grained, token-level patterns that sentence-level approaches fail to capture, and their task-specific annotation conventions are human-defined artifacts that cannot be acquired through pretraining alone. We propose Verb-ICL, a selective annotation framework for ICL-based structured prediction that addresses both challenges. Verb-ICL first selects representative examples using a token-level coverage strategy that captures local semantic patterns critical for structured prediction, then generates actionable error feedback that codifies task-specific annotation guidelines and incorporates this feedback into ICL demonstrations. We evaluate Verb-ICL on six structured prediction datasets spanning information extraction and semantic parsing. Experiments with recent LLMs show that Verb-ICL consistently outperforms strong selective annotation baselines under low-resource settings and continues to provide gains as the annotation budget increases. Extended analyses demonstrate that the generated feedback is predominantly useful across a four-category quality taxonomy, generalizes as task-level guidance beyond instance-specific corrections, and improves performance regardless of the underlying selection strategy.
CommentsAccepted to COLM 2026
