发表机构
Signal 1 AI; Layer 6 AI（Signal 1 AI; Layer 6 AI）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
该研究提出共形相关性框架，利用上下文学习示例策划与集成创建评分函数，在保持覆盖度的同时提升NLP任务的简洁性，经7项任务验证且从理论上分析了集成共形评分的多样性影响。
AI 中文摘要
摘要：许多NLP任务（如摘要抽取和抽取式问答）可简化为在覆盖度（保留足够相关信息以达成目标）和简洁性（尽可能移除无关信息）两个约束下从文档中检索相关内容。共形预测方法已被用于保证覆盖度，但需通过设计评分函数来优化简洁性。当前最优的评分函数采用手工设计的大语言模型（LLM）提示词，要求模型对内容重要性进行评分，但手动提示词工程既费力又依赖特定任务。我们提出了共形相关性（Conformal Relevance）框架，该框架利用上下文学习示例的策划与集成来创建评分函数，在保持覆盖度的同时，以最少的手动输入提升简洁性。我们展示了该框架在7项NLP任务上的应用，还从理论上研究了集成共形评分的多样性影响，给出了互补性条件（用于表征集成何时能改进最坏情况下的句子评分）以及集成改进的饱和边界。
英文摘要
Many NLP tasks, such as summarization and extractive question answering, reduce to retrieving relevant content from documents under two constraints: coverage, retaining enough pertinent information to achieve some goal, and conciseness, removing as much irrelevant information as possible. Conformal prediction methods have been used to guarantee coverage, and must be optimized for conciseness through design of a score function. State-of-the-art scoring functions use hand-engineered LLM prompts asking the model to rate the importance of content, but manual prompt engineering is labor-intensive and task-specific. We introduce the Conformal Relevance framework which uses in-context learning example curation and ensembling to create a score function which maintains coverage while improving conciseness with minimal manual input. We demonstrate this framework's application on seven NLP tasks, and also theoretically study the impact of diversity for ensembled conformal scores, giving a complementarity condition that characterizes when ensembling improves worst-case sentence scores, and a saturation bound on ensemble improvement.
CommentsFindings of EMNLP 2026. Code is available at https://github.com/layer6ai-labs/conformal-relevance
