推理时对修辞角色标注中困难示例的语义重排序
Semantic Reranking at Inference Time for Hard Examples in Rhetorical Role Labeling
AI总结:
本文提出RISE框架,在推理时利用标签语义对修辞角色标注中的困难示例进行重排序,提升模型预测的准确性和鲁棒性。
摘要:
修辞角色标注(RRL)为文档中的每个句子分配一个功能角色,广泛应用于法律、医疗和科学领域。尽管语言模型(LMs)在平均性能上表现良好,但它们在困难示例上仍然不可靠,其中预测置信度较低。现有方法通常隐式处理不确定性,将标签视为离散标识符,忽略了标签名称中编码的语义信息。我们引入RISE,一种推理时的语义重排序框架,利用标签语义来优化困难实例的预测。RISE自动识别低置信度预测,并使用对比学习的标签表示对模型输出进行重排序,无需重新训练或修改基础模型。在八个领域特定的RRL数据集上,使用七种LM(包括基于编码器和因果架构)的实验表明,在困难示例上平均获得+9.15个宏F1分数的提升。为了可解释性,我们进一步提出手动难度注释,从模型和人类视角研究难度,揭示与Cohen's kappa=0.40的中等一致程度。
英文摘要:
Rhetorical Role Labeling (RRL) assigns a functional role to each sentence in a document and is widely used in legal, medical, and scientific domains. While language models (LMs) achieve strong average performance, they remain unreliable on hard examples, where prediction confidence is low. Existing approaches typically handle uncertainty implicitly and treat labels as discrete identifiers, overlooking the semantic information encoded in label names. We introduce RISE, an inference-time semantic reranking framework that leverages label semantics to refine predictions on hard instances. RISE automatically identifies low-confidence predictions and reranks model outputs using contrastively learned label representations, without retraining or modifying the underlying model. Experiments on eight domain-specific RRL datasets with seven LMs, including encoder-based and causal architectures, show an average gain of +9.15 macro-F1 points on hard examples. For explainability, we further propose manual hardness annotations to study difficulty from both model and human perspectives, revealing a moderate agreement with Cohen's kappa = 0.40.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。