FRACTAL:基于聚合文本标签的细粒度评分
FRACTAL: Fine-Grained Scoring from Aggregate Text Labels
AI总结:
本文提出 FRACTAL,结合 MIL、LLP 与句子先验,将回复级聚合标签解聚为句子级伪标签并训练评分模型,在多任务上达到接近细粒度人工标注训练模型的效果。
摘要:
大语言模型(LLM)正越来越多地被微调,以支撑写作、事实查询、提问和推理等复杂生成任务。传统上,用于评估并进一步微调 LLM 性能的人类或模型反馈是在回复层面提供的,从而实现更快、更具成本效益的评估。然而,近期研究(Amplayo 等人 [2022]、Wu 等人 [2023])表明,句子层面的标签或许能为 LLM 优化提供更准确、更具可解释性的反馈。在本研究中,我们提出了将回复层面标签解聚为句子层面(伪)标签的方法。我们的方法利用多示例学习(MIL)和标签比例学习(LLP)技术,并结合先验信息(例如文档—句子余弦相似度),训练一个专门用于句子层面评分的模型。我们还采用相关技术,利用模型预测在句子层面对训练集进行伪标注,以用于模型训练,从而进一步提升性能。我们在六个数据集和四类任务上对所提方法进行了广泛评估:检索、问答、摘要和数学推理。结果表明,在其中大多数任务上,相比多个基线,我们的方法取得了性能提升。我们的工作首次开发了将回复层面反馈转化为句子层面评分的技术,利用句子层面的先验信息,并在多个任务上进行了全面评估,同时开展端到端微调评估;结果显示,其性能可与在细粒度人工标注标签上训练的模型相媲美。
英文摘要:
Large language models (LLMs) are being increasingly tuned to power complex generation tasks such as writing, fact-seeking, querying and reasoning. Traditionally, human or model feedback for evaluating and further tuning LLM performance has been provided at the response level, enabling faster and more cost-effective assessments. However, recent works (Amplayo et al. [2022], Wu et al. [2023]) indicate that sentence-level labels may provide more accurate and interpretable feedback for LLM optimization. In this work, we introduce methods to disaggregate response-level labels into sentence-level (pseudo-)labels. Our approach leverages multiple instance learning (MIL) and learning from label proportions (LLP) techniques in conjunction with prior information (e.g., document-sentence cosine similarity) to train a specialized model for sentence-level scoring. We also employ techniques which use model predictions to pseudo-label the train-set at the sentence-level for model training to further improve performance. We conduct extensive evaluations of our methods across six datasets and four tasks: retrieval, question answering, summarization, and math reasoning. Our results demonstrate improved performance compared to multiple baselines across most of these tasks. Our work is the first to develop response-level feedback to sentence-level scoring techniques, leveraging sentence-level prior information, along with comprehensive evaluations on multiple tasks as well as end-to-end finetuning evaluation showing performance comparable to a model trained on fine-grained human annotated labels.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。