带意图阅读——中和意图
Reading with Intent -- Neutralizing Intent
- Georgia Institute of Technology（佐治亚理工学院）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
本文针对 RAG 中情绪化上下文影响性能的问题，构建含 11 种情绪的合成数据集并训练情绪翻译模型，将段落中和后使 Reading with Intent 任务性能提升约 3%。
AI中文摘要：
对大型语言模型（LLM）的查询可分为两部分：指令/问题以及附带的上下文。大多数基准测试中，检索增强生成（RAG）系统的上下文来自 Wikipedia 或类似 Wikipedia 的文本，这些文本以中立、事实性的语气撰写。然而，当 RAG 系统检索基于互联网的内容时，会遇到具有多种语气和语言风格的文本，从而给下游任务带来挑战。Reading with Intent 任务通过评估上下文段落中不同语气如何影响模型性能来解决这一问题。在先前聚焦讽刺的工作基础上，我们扩展了这一范式：采用更好的合成数据生成方法，构建了一个将上下文段落转换为 11 种不同情绪的数据集。利用该数据集，我们训练了一个情绪翻译模型，以系统性地使段落适应指定的情绪语气。人工评估表明，经过微调成为情绪翻译器的 LLM 从合成生成的数据中受益。最后，该情绪翻译器被用于 Reading with Intent 任务，将段落转换为中立语气。通过中和段落，它缓解了讽刺性段落带来的挑战，并使该任务的总体结果提高约 3%。
英文摘要：
Queries to large language models (LLMs) can be divided into two parts: the instruction/question and the accompanying context. The context for retrieval-augmented generation (RAG) systems in most benchmarks comes from Wikipedia or Wikipedia-like texts which are written in a neutral and factual tone. However, when RAG systems retrieve internet-based content, they encounter text with diverse tones and linguistic styles, introducing challenges for downstream tasks. The Reading with Intent task addresses this issue by evaluating how varying tones in context passages affect model performance. Building on prior work that focused on sarcasm, we extend this paradigm by constructing a dataset where context passages are transformed to $11$ distinct emotions using a better synthetic data generation approach. Using this dataset, we train an emotion translation model to systematically adapt passages to specified emotional tones. The human evaluation shows that the LLM fine-tuned to become the emotion-translator benefited from the synthetically generated data. Finally, the emotion-translator is used in the Reading with Intent task to transform the passages to a neutral tone. By neutralizing the passages, it mitigates the challenges posed by sarcastic passages and improves overall results on this task by about $3\%$.
