面向主题聚焦式摘要的对数几率重加权方法
Logit Reweighting for Topic-Focused Summarization
AI总结：
本文提出通过对主题相关词元的对数几率进行直接重加权来提升摘要的主题聚焦性，并在NEWTS数据集上验证了三种技术，其中阈值选择方法能在不牺牲摘要质量的前提下改善主题相关性。
摘要：
生成符合特定主题的抽象式摘要对于语言模型而言仍然是一项重大挑战。虽然微调等标准方法资源消耗较大，而提示工程等较简单的方法往往难以维持主题聚焦，尤其是在较小的模型上。为了解决这一问题，我们提出了一种轻量级方法，通过在生成过程中直接对主题相关词元的对数几率（logits）进行重加权来增强主题相关性。我们评估了三种此类重加权技术：常数偏移（Constant Shift），即向对数几率添加一个常数值；因子缩放（Factor Scaling），即将其乘以一个因子；以及阈值选择（Threshold Selection），即有选择地提升超过概率阈值的对数几率。在NEWTS主题摘要数据集上使用Gemma-2B和Llama-3-8B模型进行的实验表明，这些技术能够有效增加主题相关词汇的使用。值得注意的是，阈值选择方法成功改善了主题聚焦，同时没有牺牲摘要质量——而这在其他方法中常常是一种权衡。我们的研究结果表明，直接重加权对数几率是微调的一种实用且资源高效的替代方案，为精确控制生成文本的主题内容提供了一条有前景的途径。
英文摘要：
Generating abstractive summaries that adhere to a specific topic remains a significant challenge for language models. While standard approaches, such as fine-tuning, are resource-intensive, simpler methods like prompt engineering often struggle to maintain topical focus, particularly with smaller models. To address this, we propose a lightweight method that enhances topical relevance by directly reweighting the logits of topic-relevant tokens during generation. We evaluate three such reweighting techniques: Constant Shift, which adds a constant value to logits; Factor Scaling, which multiplies them by a factor; and Threshold Selection, which selectively boosts logits that exceed a probability threshold. Experiments on the NEWTS topical summarization dataset, using both Gemma-2B and Llama-3-8B models, show that these techniques effectively increase the use of topic-relevant vocabulary. Notably, the Threshold Selection method successfully improves topical focus without compromising summary quality-a trade-off often seen in other approaches. Our findings demonstrate that directly reweighting logits is a practical and resource-efficient alternative to fine-tuning, offering a promising pathway for precisely controlling the thematic content of generated text.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
