Northeastern Uni在多语言反制言论生成中：通过直接偏好优化的LLM对齐增强反制言论生成
Northeastern Uni at Multilingual Counterspeech Generation: Enhancing Counter Speech Generation with LLM Alignment through Direct Preference Optimization
AI总结：
本文提出通过SFT和DPO对齐LLM并结合知识基础，显著提升了多语言环境下的反制言论生成质量，DPO模型在CS基准上超越基线并有效扩展至多语言。
摘要：
反制言论（CS）的自动生成是应对仇恨言论的关键策略，通过提供建设性和知情的回应来实现。然而，现有方法往往无法生成高质量、有影响力且可扩展的CS，尤其是在多样的语言环境中。本文提出了一种新颖的方法，通过使用监督微调（SFT）和直接偏好优化（DPO）对齐大型语言模型（LLM）来增强CS生成。我们的方法利用DPO将LLM输出与人类偏好对齐，确保上下文适当且语言适应性强的回应。此外，我们引入知识基础以增强生成CS的事实准确性和相关性。实验结果表明，DPO对齐的模型在CS基准上显著优于SFT基线，同时有效扩展到多种语言。这些发现突显了基于偏好的对齐技术在推进不同语言环境CS生成方面的潜力。模型监督和对齐以英语完成，同一模型用于报告其他语言（如巴斯克语、意大利语和西班牙语）的指标。
英文摘要：
The automatic generation of counter-speech (CS) is a critical strategy for addressing hate speech by providing constructive and informed responses. However, existing methods often fail to generate high-quality, impactful, and scalable CS, particularly across diverse linguistic contexts. In this paper, we propose a novel methodology to enhance CS generation by aligning Large Language Models (LLMs) using Supervised Fine-Tuning (SFT) and Direct Preference Optimization (DPO). Our approach leverages DPO to align LLM outputs with human preferences, ensuring contextually appropriate and linguistically adaptable responses. Additionally, we incorporate knowledge grounding to enhance the factual accuracy and relevance of generated CS. Experimental results demonstrate that DPO-aligned models significantly outperform SFT baselines on CS benchmarks while scaling effectively to multiple languages. These findings highlight the potential of preference-based alignment techniques to advance CS generation across varied linguistic settings. The model supervision and alignment is done in English and the same model is used for reporting metrics across other languages like Basque, Italian, and Spanish.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
