PANDA——来自亚洲的配对反仇恨叙事数据集：使用 LLM-as-a-Judge 创建首个中文反言论数据集
PANDA -- Paired Anti-hate Narratives Dataset from Asia: Using an LLM-as-a-Judge to Create the First Chinese Counterspeech Dataset
AI总结：
针对中文反言论资源缺失问题，本文结合 LLM-as-a-Judge、模拟退火、零样本生成与循环赛机制构建并人工验证首个中文 CS 语料库 PANDA。
摘要：
尽管现代标准汉语在全球广泛使用，但中文的反言论（CS）资源实际上仍不存在。为弥补东亚反言论研究中的这一空白，我们引入了一个现代标准普通话反言语料库，重点关注在中国大陆打击仇恨言论。本文提出了一种生成 CS 的新方法，使用 LLM-as-a-Judge、模拟退火、LLM 零样本中文生成以及循环赛算法。随后进行人工验证，以确保质量和语境相关性。本文详细介绍了用中文及其他非欧洲中心语言创建有效反言论的方法，包括哪些群体受到诋毁的独特文化模式，以及何种话语标记在程序上被标记为仇恨言论（HS）的语言模式。通过对生成语料库的分析，我们提供了有力证据，表明缺乏开源、标注恰当的中文仇恨言论数据，也表明使用 LLM-as-a-Judge 为中文候选答案评分存在局限。此外，现有语料库是首个基于东亚语言的 CS 语料库，为未来反言论生成与评估研究提供了重要资源。
英文摘要：
Despite the global prevalence of Modern Standard Chinese language, counterspeech (CS) resources for Chinese remain virtually nonexistent. To address this gap in East Asian counterspeech research we introduce the a corpus of Modern Standard Mandarin counterspeech that focuses on combating hate speech in Mainland China. This paper proposes a novel approach of generating CS by using an LLM-as-a-Judge, simulated annealing, LLMs zero-shot CN generation and a round-robin algorithm. This is followed by manual verification for quality and contextual relevance. This paper details the methodology for creating effective counterspeech in Chinese and other non-Eurocentric languages, including unique cultural patterns of which groups are maligned and linguistic patterns in what kinds of discourse markers are programmatically marked as hate speech (HS). Analysis of the generated corpora, we provide strong evidence for the lack of open-source, properly labeled Chinese hate speech data and the limitations of using an LLM-as-Judge to score possible answers in Chinese. Moreover, the present corpus serves as the first East Asian language based CS corpus and provides an essential resource for future research on counterspeech generation and evaluation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
