RENOVI：面向社会文化对话中规范违规补救的基准
RENOVI: A Benchmark Towards Remediating Norm Violations in Socio-Cultural Conversations
AI总结：
本文提出大规模多轮对话基准ReNoVi，包含人工编写与ChatGPT生成的合成数据，通过定义逐步任务序列评估并提升交互式AI系统在社会文化对话中补救规范违规的能力。
摘要：
当个体未能遵循文化上可接受的行为时，就会发生规范违规，这可能导致潜在冲突。补救规范违规需要具备社会意识以及对相关细微差别的文化敏感性。为了赋予交互式AI系统补救能力，我们提供了ReNoVi——一个包含9,258轮多轮对话的大规模语料库，标注了社会规范，并定义了一系列任务以帮助逐步理解和补救规范违规。ReNoVi由两部分组成：512个人工编写的对话（真实数据），以及通过提示学习由ChatGPT生成的8,746个合成对话。虽然收集足够的人工编写数据成本高昂，但合成对话提供了适量的数据以帮助缓解训练数据的稀缺性，并提供了评估LLMs与人类在社会规范意识上对齐程度的机会。因此，我们利用ChatGPT的能力为我们的任务生成合成训练数据。为确保人工编写和合成数据的质量，我们在数据收集过程中遵循质量控制协议。我们的实验结果证明了在社会文化对话中补救规范违规的重要性，以及从合成数据中获得的性能提升。
英文摘要：
Norm violations occur when individuals fail to conform to culturally accepted behaviors, which may lead to potential conflicts. Remediating norm violations requires social awareness and cultural sensitivity of the nuances at play. To equip interactive AI systems with a remediation ability, we offer ReNoVi - a large-scale corpus of 9,258 multi-turn dialogues annotated with social norms, as well as define a sequence of tasks to help understand and remediate norm violations step by step. ReNoVi consists of two parts: 512 human-authored dialogues (real data), and 8,746 synthetic conversations generated by ChatGPT through prompt learning. While collecting sufficient human-authored data is costly, synthetic conversations provide suitable amounts of data to help mitigate the scarcity of training data, as well as the chance to assess the alignment between LLMs and humans in the awareness of social norms. We thus harness the power of ChatGPT to generate synthetic training data for our task. To ensure the quality of both human-authored and synthetic data, we follow a quality control protocol during data collection. Our experimental results demonstrate the importance of remediating norm violations in socio-cultural conversations, as well as the improvement in performance obtained from synthetic data.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
