大语言模型生成虚假信息的模型归因：一种结合监督对比学习的领域泛化方法
Model Attribution in LLM-Generated Disinformation: A Domain Generalization Approach with Supervised Contrastive Learning
AI总结：
针对大语言模型生成虚假信息的归因难题，本文将其转化为领域泛化问题，提出基于监督对比学习的方法以增强对提示变化的鲁棒性，经多提示方法、多模型实验验证，在未见数据集上取得了当前最优的归因性能。
摘要：
大语言模型（LLM）生成虚假信息的模型归因，对厘清信息来源、遏制其传播构成了重大挑战。这项任务的难度尤为突出，因为现代大语言模型生成的虚假信息已达到类人质量。此外，生成虚假信息时所用提示方法的多样性，也让精准的来源归因变得更为复杂——这些方法会引入领域特定特征，掩盖模型的根本特征。本文提出将模型归因概念定义为一个领域泛化问题，其中每种提示方法代表一个独特的领域。我们认为，有效的归因模型必须对这些领域特定特征具备不变性，还需能在所有场景下精准识别来源模型，贴合真实世界的检测难点。为解决这一问题，我们提出了一种基于监督对比学习（Supervised Contrastive Learning）的新方法，该方法旨在增强模型对提示变化的鲁棒性，重点区分不同的来源大语言模型。我们通过严格实验评估了模型性能，实验涉及三种常见提示方法：“开放式（open-ended）”“改写（rewriting）”和“释义（paraphrasing）”，以及三款先进大语言模型：“llama 2”“chatgpt”和“vicuna”。结果表明，我们的方法在模型归因任务中效果显著，在多样化的未见数据集上均取得了当前最优性能。
英文摘要：
Model attribution for LLM-generated disinformation poses a significant challenge in understanding its origins and mitigating its spread. This task is especially challenging because modern large language models (LLMs) produce disinformation with human-like quality. Additionally, the diversity in prompting methods used to generate disinformation complicates accurate source attribution. These methods introduce domain-specific features that can mask the fundamental characteristics of the models. In this paper, we introduce the concept of model attribution as a domain generalization problem, where each prompting method represents a unique domain. We argue that an effective attribution model must be invariant to these domain-specific features. It should also be proficient in identifying the originating models across all scenarios, reflecting real-world detection challenges. To address this, we introduce a novel approach based on Supervised Contrastive Learning. This method is designed to enhance the model's robustness to variations in prompts and focuses on distinguishing between different source LLMs. We evaluate our model through rigorous experiments involving three common prompting methods: ``open-ended'', ``rewriting'', and ``paraphrasing'', and three advanced LLMs: ``llama 2'', ``chatgpt'', and ``vicuna''. Our results demonstrate the effectiveness of our approach in model attribution tasks, achieving state-of-the-art performance across diverse and unseen datasets.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
