野外指纹微调语言模型
Fingerprinting Fine-tuned Language Models in the Wild
摘要：
有人担心，语言模型（LM）生成高质量合成文本的能力可能被滥用于发送垃圾邮件、虚假信息或宣传。因此，研究界正在积极努力开发方法来检测给定文本是有机的还是合成的。虽然这是有用的第一步，但重要的是能够进一步识别作者LM以归属其来源。以前对LM进行指纹识别的工作仅限于归因于由少数（通常< 10）预训练的LM生成的合成文本。然而，诸如GPT2的LM通常以无数方式进行微调（例如，在域特定文本语料库上）。对微调LM进行指纹识别是具有挑战性的，因为在现实场景中微调LM的范围要大得多。为了应对这一挑战，我们研究了在野外微调LM的大规模指纹识别问题。使用由108个不同的微调LM生成的合成文本的真实数据集，我们进行了全面的实验，以证明现有指纹识别方法的局限性。我们的研究结果表明，微调本身是最有效的归因微调LM生成的合成文本。
英文摘要：
There are concerns that the ability of language models (LMs) to generate high quality synthetic text can be misused to launch spam, disinformation, or propaganda. Therefore, the research community is actively working on developing approaches to detect whether a given text is organic or synthetic. While this is a useful first step, it is important to be able to further fingerprint the author LM to attribute its origin. Prior work on fingerprinting LMs is limited to attributing synthetic text generated by a handful (usually < 10) of pre-trained LMs. However, LMs such as GPT2 are commonly fine-tuned in a myriad of ways (e.g., on a domain-specific text corpus) before being used to generate synthetic text. It is challenging to fingerprinting fine-tuned LMs because the universe of fine-tuned LMs is much larger in realistic scenarios. To address this challenge, we study the problem of large-scale fingerprinting of fine-tuned LMs in the wild. Using a real-world dataset of synthetic text generated by 108 different fine-tuned LMs, we conduct comprehensive experiments to demonstrate the limitations of existing fingerprinting approaches. Our results show that fine-tuning itself is the most effective in attributing the synthetic text generated by fine-tuned LMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
