我们发现,当执行重写任务时,大语言模型(LLM)更倾向于修改人类撰写的文本,而非AI生成的文本。这种倾向的产生是因为LLM通常认为AI生成的文本质量更高,因此做出的修改更少。我们提出了一种检测AI生成内容的方法:通过提示LLM重写文本并计算输出的编辑距离来实现检测。我们将这种“通过重写进行生成式AI检测(geneRative AI Detection viA Rewriting)”的方法命名为Raidar。在新闻、创意写作、学生论文、代码、Yelp评论和arXiv论文等多个领域中,Raidar显著提升了现有AI内容检测模型(包括学术模型和商业模型)的F1检测分数,提升幅度最高可达29个点。我们的方法仅基于词符号运行,无需高维特征,可兼容黑盒LLM,且对新内容具有固有鲁棒性。我们的研究结果通过机器自身的视角,揭示了机器生成文本的独特印记。
英文摘要:
We find that large language models (LLMs) are more likely to modify human-written text than AI-generated text when tasked with rewriting. This tendency arises because LLMs often perceive AI-generated text as high-quality, leading to fewer modifications. We introduce a method to detect AI-generated content by prompting LLMs to rewrite text and calculating the editing distance of the output. We dubbed our geneRative AI Detection viA Rewriting method Raidar. Raidar significantly improves the F1 detection scores of existing AI content detection models -- both academic and commercial -- across various domains, including News, creative writing, student essays, code, Yelp reviews, and arXiv papers, with gains of up to 29 points. Operating solely on word symbols without high-dimensional features, our method is compatible with black box LLMs, and is inherently robust on new content. Our results illustrate the unique imprint of machine-generated text through the lens of the machines themselves.
ConvGQR: Generative Query Reformulation for Conversational Search
University of Montreal(蒙特利尔大学);Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学高瓴人工智能学院);Institute for AI Industry Research, Tsinghua University(清华大学人工智能产业研究院)
QueStER: Query Specification for Generative keyword-based Retrieval
MILA – Quebec AI Institute & ILLS(魁北克人工智能研究所);Université Paris-Saclay & CentraleSupélec & CNRS(巴黎-萨克雷大学);Air Liquide(Air Liquide公司);Sorbonne Université & ISIR & CNRS(索邦大学)
When Compression Helps and When It Hurts: Condition-Aware Analysis of Chain-of-Thought Distillation
Ningbo Institute of Digital Twin, Eastern Institute of Technology, Ningbo(宁波数字孪生研究院,东方理工高等研究院,宁波);Viterbi School of Engineering, University of Southern California(南加州大学维特比工程学院);The Hong Kong Polytechnic University(香港理工大学);LMU Munich(慕尼黑大学);Saarland University(萨尔大学)
SongSage: A Large Musical Language Model with Lyric Generative Pre-training
School of Computer Science, Wuhan University(武汉大学计算机学院);School of Information Management, Wuhan University(武汉大学信息管理学院);Tsinghua University(清华大学);School of Artificial Intelligence, Wuhan University(武汉大学人工智能学院)
Rethinking On-policy Optimization for Query Augmentation
University of Utah(犹他大学);The University of Queensland(昆士兰大学);University of Waterloo(滑铁卢大学);New York University(纽约大学);University of Notre Dame(圣母大学);Université de Montréal(蒙特利尔大学);Google DeepMind(谷歌DeepMind);University of Oklahoma(俄克拉荷马大学)