SlopShape：识别AI生成的商业网页内容
SlopShape: Identifying AI-Generated Commercial Web Content
查看机构详情
- Sitefire
机构由 AI 辅助整理，请以论文原文为准。
浏览论文内容
中文总结 AI 辅助
本研究提出SlopShape方法，利用214维结构特征在商业内容中识别AI生成文本，实现98.0宏F1分数，并能以79.3%准确率归因来源模型，验证了结构信号的有效性。
中文摘要 AI 辅助
词级检测器几乎能完美识别未经编辑的AI生成文本，但文献记录了它们在改写情况下的脆弱性，而且词级分数既不能刻画文本特征，也无法识别出是哪个AI模型撰写的文本。我们探究能否从更深一层的结构特征来识别AI生成文本：即信息如何呈现、以何种顺序、使用何种证据、以何种口吻。我们复现了StoryScope（Russell等人，2026）——该方法曾展示此类模式可识别AI生成的小说——并将其应用于商业内容：来自268个公司域的2,250篇ChatGPT之前的真实博客文章，与来自五个前沿模型的11,250篇AI镜像文章进行对比。一个由LLM应用并经人工金标准标注会话验证（人工间kappa值为0.928，人工与模型间为0.946）的214维特征工具，仅凭其187个结构特征即可在留出公司上以98.0的宏F1分数检测出AI文章，且当每篇AI文章被其自身模型改写后，该分数保持不变（98.1）。该信号具有刻画和归因能力：AI文章具有整洁、自我宣告的形状，79.3%的AI文章被正确归因到其来源（随机概率为16.7%），而人类文章则占据罕见的结构配置。所有效应均复现了StoryScope的结果，方向一致且幅度更大。我们发布了流程、工具、提示词、代码和聚合工件。
英文摘要
Word-level detectors identify unedited AI-generated text almost perfectly, but the literature documents their brittleness under rewording, and a word-level score neither characterizes a text nor identifies which AI model wrote it. We ask whether AI-generated text can be identified one level deeper, from structural signatures: how information is presented, in what order, with what evidence, and in what voice. We replicate StoryScope (Russell et al., 2026), which showed such patterns for AI-generated fiction, on commercial content: 2,250 pre-ChatGPT human blog posts from 268 company domains against 11,250 AI mirrors from five frontier models. A 203-feature instrument, applied by an LLM and validated in a human gold-annotation session (human-human kappa 0.939, human-model 0.951), detects AI posts from its 176 structural features alone at 97.0 macro-F1 on held-out companies, nearly unchanged (96.1) when every AI post is reworded by its own model. The signal characterizes and attributes: AI posts share a tidy, self-announcing shape, 68.6% are attributed to the correct source against a 16.7% chance rate, and human posts occupy rare structural configurations. All effects replicate StoryScope's, consistent in direction and at least as large in magnitude. We release pipeline, instrument, prompts, code, and aggregate artifacts.
