arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.18152cs.CLcs.CYphysics.soc-ph
收藏

大语言模型的统计特征

The Statistical Signature of LLMs

Department of Computer Science, Sapienza University of Rome(计算机科学系,罗马萨皮恩扎大学);Department of Computer, Control and Management Engineering, Sapienza University of Rome(计算机、控制与管理工程系,罗马萨皮恩扎大学);Department of Legal, Social, and Educational Sciences, Tuscia University(法律、社会与教育科学系,图斯西亚大学)

Ortal Hadad, Edoardo Loru, Jacopo Nudo, Niccolò Di Marco, Matteo Cinelli, Walter Quattrociocchi

AI总结:

本研究通过无损压缩分析揭示大语言模型生成文本的统计特征,发现其在受控和中介情境中比人类文本更具规律性和可压缩性,但小尺度下这种差异会减弱。

摘要:

大型语言模型通过从高维分布中进行概率采样来生成文本,但这一过程如何重塑语言的结构统计组织仍不完全明确。在这里,我们展示出无损压缩提供了一种简单的、对模型不敏感的统计规律度量方法,可以直接从表面文本区分生成模式。我们分析了压缩行为在三个渐进复杂的信息生态系统中的表现:受控的人类-LLM延续、知识基础设施的生成中介(维基百科 vs. Grokipedia)以及完全合成的社会互动环境(Moltbook vs. Reddit)。在各种情境中,压缩揭示出概率生成的持久结构特征。在受控和中介情境中,LLM生成的语言表现出比人类写作更高的结构规律性和可压缩性,这与输出集中在高度重复的统计模式中一致。然而,这种特征显示出尺度依赖性:在碎片化的互动环境中,分离程度减弱,表明在小尺度下表面可区分性的根本限制。基于可压缩性的分离在各种模型、任务和领域中始终一致,并且可以直接从表面文本观察,而无需依赖模型内部或语义评估。总体而言,我们的发现介绍了一种简单且稳健的框架,用于量化生成系统如何重塑文本生产,为通信日益复杂性提供了结构视角。

英文摘要:

Large language models generate text through probabilistic sampling from high-dimensional distributions, yet how this process reshapes the structural statistical organization of language remains incompletely characterized. Here we show that lossless compression provides a simple, model-agnostic measure of statistical regularity that differentiates generative regimes directly from surface text. We analyze compression behavior across three progressively more complex information ecosystems: controlled human-LLM continuations, generative mediation of a knowledge infrastructure (Wikipedia vs. Grokipedia), and fully synthetic social interaction environments (Moltbook vs. Reddit). Across settings, compression reveals a persistent structural signature of probabilistic generation. In controlled and mediated contexts, LLM-produced language exhibits higher structural regularity and compressibility than human-written text, consistent with a concentration of output within highly recurrent statistical patterns. However, this signature shows scale dependence: in fragmented interaction environments the separation attenuates, suggesting a fundamental limit to surface-level distinguishability at small scales. This compressibility-based separation emerges consistently across models, tasks, and domains and can be observed directly from surface text without relying on model internals or semantic evaluation. Overall, our findings introduce a simple and robust framework for quantifying how generative systems reshape textual production, offering a structural perspective on the evolving complexity of communication.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑