Department of Computer Science, Sapienza University of Rome(计算机科学系,罗马萨皮恩扎大学);Department of Computer, Control and Management Engineering, Sapienza University of Rome(计算机、控制与管理工程系,罗马萨皮恩扎大学);Department of Legal, Social, and Educational Sciences, Tuscia University(法律、社会与教育科学系,图斯西亚大学)
Ortal Hadad, Edoardo Loru, Jacopo Nudo, Niccolò Di Marco, Matteo Cinelli, Walter Quattrociocchi
大型语言模型通过从高维分布中进行概率采样来生成文本,但这一过程如何重塑语言的结构统计组织仍不完全明确。在这里,我们展示出无损压缩提供了一种简单的、对模型不敏感的统计规律度量方法,可以直接从表面文本区分生成模式。我们分析了压缩行为在三个渐进复杂的信息生态系统中的表现:受控的人类-LLM延续、知识基础设施的生成中介(维基百科 vs. Grokipedia)以及完全合成的社会互动环境(Moltbook vs. Reddit)。在各种情境中,压缩揭示出概率生成的持久结构特征。在受控和中介情境中,LLM生成的语言表现出比人类写作更高的结构规律性和可压缩性,这与输出集中在高度重复的统计模式中一致。然而,这种特征显示出尺度依赖性:在碎片化的互动环境中,分离程度减弱,表明在小尺度下表面可区分性的根本限制。基于可压缩性的分离在各种模型、任务和领域中始终一致,并且可以直接从表面文本观察,而无需依赖模型内部或语义评估。总体而言,我们的发现介绍了一种简单且稳健的框架,用于量化生成系统如何重塑文本生产,为通信日益复杂性提供了结构视角。
英文摘要:
Large language models generate text through probabilistic sampling from high-dimensional distributions, yet how this process reshapes the structural statistical organization of language remains incompletely characterized. Here we show that lossless compression provides a simple, model-agnostic measure of statistical regularity that differentiates generative regimes directly from surface text. We analyze compression behavior across three progressively more complex information ecosystems: controlled human-LLM continuations, generative mediation of a knowledge infrastructure (Wikipedia vs. Grokipedia), and fully synthetic social interaction environments (Moltbook vs. Reddit). Across settings, compression reveals a persistent structural signature of probabilistic generation. In controlled and mediated contexts, LLM-produced language exhibits higher structural regularity and compressibility than human-written text, consistent with a concentration of output within highly recurrent statistical patterns. However, this signature shows scale dependence: in fragmented interaction environments the separation attenuates, suggesting a fundamental limit to surface-level distinguishability at small scales. This compressibility-based separation emerges consistently across models, tasks, and domains and can be observed directly from surface text without relying on model internals or semantic evaluation. Overall, our findings introduce a simple and robust framework for quantifying how generative systems reshape textual production, offering a structural perspective on the evolving complexity of communication.
Rice University(莱斯大学);Dartmouth College(达特茅斯学院);Georgia Institute of Technology(佐治亚理工学院);University of California, Berkeley(加州大学伯克利分校);Lawrence Berkeley National Laboratory(劳伦斯伯克利国家实验室)
StatABench: Dataset and Framework for Evaluating Statistical Analysis Capabilities of LLMs
Southern University of Science and Technology(南方科技大学);Alibaba Group(阿里巴巴集团);The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));The University of Hong Kong(香港大学)
Measuring Behavioural Signatures of Large Language Models through Psychometric Profiling
The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));Central China Normal University(华中师范大学);Shandong Science and Technology Press, Shandong Publishing Group(山东科学技术出版社,山东出版集团);Bielefeld University(比勒费尔德大学);South China Normal University(华南师范大学);Frankfurt Institute for Advanced Studies(法兰克福高等研究院)
A Comparative Analysis of LLM Memorization at Statistical and Internal Levels: Cross-Model Commonalities and Model-Specific Signatures
Department of Computer Science, The University of Tokyo(东京大学计算机科学系);Research and Development Center for Large Language Models, National Institute of Informatics(信息学研究院大语言模型研究与开发中心)