arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.18468cs.CYcs.CL
收藏

算法无意识:大型语言模型中的结构机制与隐性偏见

The Algorithmic Unconscious: Structural Mechanisms and Implicit Biases in Large Language Models

Philippe Boisnard

AI总结:

本文探讨了大型语言模型中结构性机制与隐性偏见,揭示了分词、注意力等机制对偏见的影响,并提出技术诊所框架以促进AI基础设施的批判性利用。

摘要:

本文引入了"算法无意识"的概念,用以指代在大型语言模型(LLMs)中运作但无法被模型自身或其用户所触及的结构性决定因素。与将AI偏见仅归因于数据集组成或人类意图性的投影的方法不同,我们认为一组显著的偏见直接源自模型本身的机制:分词、注意力、统计优化和对齐过程。通过将偏见视为基础设施现象,这种方法解决了围绕当代LLMs责任、中立性和修正的理论模糊性。基于对平行句子语料库中分词的比较分析,我们表明阿拉伯语(现代标准阿拉伯语和马格里布方言)相对于英语的分词数量存在系统性膨胀,比例范围从1.6倍到接近4倍,具体取决于基础设施(OpenAI、Anthropic、SentencePiece/Mistral)。这种过度分词构成了一种可测量的基础设施偏见,机械地增加了推理成本,限制了上下文空间的访问,并改变了模型表示中的注意力加权。我们还将这些实证发现与三个其他结构性机制联系起来:因果偏见(相关性与因果性)、通过维度坍缩消除边缘化特征以及由安全对齐诱导的规范性偏见。最后,我们提出了一种基于分词制度审计、潜在空间拓扑和对齐系统的模型技术诊所框架,作为对AI基础设施批判性利用的必要条件。

英文摘要:

This article introduces the concept of the algorithmic unconscious to designate the set of structural determinations that operate within large language models (LLMs) without being accessible either to the model's own reflexivity or to that of its users. In contrast to approaches that reduce AI bias solely to dataset composition or to the projection of human intentionality, we argue that a significant class of biases emerges directly from the technical mechanisms of the models themselves: tokenization, attention, statistical optimization, and alignment procedures. By framing bias as an infrastructural phenomenon, this approach resolves a central theoretical ambiguity surrounding responsibility, neutrality, and correction in contemporary LLMs. Based on a comparative analysis of tokenization across a corpus of parallel sentences, we show that Arabic languages (Modern Standard Arabic and Maghrebi dialects) undergo a systematic inflation in token count relative to English, with ratios ranging from 1.6x to nearly 4x depending on the infrastructure (OpenAI, Anthropic, SentencePiece/Mistral). This over-segmentation constitutes a measurable infrastructural bias that mechanically increases inference costs, constrains access to contextual space, and alters attentional weighting within model representations. We relate these empirical findings to three additional structural mechanisms: causal bias (correlation vs causation), the erasure of minoritized features through dimensional collapse, and normative biases induced by safety alignment. Finally, we propose a framework for a technical clinic of models, grounded in the audit of tokenization regimes, latent space topology, and alignment systems, as a necessary condition for the critical appropriation of AI infrastructures.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑