粒度轴:语言模型中社会角色的微到宏观潜在方向
The Granularity Axis: A Micro-to-Macro Latent Direction for Social Roles in Language Models
AI总结:
研究揭示语言模型中社会角色粒度的潜在方向,通过定义粒度轴,发现其主导了角色表示空间的几何结构,并通过实验验证其在不同层级和模型间的稳定性与因果相关性。
摘要:
大型语言模型(LLMs)常被提示扮演从个体到机构的社会角色,但其内部表示是否编码角色粒度尚不明确。本文定义了基于对比的粒度轴,即宏观与微观角色隐藏状态的差异。在Qwen3-8B中,该轴与角色表示空间的主轴(PC1)在余弦相似度0.972下对齐,解释了52.6%的方差。通过构建75个社会角色并收集91,200个条件响应,提取角色隐藏状态并投影至轴上,发现角色投影在所有层级上单调增加,且在不同层、提示变体、端点定义、留出分割和评分过滤子集上保持稳定,并可迁移至Llama-3.1-8B-Instruct。该轴还具有因果相关性:沿轴激活引导可使响应粒度按预测方向变化,如Llama在五点宏观尺度上从2.00增至3.17。两个模型在可控性上存在差异,表明引导依赖于各模型的默认操作模式。总体而言,研究发现社会角色粒度不仅是表面特征,而是结构化、有序且可因果操控的潜在方向。
英文摘要:
Large language models (LLMs) are prompted to adopt social roles from individuals to institutions, but whether their representations encode this difference in social granularity remains unclear. We discover a contrast-based Granularity Axis, the difference between mean macro- and micro-role hidden states, that aligns with the dominant geometry of role space: in Qwen3-8B, its cosine with PC1 is $0.972$, and PC1 explains $52.6\%$ of the variance. Across 75 roles at five levels, with 90,000 role-conditioned and 1,200 default responses per model, projections increase monotonically and remain stable across layers, prompts, and response-quality filters. The structure transfers to Llama-3.1-8B-Instruct and scales to Qwen3-32B and Llama-3.1-70B-Instruct, whose PC1 shares are $45.99\%$ and $31.23\%$. Independent five-rater validation ($α=0.961$) preserves the ordering; identical continuations and affect/formality controls confirm scale-related variation beyond generated wording. The axis is also behaviorally causal: positive steering moves Llama from $2.00$ to $3.17$ on a five-point macro scale for prompts admitting local responses. Steering strength and stability depend on the model's default operating regime. Together, these findings establish social granularity as a representational primitive: a continuous, ordered, causally manipulable direction organizing role-conditioned generation across model families.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。