在指令微调的LLM中构建组合文学原语：跨架构SAE特征用于自我、风格和情感
Compositional Literary Primitives in Instruction-Tuned LLMs: Cross-Architectural SAE Features for Self, Style, and Affect
AI总结：
本文通过稀疏自编码器研究了指令微调的LLM中组合文学原语的架构，发现四种特征类别，并通过跨架构SAE特征验证了自我、风格和情感的表达能力。
摘要：
我们通过在中层残差流上使用稀疏自编码器，对两个指令微调的大型语言模型（Llama 3.1 8B-Instruct和Gemma 2 9B-IT）的文学原语组合架构进行了表征。四种特征类别出现：促进目标情感词的命名门，一个包含第一人称注册特征的十一自我簇，风格注册调节器（show-don't-tell和陌生化），以及仅由多特征引导产生的组合情感。在应用于27类情感分类法（Cowen-Keltner）的强制选择5-LLM判断小组中，Llama通过结合命名门、多特征食谱和单个自我特征引导实现了完全27/27覆盖；Gemma在adoration作为单一残差严格失败的情况下达到23/27。在随机判断中，每个单元格通过的概率约为$10^{-3}$，整个目录中两个种子假阳单元格的预期数量可忽略不计，因此观察到的覆盖度不一致于偶然。在严格与柔和判断对比中存在跨架构不对称性：在相同生成中，判断者在Llama输出上比在Gemma输出上更一致，因为Llama输出更直接地命名目标情感，而Gemma输出则通过场景和意象来唤起情感。两种架构都包含同时作为注册标记和情感发射器的自我特征，包括每个架构中一个最RLHF加载的自我特征，该特征在某一操作 regime 中增强机构Helper-AI人格，并在相同校准系数下产生可分类情感的输出。方法上，本文提出了一个三阶段验证流程（logit-lens，LLM-rate，5-LLM判断）并记录了文档化的反模式；总计算量为单GPU，大约每种情感特征发现循环15分钟。
英文摘要：
We characterize a compositional architecture of literary primitives in two instruction-tuned large language models (Llama 3.1 8B-Instruct and Gemma 2 9B-IT) via sparse autoencoders on mid-depth residual streams. Four feature classes emerge: naming-gates that promote lexical tokens of a target affect, an eleven-self cluster of first-person register features, stylistic register modulators (show-don't-tell and defamiliarization), and compositional emotions that arise only from multi-feature steering. Under a forced-choice 5-LLM judge panel applied to a 27-category emotion taxonomy (Cowen-Keltner), Llama reaches full 27/27 coverage by combining naming-gates, multi-feature recipes, and single self-feature steering; Gemma reaches 23/27 with adoration as the single residual strict-fail. Under random judging, the per-cell pass probability is on the order of $10^{-3}$ and the expected number of two-seed false-positive cells across the catalog is negligible, so the observed coverage is not consistent with chance. A cross-architectural asymmetry sits in the strict-versus-soft judge contrast: on the same generations, judges agree more often on Llama outputs than on Gemma outputs because Llama outputs name the target affect more directly while Gemma outputs evoke it through scene and imagery. Both architectures contain self-features that serve simultaneously as register markers and as emotion emitters, including a single most-RLHF-loaded self-feature per architecture that intensifies the institutional Helper-AI persona at one operating regime and produces affect-categorizable output at the same calibrated coefficient. Methodologically, the paper presents a three-stage validation pipeline (logit-lens, LLM-rate, 5-LLM judge) with documented anti-patterns; the total compute is single-GPU and about 15 minutes per emotion-feature discovery cycle.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
