DecodeShare：追踪大语言模型解码时刻决策的共享子空间
DecodeShare: Tracing the Shared Subspace of LLM Decode-Time Decisions
浏览论文内容
中文总结 AI 辅助
研究大语言模型解码时任务通用结构，提出DecodeShare协议识别共享子空间并测试其因果作用，实验显示干扰该子空间对决策性能影响大，且其对激活引导有实际意义，能分离组件功能、提供可靠信号。
中文摘要 AI 辅助
大语言模型用一组参数处理多项任务，在KV缓存推理中，不清楚解码时而非预填充时使用何种任务通用结构。我们提出DecodeShare协议，它能识别解码时隐藏状态中跨任务一致共享的低维子空间，并通过仅在解码时移除该子空间来测试其因果作用。实验表明，干扰发现的共享子空间对决策性能的影响远大于干扰预填充派生或随机子空间。此外，该共享子空间对激活引导有实际影响，能分离两个组件的功能作用，为下游部署提供更可靠信号。
英文摘要
Large language models (LLMs) handle many tasks with one set of parameters, but under KV-cached inference it is unclear what task-general structure, if any, is used at decode time rather than during prefill. We propose DecodeShare, a protocol that identifies a low-dimensional subspace consistently shared across tasks in decode-time hidden states, and then tests its causal role by removing that subspace only during decoding. In our experiments, disturbing the discovered shared subspace degrades decision performance far more than disturbing either a prefill-derived or random subspace under the same intervention budget. We further show this decode-shared subspace has practical consequences for activation steering: common steering directions can overlap the task-general decode channel. Projecting out this shared subspace directly separates the functional roles of the two components, while evaluating steering vectors at decode-time yields more reliable signal for downstream deployment than prefill-based proxies. Despite its compactness, the shared subspace can serve as a high-leverage causal channel at decode time. Code is available at: https://github.com/Zishan-Shao/decodeshare.git.
