块稀疏特征化器（BSF）的深入分析
A Deeper Analysis of Block-Sparse Featurizers
- Columbia University（哥伦比亚大学）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
本文深入分析块稀疏特征化器（BSF）的优缺点，针对其存在的经典稀疏自编码器失效模式，提出锦标赛Top-K选择规则等架构改进，并将块范式扩展至跨编码器。
AI中文摘要：
近期提出的块稀疏特征化器（BSF；Fel等人，2026）与稀疏自编码器（SAE）类似，但其原子单元是小子空间（方向块）而非单个方向，专为低维流形上的特征设计，这类特征在视觉领域尤为常见。本研究分析BSF的优缺点，发现其仍存在经典SAE的失效模式，如特征分裂与组合。我们对BSF提出多项架构改进，包括锦标赛Top-K选择规则以显著减少特征分裂，还将块范式扩展至跨编码器（crosscoder）。
英文摘要：
The recently introduced block-sparse featurizer (BSF; Fel et al., 2026) is similar to a sparse autoencoder (SAE), but its atomic unit is a small subspace (a block of directions) rather than a single direction. It is designed for features that live on low-dimensional manifolds, which are especially frequent in vision. This work studies the BSF's strengths and weaknesses, finding how it still somewhat suffers from classic SAE failure modes, like feature splitting and composition. We propose several architectural changes to the BSF, including a Tournament Top-K selection rule that significantly reduces feature splitting, and we also extend the block paradigm to the crosscoder.
