可解释表征地标的认证近似
Certified Approximation for Interpretable Representer Landmarks
浏览论文内容
中文总结 AI 辅助
CAIRN框架对表征解释中的核近似误差进行端到端认证，推导精确方差并给出前K集合的高概率保证，同时通过残差贪心选择显著提升类别覆盖度，使可靠性可量化。
中文摘要 AI 辅助
表征解释（Representer explanations）对自监督表征影响最大的训练地标进行排序。在大规模场景下，该排序依赖于对经验神经正切核（eNTK）的多达四层堆叠近似，包括随机输出头、参数草图、地标采样和系数拟合。现有分析分别对每个近似进行界定，但没有任何方法能针对它们的组合误差认证前$K$个集合。我们提出了CAIRN（可解释表征地标的认证近似）框架，将该误差传递到排序中。我们推导了草图化多头eNTK的精确方差，与测量结果匹配在$4\%$以内，而Johnson-Lindenstrauss界的误差高达$2.5$倍。这为固定系数拟合提供了高概率的前$K$认证，同时为丢弃的谱质量提供了精确的残差迹认证。一个精确的乘积方差恒等式将核误差与拟合变异性分离，并识别出何时更大的核预算仍能锐化排序。随机Lanczos求积（SLQ）在$0.72\%$内估计有效维度，并在无需密集特征分解的情况下指导地标预算。我们表明残差质量不能控制类别覆盖度，残差贪心选择将$k$-means++的最坏覆盖度超额从$8.5$倍降至$1.55$倍（在草图化eNTK上为$4$倍）。跨视图初始化器在五个（AUI）到全部六个（CSI）设置中优于主成分初始化。与KREPES Gauss-Newton求解器相比，CAIRN收敛速度快$2.5$至$11.3$倍，在MNIST上最多落后$0.31$分，最多提升$3.14$分。综合来看，这些结果使表征解释的可靠性变得可测量，并展示了近似预算的最佳投入方向。
英文摘要
Representer explanations rank the training landmarks that most influence a self-supervised representation. At scale, this ranking rests on up to four stacked approximations of the empirical neural tangent kernel (eNTK). These are random output heads, a parameter sketch, landmark sampling and a coefficient fit. Existing analyses bound each approximation separately, but none certifies the top-$K$ set against their combined error. We introduce CAIRN (Certified Approximation for Interpretable Representer laNdmarks), a framework that carries this error through to the ranking. We derive the exact variance of the sketched multi-head eNTK, which matches measurement within $4\%$ where Johnson-Lindenstrauss bounds err by up to $2.5\times$. This yields a high-probability top-$K$ certificate for a fixed coefficient fit, alongside exact residual-trace certificates for discarded spectral mass. An exact product-variance identity separates kernel error from fit variability and identifies when a larger kernel budget can still sharpen a ranking. Stochastic Lanczos Quadrature (SLQ) estimates the effective dimension within $0.72\%$ and guides the landmark budget without dense eigendecomposition. We show that residual mass does not control class coverage, and residual-greedy selection cuts the worst coverage excess of $k$-means++ from $8.5\times$ to $1.55\times$ ($4\times$ on the sketched eNTK). Cross-view initializers outperform principal-component initialization in five (AUI) to all six (CSI) settings. Against the KREPES Gauss-Newton solver, CAIRN converges $2.5$ to $11.3\times$ faster, trails by at most $0.31$ points and gains up to $3.14$ points on MNIST. Together, these results make the reliability of representer explanations measurable and show where approximation budgets are best spent.
发表机构
- Purdue University（普渡大学）
机构由 AI 辅助整理，请以论文原文为准。
