发表机构
Zhejiang University（浙江大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文提出令牌参数化视角，设计轻量级四步编码器Braco，在极端压缩下实现高准确率与高效率，显著降低延迟和计算量。
AI 中文摘要
视觉令牌压缩对于提升视觉-语言模型的效率是有效的，但在极端压缩预算下，令牌剪枝可能破坏视觉接地，而学习的重采样器会增加参数数量、注意力成本和训练复杂度。我们通过令牌参数化的视角重新审视压缩，将（i）基变换和结构化截断（保留子空间/可压缩性）与（ii）坐标组织（优化和跨模态对齐）分离。这一视角产生了两个耦合的目标，即可压缩性和可学习性，我们将其形式化为统一的泛函。在这些目标的指导下，我们设计了Braco，一个轻量级的四步编码器，结合了变换基截断、与输入无关的基坐标嵌入、依赖于预算的正交重参数化以及来自轻量级池化的学习空间残差令牌。实验表明，在$23\times$--$64\times$压缩下，Braco形成了有利的经验准确率-效率前沿，并在$144\times$压缩下保持竞争力，达到95.2%的准确率，同时相对于未压缩的上限将预填充FLOPs减少了84.2%--86.7%。与先前方法相比，Braco在匹配或提高准确率的同时，实现了高达约36%的端到端加速，并使用$16.6\times$/$78.8\times$更低的压缩器延迟/FLOPs。
英文摘要
Visual-token compression is effective for improving the efficiency of vision-language models, but under extreme compression budgets, token pruning can break visual grounding while learned resamplers increase parameter count, attention cost, and training complexity. We revisit compression through a token parameterization lens, separating (i) basis transformation and structured truncation (retained subspace/compressibility) from (ii) coordinate organization (optimization and cross-modal alignment). This view yields two coupled objectives, compressibility and learnability, which we formalize as unified functionals. Guided by these objectives, we design Braco, a lightweight four-step coder that combines transform-basis truncation, input-independent basis-coordinate embeddings, budget-dependent orthogonal re-parameterization, and learned spatial residual tokens from lightweight pooling. Experiments show that Braco forms the favorable empirical accuracy-efficiency frontier under $23\times$--$64\times$ compression and remains competitive at $144\times$, reaching 95.2% accuracy while reducing prefill FLOPs by 84.2%--86.7% relative to the uncompressed upper bound. Against prior methods, Braco matches or improves accuracy while achieving up to approximately 36% end-to-end speedup and using $16.6\times$/$78.8\times$ lower compressor latency/FLOPs.
CommentsAccepted at NeurIPS 2026 (Spotlight). Code: https://github.com/zrrraa/Braco
