基于频谱对齐的通用纠错码变换器剪枝
Spectral-Aligned Pruning for Universal Error-Correcting Code Transformers
AI总结：
本文提出SAP框架，通过频谱分析实现跨码本结构剪枝，结合低秩适应降低计算和内存开销，实现高效纠错解码。
摘要：
基于变换器的通用通道解码器，如基础纠错码变换器（FECCT），通过共享骨干网络在不同码族中实现竞争性解码性能，可选地通过代码特定微调。然而，FECCT的高计算复杂度和大参数足迹对实际部署构成重大障碍。为解决这些问题，我们研究了FECCT的结构剪枝，并提出了Spectral-Aligned Pruning（SAP），一种结构感知框架，通过利用相应双分图的频谱实现跨码本结构剪枝掩码的重用。SAP基于经典的代码图分析：两个代数最大的邻接特征值提供紧凑的频谱代理，用于度量尺度、扩展比和最小距离下界。这些量直接相关于解码性能：度量尺度反映编码词位和奇偶校验位的连接密度；扩展比影响信息在双分图中的传播；最小距离表征编码词分离。基于此连接，SAP使用这两个主导特征值作为轻量级代码签名用于剪枝掩码检索。经验上，这种二维签名在我们的评估中与更高维频谱签名具有同等稳定性。剪枝后，SAP通过参数高效的低秩适应（LoRA）进行每码恢复，实现共享剪枝骨干的同时仅存储少量代码特定适配器参数。在多样化的码上实验显示，SAP在解码性能上与专门的每码剪枝相当，同时通过核级结构剪枝显著降低计算成本和模型内存足迹。
英文摘要：
Universal channel decoders based on transformers-such as the Foundation Error Correction Code Transformer (FECCT)-achieve competitive decoding performance across diverse code families with a single shared backbone, optionally followed by code-specific finetuning. However, the high computational complexity and large parameter footprint of FECCT present substantial obstacles to practical deployment. To address these challenges, we investigate structured pruning for FECCT and propose Spectral-Aligned Pruning (SAP), a structure-aware framework that enables cross-code reuse of structured pruning masks by leveraging the spectrum of the corresponding bipartite graph. SAP is grounded in classical graph analysis of codes: the two algebraically largest adjacency eigenvalues provide compact spectral proxies for degree scale, expansion ratio, and minimum-distance lower bounds. These quantities are directly relevant to decoding performance: degree scale reflects how densely codeword bits and parity checks are connected; expansion ratio influences how information propagates across the bipartite graph; and minimum distance characterizes codeword separation. Based on this connection, SAP uses these two leading eigenvalues as a lightweight code signature for pruning-mask retrieval. Empirically, this two-dimensional signature yields stable library selection equivalent to higher-dimensional spectral signatures in our evaluation. After pruning, SAP uses parameter-efficient low-rank adaptation (LoRA) to recover decoding performance while storing only a small number of additional adapter parameters per code. Experiments across diverse codes show that SAP achieves decoding performance comparable to dedicated per-code pruning, while enabling substantial reductions in computational cost and model memory footprint through kernel-level structured pruning. Code is available at https://github.com/iil-postech/sap.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
