arXiv 2308.088062023-12-29cs.CV

自蒸馏正则化连接时序分类损失用于文本识别：一种简单而有效的方法

Self-distillation Regularized Connectionist Temporal Classification Loss for Text Recognition: A Simple Yet Effective Approach

Huawei Technologies Co., Ltd（华为技术有限公司）

Ziyin Zhang, Ning Lu, Minghui Liao, Yongshuai Huang, Cheng Li, Min Wang, Wei Peng

备注：: Ziyin Zhang and Ning Lu are co-first authors. Accepted by AAAI2024. Repo: https://github.com/zzyhlyoko/DCTC

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对CTC损失忽略字符级学习导致精度下降的问题，提出自蒸馏正则化CTC损失（DCTC），引入帧级正则化项并利用最大后验估计解决蒸馏不一致性，无需额外参数或数据，可提升文本识别准确率高达2.6%。

摘要：

文本识别方法正在快速发展。一些先进技术，如强大的模块、语言模型以及无监督和半监督学习方案，不断推动公共基准上的性能提升。然而，如何从损失函数的角度更好地优化文本识别模型的问题在很大程度上被忽视了。基于CTC的方法在实践中被广泛使用，因其在性能和推理速度之间取得了良好的平衡，但仍面临精度下降的问题。这是因为CTC损失强调对整个序列目标的优化，而忽略了对单个字符的学习。我们提出了一种针对基于CTC模型的自蒸馏方案来解决这一问题。它在CTC损失中引入了一个帧级正则化项以强调个体监督，并利用潜在对齐的最大后验估计来解决基于CTC模型之间蒸馏中出现的不一致性问题。我们将这种正则化的CTC损失称为蒸馏连接时序分类（DCTC）损失。DCTC损失无需额外模块，不需要额外参数、更长的推理延迟，也不需要额外的训练数据或阶段。在公共基准上的大量实验表明，DCTC可以在没有任何这些缺点的情况下，将文本识别模型的准确率提升高达2.6%。

英文摘要：

Text recognition methods are gaining rapid development. Some advanced techniques, e.g., powerful modules, language models, and un- and semi-supervised learning schemes, consecutively push the performance on public benchmarks forward. However, the problem of how to better optimize a text recognition model from the perspective of loss functions is largely overlooked. CTC-based methods, widely used in practice due to their good balance between performance and inference speed, still grapple with accuracy degradation. This is because CTC loss emphasizes the optimization of the entire sequence target while neglecting to learn individual characters. We propose a self-distillation scheme for CTC-based model to address this issue. It incorporates a framewise regularization term in CTC loss to emphasize individual supervision, and leverages the maximizing-a-posteriori of latent alignment to solve the inconsistency problem that arises in distillation between CTC-based models. We refer to the regularized CTC loss as Distillation Connectionist Temporal Classification (DCTC) loss. DCTC loss is module-free, requiring no extra parameters, longer inference lag, or additional training data or phases. Extensive experiments on public benchmarks demonstrate that DCTC can boost text recognition model accuracy by up to 2.6%, without any of these drawbacks.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。