TD3Net: 一种用于唇读的时序密集连接多稀疏卷积网络
TD3Net: A temporal densely connected multi-dilated convolutional network for lipreading
AI总结：
TD3Net通过结合密集连接和多稀疏卷积，提升唇读任务的性能和效率
摘要：
在单词级唇读方法中，通常采用两阶段框架，包含独立的前端和后端架构来建模动态唇部运动。每个组件都已得到广泛研究，在后端架构中，时间卷积网络（TCNs）已被广泛应用于最先进的方法中。最近，dense skip connections被引入TCNs以缓解接收场密度有限的问题，从而改进复杂时间表示的建模。然而，其性能仍受到限制，因为潜在的信息丢失可能发生在唇部运动连续性方面，由于接收场中的盲区。为了解决这一限制，我们提出了TD3Net，一种结合dense skip connections和多稀疏时间卷积的时序密集连接多稀疏卷积网络，作为后端架构。TD3Net通过应用不同的扩张因子到skip-connected特征上，覆盖了宽而密集的接收场，没有盲区。在使用两个大型公开可用数据集，即Lip Reading in the Wild (LRW)和LRW-1000的单词级唇读任务中，实验结果表明，所提出的方法在性能上与最先进的方法相当。它在参数数量更少和浮点运算更低的情况下实现了更高的准确性，相比现有的基于TCN的后端架构。此外，可视化结果表明，我们的方法有效地利用了多样的时间特征，同时保持了时间连续性，在唇读系统中具有显著优势。代码可在我们的GitHub仓库（https://github.com/Leebh-kor/TD3Net）中获得。
英文摘要：
The word-level lipreading approach typically employs a two-stage framework with separate frontend and backend architectures to model dynamic lip movements. Each component has been extensively studied, and in the backend architecture, temporal convolutional networks (TCNs) have been widely adopted in state-of-the-art methods. Recently, dense skip connections have been introduced in TCNs to mitigate the limited density of the receptive field, thereby improving the modeling of complex temporal representations. However, their performance remains constrained owing to potential information loss regarding the continuous nature of lip movements, caused by blind spots in the receptive field. To address this limitation, we propose TD3Net, a temporal densely connected multi-dilated convolutional network that combines dense skip connections and multi-dilated temporal convolutions as the backend architecture. TD3Net covers a wide and dense receptive field without blind spots by applying different dilation factors to skip-connected features. Experimental results on a word-level lipreading task using two large publicly available datasets, Lip Reading in the Wild (LRW) and LRW-1000, indicate that the proposed method achieves performance comparable to state-of-the-art methods. It achieved higher accuracy with fewer parameters and lower floating-point operations compared to existing TCN-based backend architectures. Moreover, visualization results suggest that our approach effectively utilizes diverse temporal features while preserving temporal continuity, presenting notable advantages in lipreading systems. The code is available at our GitHub repository (https://github.com/Leebh-kor/TD3Net).
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
