TCCT-Net：基于行为特征信号实现快速高效参与度估计的双流网络架构
TCCT-Net: Two-Stream Network Architecture for Fast and Efficient Engagement Estimation via Behavioral Feature Signals
AI总结：
针对现有参与度估计深度模型复杂度高、难实时部署的问题，提出融合时空CT流与时频TC流的TCCT-Net架构，仅用2种头部姿态特征在EngageNet上超基线，推理速度较图像RNN方法提升一个数量级。
摘要：
参与度分析在医疗、教育、广告、服务领域有着广泛应用。用于此类分析的深度神经网络架构复杂，需要大量输入数据、算力与推理时间。这些限制给将系统嵌入设备实现实时应用带来了挑战。为解决这些局限，我们提出了一种新颖的双流特征融合“张量-卷积与卷积-Transformer网络”（TCCT-Net）架构。为更好地学习时空域中的有效模式，我们设计了融合卷积-Transformer混合结构的“CT”流。同时，为高效提取时频域的丰富模式并提升处理速度，我们引入了“TC”流，该流采用连续小波变换（CWT）将信息表示为二维张量形式。在EngageNet数据集上的评估结果显示，所提方法仅使用两种行为特征（头部姿态旋转角度），性能优于现有基线模型，而基线模型需用到98种特征。此外，对比分析表明，与当前最优的基于图像的循环神经网络（RNN）方法相比，TCCT-Net的架构在推理速度上实现了数量级的提升。代码将发布于https://github.com/vedernikovphoto/TCCT_Net。
英文摘要：
Engagement analysis finds various applications in healthcare, education, advertisement, services. Deep Neural Networks, used for analysis, possess complex architecture and need large amounts of input data, computational power, inference time. These constraints challenge embedding systems into devices for real-time use. To address these limitations, we present a novel two-stream feature fusion "Tensor-Convolution and Convolution-Transformer Network" (TCCT-Net) architecture. To better learn the meaningful patterns in the temporal-spatial domain, we design a "CT" stream that integrates a hybrid convolutional-transformer. In parallel, to efficiently extract rich patterns from the temporal-frequency domain and boost processing speed, we introduce a "TC" stream that uses Continuous Wavelet Transform (CWT) to represent information in a 2D tensor form. Evaluated on the EngageNet dataset, the proposed method outperforms existing baselines, utilizing only two behavioral features (head pose rotations) compared to the 98 used in baseline models. Furthermore, comparative analysis shows TCCT-Net's architecture offers an order-of-magnitude improvement in inference speed compared to state-of-the-art image-based Recurrent Neural Network (RNN) methods. The code will be released at https://github.com/vedernikovphoto/TCCT_Net.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
