批归一化在端到端视频学习中的陷阱：手术工作流分析研究
On the Pitfalls of Batch Normalization for End-to-End Video Learning: A Study on Surgical Workflow Analysis
AI总结：
本文揭示批归一化在端到端视频学习中的陷阱，并证明无BN的CNN-LSTM通过优化时序上下文训练策略，在手术工作流基准上达到最优性能。
摘要：
批归一化（BN）依赖于批次中其他样本的独特性质，已知会在包括序列建模在内的多个任务中引发问题。然而，尽管批归一化在卷积神经网络（CNN）中广泛用于特征提取，但在长视频理解中，与批归一化相关的问题却鲜有研究。尤其是在手术工作流分析中，由于缺乏预训练的特征提取器，导致采用了复杂的多阶段训练流程，对批归一化问题的认识不足可能掩盖了端到端训练CNN和时序模型的优势。本文分析了视频学习中批归一化的陷阱，包括针对在线任务的特有问题，如预测中的“作弊”效应。我们观察到，批归一化的性质对端到端学习构成了重大障碍。然而，使用无批归一化的骨干网络，即使简单的CNN-LSTM，通过利用充分最大化时序上下文的端到端训练策略，也能在三个手术工作流基准上超越现有技术水平。我们得出结论，认识批归一化的陷阱对于手术任务中的有效端到端学习至关重要。通过在自然视频数据集上复现结果，我们希望我们的见解也能惠及视频学习的其他领域。代码可在以下网址获取：https://gitlab.com/nct_tso_public/pitfalls_bn
英文摘要：
Batch Normalization's (BN) unique property of depending on other samples in a batch is known to cause problems in several tasks, including sequence modeling. Yet, BN-related issues are hardly studied for long video understanding, despite the ubiquitous use of BN in CNNs (Convolutional Neural Networks) for feature extraction. Especially in surgical workflow analysis, where the lack of pretrained feature extractors has led to complex, multi-stage training pipelines, limited awareness of BN issues may have hidden the benefits of training CNNs and temporal models end to end. In this paper, we analyze pitfalls of BN in video learning, including issues specific to online tasks such as a 'cheating' effect in anticipation. We observe that BN's properties create major obstacles for end-to-end learning. However, using BN-free backbones, even simple CNN-LSTMs beat the state of the art {\color{\colorrevtwo}on three surgical workflow benchmarks} by utilizing adequate end-to-end training strategies which maximize temporal context. We conclude that awareness of BN's pitfalls is crucial for effective end-to-end learning in surgical tasks. By reproducing results on natural-video datasets, we hope our insights will benefit other areas of video learning as well. Code is available at: \url{https://gitlab.com/nct_tso_public/pitfalls_bn}
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
