比特流动作识别即字节建模
Bitstream Action Recognition is Byte Modeling
浏览论文内容
中文总结 AI 辅助
该研究针对比特流损坏导致动作识别失效的问题，提出BRACE框架，结合RBCS构建首个大规模BAR数据集，实验表明其鲁棒性优于对比方法。
中文摘要 AI 辅助
传统动作识别通常依赖于比特流的成功像素解码。然而，比特流在存储或传输过程中出现损坏可能会导致严重的视觉伪影甚至解码失败，给可靠的动作识别带来重大挑战。比特流动作识别（BAR）旨在克服对解码的依赖以及对损坏的脆弱性。本文提出了一种新颖的BAR框架：基于锚定损坏嵌入的比特流识别（BRACE）。BRACE是一种双分支字节建模架构，将损坏的比特流及其完整对应物视为同一动作的两种字节实现，这通过完整锚定表示对齐（IARA）引导生成丰富且稳定的表示，以实现对损坏的鲁棒性。完整表示作为稳定锚，损坏表示在嵌入和决策级别通过不可靠锚抑制（UAS）与完整表示对齐，整个过程在表示空间中进行，无需修复比特流。为解决实际中损坏比特流稀缺的问题，我们引入了真实世界比特流损坏模拟器（RBCS），这是一个四参数模拟器，可重现传输和存储中出现的位翻转和字节丢失错误。基于RBCS，我们构建了第一个大规模BAR数据集（BAR-D），包含BAR-Stanford40和BAR-PPMI子集，涵盖不同的损坏类型和严重程度。最后，我们在BAR-D上构建了一个大型基准，涉及来自像素、压缩和比特流领域的14种动作识别方法。大量实验表明，BRACE对比特流损坏的鲁棒性优于所有对比方法。消融研究进一步验证了所提出的RBCS增强和IARA的有效性。
英文摘要
Conventional action recognition typically relies on successful pixel decoding of the bitstream. However, bitstream corruption during storage or transmission may cause severe visual artifacts or even decoding failure, posing a significant challenge to reliable action recognition. Bitstream Action Recognition (BAR) aims to overcome the dependency on decoding and the vulnerability to corruption. In this paper, we propose a novel BAR framework, Bitstream Recognition via Anchoring Corrupted Embeddings (BRACE). BRACE is a dual-branch byte-modeling architecture that treats a corrupted bitstream and its intact counterpart as two byte realizations of the same action. This guides the generation of rich and stable representations for robustness to corruption through Intact-Anchored Representation Alignment (IARA). The intact representation serves as a stable anchor, and the corrupted one is aligned to it at the embedding and decision levels under Unreliable-Anchor Suppression (UAS), entirely in representation space and without repairing the bitstream. To address the scarcity of corrupted bitstreams in practice, we introduce the Real-world Bitstream Corruption Simulator (RBCS), a four-parameter simulator that reproduces bit-flip and byte-loss errors arising in transmission and storage. Building on RBCS, we construct the first large-scale BAR dataset (BAR-D), which comprises the BAR-Stanford40 and BAR-PPMI subsets and spans diverse corruption types and severity levels. Finally, we build a large benchmark on BAR-D involving 14 action recognition methods from the pixel, compressed, and bitstream domains. Extensive experiments demonstrate that BRACE has superior robustness to bitstream corruption than all comparison methods. Ablation studies further validate the effectiveness of the proposed RBCS augmentation and IARA.
发表机构
- School of Electronic Information and Communications, Huazhong University of Science and Technology（华中科技大学电子信息与通信学院）
- School of Electrical and Electronics Engineering, Nanyang Technological University（南洋理工大学电气与电子工程学院）
- Institute for Infocomm Research, Agency for Science, Technology and Research (A*STAR)（新加坡科技研究局信息通信研究院）
机构由 AI 辅助整理，请以论文原文为准。
