高效时间动作分割：基于边界感知查询投票
Efficient Temporal Action Segmentation via Boundary-aware Query Voting
AI总结：
针对时间动作分割计算成本高的问题，提出边界感知Transformer网络BaFormer，通过实例查询和全局查询实现高效分割，仅用DiffAct 6%的运行时间达到相当或更优精度。
摘要：
尽管时间动作分割（TAS）的性能近年来有所提升，但取得令人满意的结果往往伴随着高昂的计算成本，这源于密集输入、复杂的模型结构以及资源密集型的后处理需求。为了在保持性能的同时提高效率，我们提出了一种以逐段分类为核心的新视角。通过利用Transformer的能力，我们将每个视频段标记为实例令牌，并赋予其固有的实例分割属性。为实现高效的动作分割，我们引入了BaFormer，一种边界感知的Transformer网络。它采用实例查询进行实例分割，并使用全局查询进行类别无关的边界预测，从而产生连续的段提议。在推理阶段，BaFormer采用一种简单而有效的投票策略，基于实例分割对边界段进行分类。值得注意的是，作为一种单阶段方法，BaFormer显著降低了计算成本，仅使用最先进方法DiffAct运行时间的6%，同时在多个流行基准上取得了更好或相当的性能。该项目的代码可在https://github.com/peiyao-w/BaFormer公开获取。
英文摘要：
Although the performance of Temporal Action Segmentation (TAS) has improved in recent years, achieving promising results often comes with a high computational cost due to dense inputs, complex model structures, and resource-intensive post-processing requirements. To improve the efficiency while keeping the performance, we present a novel perspective centered on per-segment classification. By harnessing the capabilities of Transformers, we tokenize each video segment as an instance token, endowed with intrinsic instance segmentation. To realize efficient action segmentation, we introduce BaFormer, a boundary-aware Transformer network. It employs instance queries for instance segmentation and a global query for class-agnostic boundary prediction, yielding continuous segment proposals. During inference, BaFormer employs a simple yet effective voting strategy to classify boundary-wise segments based on instance segmentation. Remarkably, as a single-stage approach, BaFormer significantly reduces the computational costs, utilizing only 6% of the running time compared to state-of-the-art method DiffAct, while producing better or comparable accuracy over several popular benchmarks. The code for this project is publicly available at https://github.com/peiyao-w/BaFormer.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
