CASR:通过边缘化帧级因果关系优化动作分割
CASR: Refining Action Segmentation via Marginalizing Frame-levle Causal Relationships
AI总结:
针对帧级因果关系噪声多、难表达宏观动作语义的问题,提出CASR方法,通过边缘化帧级因果关系构建段级因果邻接矩阵以优化TAS结果,还提出CED评估指标,实验显示其在分割性能、因果可解释性与泛化性上均优于现有方法。
摘要:
深度学习与因果发现的结合提升了时间动作分割(TAS)任务的可解释性。然而,帧级因果关系存在大量段级之外的复杂噪声,无法直接表达宏观动作语义。为此,我们提出因果抽象分割优化器(Causal Abstraction Segmentation Refiner, CASR),它能通过边缘化帧级因果关系增强视频因果性,优化多种模型的TAS结果。具体而言,我们定义了等价的帧级因果模型与段级因果模型,使由边缘化帧级因果关系构建的因果邻接矩阵具备表征段级因果关系的能力。CASR通过缩小我们构建的因果邻接矩阵与骨干模型预分割结果之间的差异来实现优化。此外,我们提出了一种新的评估指标因果编辑距离(Causal Edit Distance, CED),用于评估因果可解释性。在主流数据集上的大量实验结果表明,CASR在动作分割性能、因果可解释性以及泛化性方面均显著优于现有各类方法。
英文摘要:
Integrating deep learning and causal discovery has increased the interpretability of Temporal Action Segmentation (TAS) tasks. However, frame-level causal relationships exist many complicated noises outside the segment-level, making it infeasible to directly express macro action semantics. Thus, we propose Causal Abstraction Segmentation Refiner (CASR), which can refine TAS results from various models by enhancing video causality in marginalizing frame-level casual relationships. Specifically, we define the equivalent frame-level casual model and segment-level causal model, so that the causal adjacency matrix constructed from marginalized frame-level causal relationships has the ability to represent the segmnet-level causal relationships. CASR works out by reducing the difference in the causal adjacency matrix between we constructed and pre-segmentation results of backbone models. In addition, we propose a novel evaluation metric Causal Edit Distance (CED) to evaluate the causal interpretability. Extensive experimental results on mainstream datasets indicate that CASR significantly surpasses existing various methods in action segmentation performance, as well as in causal explainability and generalization.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。