MulCPred：学习多模态概念用于可解释行人动作预测
MulCPred: Learning Multi-modal Concepts for Explainable Pedestrian Action Prediction
AI总结：
针对现有行人动作预测方法缺乏可解释性的问题，本文提出MulCPred框架，通过线性聚合器、通道级重校准模块和特征正则化损失解决以往方法的局限，在多数据集上验证了其可解释性提升且性能下降不明显，还具备进一步泛化的潜力。
摘要：
行人动作预测在自动驾驶等诸多应用中具有重要意义，但当前最优方法缺乏可解释性，难以给出可信的预测结果。本文提出一种名为MulCPred的新型框架，它以训练样本表征的多模态概念为基础对预测结果进行解释。以往基于概念的方法存在以下局限：1）无法直接应用于多模态场景；2）缺乏局部性，无法关注输入中的细节；3）存在模式崩溃问题。本文通过以下方法针对性解决了这些局限：1）采用线性聚合器将概念的激活结果整合为预测结果，该聚合器关联不同模态的概念，并能事前解释概念与预测结果之间的相关性；2）提出通道级重校准模块，用于关注局部时空区域，使概念具备局部性；3）引入特征正则化损失，促使概念学习多样化的模式。研究在多个数据集和任务上对MulCPred进行了评估，定性和定量结果均表明，MulCPred在提升行人动作预测可解释性方面表现出色，且不会造成明显的性能下降。此外，通过移除MulCPred中不可识别的概念，其跨数据集预测性能得到提升，这表明MulCPred具备进一步提升泛化能力的可行性。
英文摘要：
Pedestrian action prediction is of great significance for many applications such as autonomous driving. However, state-of-the-art methods lack explainability to make trustworthy predictions. In this paper, a novel framework called MulCPred is proposed that explains its predictions based on multi-modal concepts represented by training samples. Previous concept-based methods have limitations including: 1) they cannot directly apply to multi-modal cases; 2) they lack locality to attend to details in the inputs; 3) they suffer from mode collapse. These limitations are tackled accordingly through the following approaches: 1) a linear aggregator to integrate the activation results of the concepts into predictions, which associates concepts of different modalities and provides ante-hoc explanations of the relevance between the concepts and the predictions; 2) a channel-wise recalibration module that attends to local spatiotemporal regions, which enables the concepts with locality; 3) a feature regularization loss that encourages the concepts to learn diverse patterns. MulCPred is evaluated on multiple datasets and tasks. Both qualitative and quantitative results demonstrate that MulCPred is promising in improving the explainability of pedestrian action prediction without obvious performance degradation. Furthermore, by removing unrecognizable concepts from MulCPred, the cross-dataset prediction performance is improved, indicating the feasibility of further generalizability of MulCPred.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
