用于参与者行为推断和预测的入口翻转Transformer
Entry-Flipped Transformer for Inference and Prediction of Participant Behavior
摘要:
一些团体活动,如团队运动和编排舞蹈,涉及参与者之间的密切互动。在这里,我们调查的任务推断和预测参与者的行为,在运动路径和行动,在这种条件下。我们缩小了问题,估计一组目标参与者如何对其他观察到的参与者的行为作出反应。我们的主要思想是以一种对帧式推理和预测过程中的错误积累具有鲁棒性的方式对参与者之间的时空关系进行建模。我们提出了一种新的入口翻转的Transformer(EF-Transformer),它通过时空域上的注意力机制来模拟参与者之间的关系。与典型的Transformers不同,我们通过翻转查询,键和值条目的顺序来解决错误累积的问题,以增加当前帧中观察到的特征的重要性和保真度。对比实验表明,我们的EF-Transformer在新收集的网球双打数据集,Ceilidh舞蹈数据集和两个行人数据集上实现了最佳性能。此外,它也表明,我们的EF-变压器是更好地限制累积误差和从错误的估计恢复。
英文摘要:
Some group activities, such as team sports and choreographed dances, involve closely coupled interaction between participants. Here we investigate the tasks of inferring and predicting participant behavior, in terms of motion paths and actions, under such conditions. We narrow the problem to that of estimating how a set target participants react to the behavior of other observed participants. Our key idea is to model the spatio-temporal relations among participants in a manner that is robust to error accumulation during frame-wise inference and prediction. We propose a novel Entry-Flipped Transformer (EF-Transformer), which models the relations of participants by attention mechanisms on both spatial and temporal domains. Unlike typical transformers, we tackle the problem of error accumulation by flipping the order of query, key, and value entries, to increase the importance and fidelity of observed features in the current frame. Comparative experiments show that our EF-Transformer achieves the best performance on a newly-collected tennis doubles dataset, a Ceilidh dance dataset, and two pedestrian datasets. Furthermore, it is also demonstrated that our EF-Transformer is better at limiting accumulated errors and recovering from wrong estimations.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。