LAEO-Net++：重新审视视频中相互注视的人
LAEO-Net++: revisiting people Looking At Each Other in videos
摘要：
捕捉人们的“相互凝视”对于理解和解释他们之间的社会互动至关重要。为此，本文提出了检测视频序列中的人看着对方（LAEO）的问题。为此，我们提出了LAEO-Net++，这是一种新的深度CNN，用于确定视频中的LAEO。与以往的工作相比，LAEO-Net++以时空轨迹为输入，对整个轨迹进行推理。它由三个分支组成，一个用于每个角色的跟踪头部，另一个用于它们的相对位置。此外，我们还介绍了两个新的LAEO数据集：UCO-LAEO和AVA-LAEO。一个全面的实验评估表明，LAEO-Net++能够成功地确定两个人是否是LAEO以及发生这种情况的时间窗口。我们的模型在现有的TVHID-LAEO视频数据集上实现了最先进的结果，显著优于以前的方法。最后，我们将LAEO-Net++应用于社交网络，在那里我们根据LAEO的频率和持续时间自动推断成对人之间的社交关系，并表明LAEO可以成为指导搜索视频中人类交互的有用工具。该代码可在https://github.com/AVAuco/laeonetplus上获得。
英文摘要：
Capturing the 'mutual gaze' of people is essential for understanding and interpreting the social interactions between them. To this end, this paper addresses the problem of detecting people Looking At Each Other (LAEO) in video sequences. For this purpose, we propose LAEO-Net++, a new deep CNN for determining LAEO in videos. In contrast to previous works, LAEO-Net++ takes spatio-temporal tracks as input and reasons about the whole track. It consists of three branches, one for each character's tracked head and one for their relative position. Moreover, we introduce two new LAEO datasets: UCO-LAEO and AVA-LAEO. A thorough experimental evaluation demonstrates the ability of LAEO-Net++ to successfully determine if two people are LAEO and the temporal window where it happens. Our model achieves state-of-the-art results on the existing TVHID-LAEO video dataset, significantly outperforming previous approaches. Finally, we apply LAEO-Net++ to a social network, where we automatically infer the social relationship between pairs of people based on the frequency and duration that they LAEO, and show that LAEO can be a useful tool for guided search of human interactions in videos. The code is available at https://github.com/AVAuco/laeonetplus.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
