追溯行动：通过逆转“打开”视频来学习“关闭”
Retro-Actions: Learning 'Close' by Time-Reversing 'Open' Videos
摘要：
我们调查视频变换，导致类同质标签变换。这些视频转换一致地维护或修改每个类中所有视频的标签。我们提出了一种通用的方法来发现不变类，其变换的例子保持其标签;对等变类，其变换的例子交换其标签;和新颖的生成类，其变换的例子属于一个新的类以外的数据集。标签变换提供了以前在视频识别中未探索的额外监督，有利于数据增强，并通过从其对应物的变换视频中学习类来实现零样本学习机会。在这样的视频变换，我们研究水平翻转，时间反转，和他们的组成。我们强调了在视频中天真地使用水平翻转作为数据增强的一种形式的错误。接下来，我们通过人类感知研究来验证时间反转视频的真实性，在该研究中，人们对向前和时间反转视频表现出相同的偏好。最后，我们在两个数据集Jester和Something-Something上测试了我们的方法，评估了零样本学习和数据增强的三种视频变换。我们的研究结果表明，在zero-shot设置中，可以通过缩小来学习放大等手势，以及具有状态转换的更复杂的动作，例如从某物中挖出某物或将某物埋在某物中。
英文摘要：
We investigate video transforms that result in class-homogeneous label-transforms. These are video transforms that consistently maintain or modify the labels of all videos in each class. We propose a general approach to discover invariant classes, whose transformed examples maintain their label; pairs of equivariant classes, whose transformed examples exchange their labels; and novel-generating classes, whose transformed examples belong to a new class outside the dataset. Label transforms offer additional supervision previously unexplored in video recognition benefiting data augmentation and enabling zero-shot learning opportunities by learning a class from transformed videos of its counterpart. Amongst such video transforms, we study horizontal-flipping, time-reversal, and their composition. We highlight errors in naively using horizontal-flipping as a form of data augmentation in video. Next, we validate the realism of time-reversed videos through a human perception study where people exhibit equal preference for forward and time-reversed videos. Finally, we test our approach on two datasets, Jester and Something-Something, evaluating the three video transforms for zero-shot learning and data augmentation. Our results show that gestures such as zooming in can be learnt from zooming out in a zero-shot setting, as well as more complex actions with state transitions such as digging something out of something from burying something in something.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
