通过测量动词与副词的文本关系学习动作变化
Learning Action Changes by Measuring Verb-Adverb Textual Relationships
AI总结：
本工作通过测量动词与副词的文本关系生成回归目标，将动作变化预测转化为回归任务，在多个数据集上取得最先进结果，并构建了高质量数据集AIR以提升学习与评估可靠性。
摘要：
本工作的目标是理解视频中动作的执行方式。即给定一个视频，我们旨在预测一个副词，该副词指示应用于动作的修改（例如“精细地”切）。我们将此问题转化为回归任务。我们测量动词与副词之间的文本关系，以生成一个回归目标，代表我们旨在学习的动作变化。我们在多个数据集上测试了我们的方法，并在副词预测和反义词分类上均取得了最先进的结果。此外，当我们放宽两个通常假设的条件时，即测试期间动作标签的可用性以及副词作为反义词的配对，我们的表现优于先前的工作。现有的副词识别数据集要么存在噪声，导致学习困难，要么包含其外观不受副词影响的动作，导致评估不够可靠。为了解决这一问题，我们收集了一个新的高质量数据集：食谱中的副词（Adverbs in Recipes, AIR）。我们专注于教学食谱视频，精心挑选了一组在不同执行方式下表现出有意义视觉变化的动作。AIR中的视频经过更紧密的裁剪，并由多名标注者手动审查以确保高标注质量。结果表明，鉴于AIR更干净的视频，模型从中学习得更好。同时，AIR上的副词预测具有挑战性，表明仍有相当大的改进空间。
英文摘要：
The goal of this work is to understand the way actions are performed in videos. That is, given a video, we aim to predict an adverb indicating a modification applied to the action (e.g. cut "finely"). We cast this problem as a regression task. We measure textual relationships between verbs and adverbs to generate a regression target representing the action change we aim to learn. We test our approach on a range of datasets and achieve state-of-the-art results on both adverb prediction and antonym classification. Furthermore, we outperform previous work when we lift two commonly assumed conditions: the availability of action labels during testing and the pairing of adverbs as antonyms. Existing datasets for adverb recognition are either noisy, which makes learning difficult, or contain actions whose appearance is not influenced by adverbs, which makes evaluation less reliable. To address this, we collect a new high quality dataset: Adverbs in Recipes (AIR). We focus on instructional recipes videos, curating a set of actions that exhibit meaningful visual changes when performed differently. Videos in AIR are more tightly trimmed and were manually reviewed by multiple annotators to ensure high labelling quality. Results show that models learn better from AIR given its cleaner videos. At the same time, adverb prediction on AIR is challenging, demonstrating that there is considerable room for improvement.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
