下一步做什么？从第一人称教学视频中记忆技能
What to Do Next? Memorizing skills from Egocentric Instructional Video
AI总结：
本文研究第一人称仿真环境中的交互式动作规划，提出结合拓扑可供性记忆与Transformer的方法，以学习环境表示、选择动作并检测偏差，实验证明其性能提升且对动作偏差具有鲁棒性。
摘要：
通过演示学习执行活动需要从观察中提取关于环境的有意义信息。在本研究中，我们探讨了在第一人称视角下的仿真环境中规划高层次目标导向动作的挑战。我们提出了一个新任务——交互式动作规划，并提出了一种将拓扑可供性记忆与Transformer架构相结合的方法。通过提取可供性来记忆环境结构的过程有助于根据上下文选择合适的动作。此外，该记忆模型使我们能够在完成特定目标的过程中检测动作偏差。为了评估该方法的通用性，我们在一个逼真的交互式仿真环境中对其进行了评估。我们的实验结果表明，所提出的方法能够学习有意义的表示，从而在动作偏差发生时获得更好的性能并具有鲁棒性。
英文摘要：
Learning to perform activities through demonstration requires extracting meaningful information about the environment from observations. In this research, we investigate the challenge of planning high-level goal-oriented actions in a simulation setting from an egocentric perspective. We present a novel task, interactive action planning, and propose an approach that combines topological affordance memory with transformer architecture. The process of memorizing the environment's structure through extracting affordances facilitates selecting appropriate actions based on the context. Moreover, the memory model allows us to detect action deviations while accomplishing specific objectives. To assess the method's versatility, we evaluate it in a realistic interactive simulation environment. Our experimental results demonstrate that the proposed approach learns meaningful representations, resulting in improved performance and robust when action deviations occur.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
