多时间尺度自举学习行为表征
Learning Behavior Representations Through Multi-Timescale Bootstrapping
摘要:
自然行为包括不可预测的动态,可以突然切换,并在许多不同的时间尺度上展开。虽然在约束或简化的基于任务的条件下,在构建行为表征方面取得了一些成功,但这些模型中的许多模型不能应用于自由和自然的环境,因为它们假设了单一尺度的时间动态。在这项工作中,我们引入了多尺度引导(BAMS),一个多尺度的行为表示学习模型:我们结合了一个池化模块,该模块聚合了在具有不同时间感受野的编码器上提取的特征,并设计了一组潜在目标来引导每个空间中的表示,以鼓励在不同时间尺度上解开纠缠。我们首先将我们的方法应用于四足动物在不同地形类型中导航的数据集,并表明我们的模型捕获了行为的时间复杂性。然后,我们将我们的方法应用于MABe 2022多智能体行为挑战,其中我们的模型在两个子任务中排名第三和第一,并显示了在分析行为时纳入多时间尺度的重要性。
英文摘要:
Natural behavior consists of dynamics that are both unpredictable, can switch suddenly, and unfold over many different timescales. While some success has been found in building representations of behavior under constrained or simplified task-based conditions, many of these models cannot be applied to free and naturalistic settings due to the fact that they assume a single scale of temporal dynamics. In this work, we introduce Bootstrap Across Multiple Scales (BAMS), a multi-scale representation learning model for behavior: we combine a pooling module that aggregates features extracted over encoders with different temporal receptive fields, and design a set of latent objectives to bootstrap the representations in each respective space to encourage disentanglement across different timescales. We first apply our method on a dataset of quadrupeds navigating in different terrain types, and show that our model captures the temporal complexity of behavior. We then apply our method to the MABe 2022 Multi-agent behavior challenge, where our model ranks 3rd overall and 1st on two subtasks, and show the importance of incorporating multi-timescales when analyzing behavior.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。