超越跟踪：使用深度学习发现生物群中的新相互作用
Beyond Tracking: Using Deep Learning to Discover Novel Interactions in Biological Swarms
摘要：
大多数用于理解生物群体的深度学习框架都被设计为将群体行为的感知模型与个体级数据相匹配（例如，个体的所识别特征的空间坐标），其已经从视频观察中单独地收集。尽管在自动跟踪方面取得了相当大的进展，但当同时跟踪大量动物时，这些方法仍然非常昂贵或不可靠。此外，这种方法假设人类选择的特征包括足够的特征来解释集体行为中的重要模式。为了解决这些问题，我们建议训练深度网络模型，以直接从整个视图的通用图形特征预测系统级状态，这可以相对便宜地以完全自动化的方式收集。由于所得到的预测模型不是基于人类理解的预测因子，因此我们使用解释性模块（例如，Grad-CAM）将隐藏在深层网络模型的潜在变量中的信息与视频数据本身相结合，以向人类观察者传达观察到的个人行为的哪些方面在预测群体行为方面最具信息性。这代表了行为生态学中增强智能的一个例子--人类-人工智能团队中的知识共同创造。作为概念证明，我们利用一个超过50只Harpegnathos saltator蚂蚁的殖民地的20天视频记录来展示，在没有任何单独注释的情况下，经过训练的模型可以在视频帧中生成一个“重要性地图”，以突出重要行为的区域，例如决斗（AI没有先验知识），这在生殖层次结构重建的解决方案中发挥作用。根据实证结果，我们还讨论了潜在的用途和当前的挑战。
英文摘要：
Most deep-learning frameworks for understanding biological swarms are designed to fit perceptive models of group behavior to individual-level data (e.g., spatial coordinates of identified features of individuals) that have been separately gathered from video observations. Despite considerable advances in automated tracking, these methods are still very expensive or unreliable when tracking large numbers of animals simultaneously. Moreover, this approach assumes that the human-chosen features include sufficient features to explain important patterns in collective behavior. To address these issues, we propose training deep network models to predict system-level states directly from generic graphical features from the entire view, which can be relatively inexpensive to gather in a completely automated fashion. Because the resulting predictive models are not based on human-understood predictors, we use explanatory modules (e.g., Grad-CAM) that combine information hidden in the latent variables of the deep-network model with the video data itself to communicate to a human observer which aspects of observed individual behaviors are most informative in predicting group behavior. This represents an example of augmented intelligence in behavioral ecology -- knowledge co-creation in a human-AI team. As proof of concept, we utilize a 20-day video recording of a colony of over 50 Harpegnathos saltator ants to showcase that, without any individual annotations provided, a trained model can generate an "importance map" across the video frames to highlight regions of important behaviors, such as dueling (which the AI has no a priori knowledge of), that play a role in the resolution of reproductive-hierarchy re-formation. Based on the empirical results, we also discuss the potential use and current challenges.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
