针对多智能体系统底层图结构难以识别的问题,提出利用注意力机制学习节点表示并依据注意力强度推断网络拓扑的数据驱动模型,在未知动力学下仍取得良好链路预测 F1 分数。
摘要:
准确识别多智能体系统的底层图结构仍是一项困难挑战。我们的工作提出了一种新颖的基于机器学习的解决方案,该方案利用注意力机制,通过学习节点表示来预测多智能体系统的未来状态。随后,根据注意力值的强度推断图结构。该方法被应用于线性一致性动力学和 Kuramoto 振子的非线性动力学,通过学习良好的智能体表示实现了对图的隐式学习。我们的结果表明,所提出的数据驱动图注意力机器学习模型能够识别多智能体系统中的网络拓扑,即使在底层动态模型未知时也是如此,链路预测中取得的 F1 分数证明了这一点。
英文摘要:
Accurately identifying the underlying graph structures of multi-agent systems remains a difficult challenge. Our work introduces a novel machine learning-based solution that leverages the attention mechanism to predict future states of multi-agent systems by learning node representations. The graph structure is then inferred from the strength of the attention values. This approach is applied to both linear consensus dynamics and the non-linear dynamics of Kuramoto oscillators, resulting in implicit learning of the graph by learning good agent representations. Our results demonstrate that the presented data-driven graph attention machine learning model can identify the network topology in multi-agent systems, even when the underlying dynamic model is not known, as evidenced by the F1 scores achieved in the link prediction.
Acceptability Judgements via Examining the Topology of Attention Maps
Skolkovo Institute of Science and Technology(斯科尔科沃科学技术研究院);AIRI(AIRI(人工智能研究院));SberDevices(SberDevices(俄联邦储蓄银行设备公司));HSE University(高等经济大学);Huawei Noah’s Ark lab(华为诺亚方舟实验室);Center for Information and Language Processing (CIS), LMU Munich(慕尼黑大学信息与语言处理中心);CNRS(法国国家科学研究中心)
本文利用拓扑数据分析提取注意力图的几何特征,用于语法可接受性二元判断和最小对比较,在多语言 CoLA 与 BLiMP 上显著提升性能并达到人类水平。
Ltri-LLM: Streaming Long Context Inference for LLMs with Training-Free Dynamic Triangular Attention Pattern
Meituan Inc.(美团公司);Aerospace Information Research Institute, Chinese Academy of Sciences(中国科学院空天信息创新研究院);Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所)
HATA: Trainable and Hardware-Efficient Hash-Aware Top-k Attention for Scalable Large Model Inference
University of Science and Technology of China(中国科学技术大学);Institute of Artificial Intelligence(人工智能研究院);Hefei Comprehensive National Science Center(合肥综合性国家科学中心);Huawei Technologies(华为技术有限公司);Peking University(北京大学)