By incorporating the graph structural information into Transformers, graph Transformers have exhibited promising performance for graph representation learning in recent years. Existing graph Transformers leverage specific strategies, such as Laplacian eigenvectors and shortest paths of the node pairs, to preserve the structural features of nodes and feed them into the vanilla Transformer to learn the representations of nodes. It is hard for such predefined rules to extract informative graph structural features for arbitrary graphs whose topology structure varies greatly, limiting the learning capacity of the models. To this end, we propose an adaptive graph Transformer, termed Multi-Neighborhood Attention based Graph Transformer (MNA-GT), which captures the graph structural information for each node from the multi-neighborhood attention mechanism adaptively. By defining the input to perform scaled-dot product as an attention kernel, MNA-GT constructs multiple attention kernels based on different hops of neighborhoods such that each attention kernel can capture specific graph structural information of the corresponding neighborhood for each node pair. In this way, MNA-GT can preserve the graph structural information efficiently by incorporating node representations learned by different attention kernels. MNA-GT further employs an attention layer to learn the importance of different attention kernels to enable the model to adaptively capture the graph structural information for different nodes. Extensive experiments are conducted on a variety of graph benchmarks, and the empirical results show that MNA-GT outperforms many strong baselines.
Tokenized Graph Transformer with Neighborhood Augmentation for Node Classification in Large Graphs
Institute of Artificial Intelligence, Huazhong University of Science and Technology（华中科技大学人工智能研究院）；School of Computer Science and Technology, Huazhong University of Science and Technology（华中科技大学计算机科学与技术学院）；Hopcroft Center on Computing Science, Huazhong University of Science and Technology（华中科技大学霍普克罗夫特计算科学中心）
School of Computer Science and Technology, Huazhong University of Science and Technology（华中科技大学计算机科学与技术学院）；Hopcroft Center on Computing Science, Huazhong University of Science and Technology（华中科技大学计算科学霍普克罗夫特中心）；Department of Computer Science, Cornell University（康奈尔大学计算机科学系）
Attention Beyond Neighborhoods: Reviving Transformer for Graph Clustering
School of Computer Science and Engineering, University of Electronic Science and Technology of China（电子科技大学计算机科学与工程学院）；Michigan State University（密歇根州立大学）；Alibaba Group（阿里巴巴集团）
Neighborhood-Adaptive Generalized Linear Graph Embedding with Latent Pattern Mining
School of Computer and Information, Anhui Normal University（安徽师范大学计算机与信息学院）；Anhui Provincial Key Laboratory of Industrial Intelligent Data Security（安徽省工业智能数据安全重点实验室）；Institute of Automation, State Key Laboratory of Multimodal Artificial Intelligence Systems, Chinese Academy of Sciences（自动化研究所，多模态人工智能系统国家重点实验室，中国科学院）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）
Spatially-Adaptive Conformal Graph Transformer for Indoor Localization in Wi-Fi Driven Networks
Department of Computer Science, The University of Vermont（佛罗里达大学计算机科学系）；Department of Computer Science & Engineering, American University of Sharjah（阿曼大学计算机科学与工程系）
NEAT: Neighborhood-Guided, Efficient, Autoregressive Set Transformer for 3D Molecular Generation
Department of Pharmaceutical Sciences, University of Vienna（维也纳大学药学系）；Christian Doppler Laboratory for Molecular Informatics in the Biosciences（生物医学分子信息学克里斯蒂安·多普勒实验室）；Vienna Doctoral School of Pharmaceutical, Nutritional and Sport Sciences（维也纳药学、营养学和运动科学博士学院）
ParBalans: Parallel Multi-Armed Bandits-based Adaptive Large Neighborhood Search
Department of Mechanical & Industrial Engineering, Northeastern University（东北大学机械与工业工程系）；AI Center of Excellence, Fidelity Investments（fidelity investments 精英研究中心）；Department of Computer Science, University of Southern California（南加州大学计算机科学系）；Department of Computer Science, Brown University（布朗大学计算机科学系）
Balans: Multi-Armed Bandits-based Adaptive Large Neighborhood Search for Mixed-Integer Programming Problem
Department of Computer Science, University of Southern California（南加州大学计算机科学系）；AI Center of Excellence, Fidelity Investments（富达投资人工智能卓越中心）；Department of Computer Science, Brown University（布朗大学计算机科学系）
Ferdowsi University of Mashhad（马什哈德费尔多西大学）；Center of Excellence in Soft Computing and Intelligent Information Processing, Ferdowsi University of Mashhad（马什哈德费尔多西大学软计算与智能信息处理卓越中心）
ECG-NAT: A Self-supervised Neighborhood Attention Transformer for Multi-lead Electrocardiogram Classification
Department of Computer Engineering, University of Kurdistan（库尔德斯坦大学计算机工程系）；Department of Mathematics and Operational Research, University of Mons（蒙斯大学数学与运筹学系）
New Adaptive Mechanism for Large Neighborhood Search using Dual Actor-Critic
School of Intelligent Science and Technology, Nanjing University of Science and Technology（智能科学与技术学院，南京理工大学）；Department of Mechanical Engineering, North China Electric Power University, Baoding 071003, China（机械工程系，华北电力大学，保定071003，中国）；EM Normandie Business School, Métis Lab, 92110, Clichy, France（EM诺曼底商学院，Métis实验室，92110，法国）；Université Paris-Saclay, CentraleSupélec, Laboratoire Génie Industriel, 91190, Gif-sur-Yvette, France（巴黎萨克雷大学，CentraleSupélec，工业工程实验室，91190，法国）
School of Automation, Northwestern Polytechnical University（西北工业大学自动化学院）；Information Engineering College of Yangzhou University（扬州大学信息工程学院）；State Key Lab of Intelligent Technologies and Systems, Beijing National Research Center for Information Science and Technology（智能技术与系统国家重点实验室，北京信息科学与技术国家研究中心）；Department of Automation, Tsinghua University（清华大学自动化系）
在本文中，我们揭示了度量学习将遭受严重的不可分割的问题，如果没有信息样本挖掘。由于不可分样本中经常会混入硬样本，现有的信息样本挖掘策略在处理不可分样本时可能会带来一些副作用，如目标函数不稳定等。针对这一问题，提出了一种新的距离度量学习算法，即自适应邻域度量学习（ANML）。在ANML中，我们设计了两个阈值来自适应地识别训练过程中不可分的相似和不相似样本，从而在同一过程中实现不可分样本去除和度量参数学习。由于所提出的ANML的非连续性，我们开发了一个巧妙的功能，命名为\emph{log-exp mean function}构建一个连续的配方来代替它，它可以有效地解决了梯度下降方法。与Triplet loss类似，ANML可用于学习线性嵌入和深度嵌入。通过分析所提出的方法，我们发现它有一些有趣的性质。例如，当ANML用于学习线性嵌入时，当前著名的度量学习算法如大间隔最近邻（LMNN）和邻域分量分析（NCA）通过设置不同的参数值而成为所提出的ANML的特例。当它被用来学习深度特征时，最先进的深度度量学习算法如Triplet loss、Lifted structure loss和Multi-similarity loss成为ANML的特例。此外，在我们的方法中提出的\emph{log-exp mean function}提供了一个新的视角来回顾深度度量学习方法，如Prox-NCA和N对损失…
Adaptive Rank Allocation: Speeding Up Modern Transformers with RaNA Adapters
Institute of Computational and Mathematical Engineering, Stanford University（计算与数学工程研究所，斯坦福大学）；Department of Computer Science, Stanford University（计算机科学系，斯坦福大学）
