Rank and Align: Towards Effective Source-free Graph Domain Adaptation
School of Computer Science, National Key Laboratory for Multimedia Information Processing(多媒体信息处理国家重点实验室计算机学院);University of California, Los Angeles(加州大学洛杉矶分校);University of International Business and Economics(对外经济贸易大学);Anker Innovations(安克创新);Peking University(北京大学);Peking University-Anker Embodied AI Lab(北京大学-安克具身智能实验室)
针对无源图领域自适应问题,提出Rank and Align (RNA)方法,通过谱序列化进行鲁棒语义学习,并利用对抗性边采样提取域不变子图以减少域差异,实验证明其有效。
摘要:
图神经网络(GNNs)在图领域自适应中取得了令人印象深刻的性能。然而,由于隐私和存储问题,在真实场景中可能无法获取大量的源图。为此,我们研究了无源图领域自适应这一尚未充分探索但实用的问题,该方法将知识从源模型而非源图迁移到目标域。为解决此问题,我们引入了一种名为Rank and Align (RNA)的新型基于GNN的方法,该方法利用谱序列化对图相似性进行排序以进行鲁棒语义学习,并将不和谐图与接近源域的和谐图对齐以进行子图提取。具体而言,为克服标签稀缺,我们采用谱序列化算法推断鲁棒的成对排序,利用相似性学习目标指导语义学习。为描绘分布偏移,我们利用谱聚类和轮廓系数检测源模型能轻松分类的和谐图。为减少潜在域差异,我们通过对抗性边采样过程从不和谐图中提取域不变子图,指导GNNs的不变性学习。在多个基准数据集上的大量实验证明了我们提出的RNA的有效性。
英文摘要:
Graph neural networks (GNNs) have achieved impressive performance in graph domain adaptation. However, extensive source graphs could be unavailable in real-world scenarios due to privacy and storage concerns. To this end, we investigate an underexplored yet practical problem of source-free graph domain adaptation, which transfers knowledge from source models instead of source graphs to a target domain. To solve this problem, we introduce a novel GNN-based approach called Rank and Align (RNA), which ranks graph similarities with spectral seriation for robust semantics learning, and aligns inharmonic graphs with harmonic graphs which close to the source domain for subgraph extraction. In particular, to overcome label scarcity, we employ the spectral seriation algorithm to infer the robust pairwise rankings, which can guide semantic learning using a similarity learning objective. To depict distribution shifts, we utilize spectral clustering and the silhouette coefficient to detect harmonic graphs, which the source model can easily classify. To reduce potential domain discrepancy, we extract domain-invariant subgraphs from inharmonic graphs by an adversarial edge sampling process, which guides the invariant learning of GNNs. Extensive experiments on several benchmark datasets demonstrate the effectiveness of our proposed RNA.
Australian Centre for Robotics, School of Aerospace, Mechanical and Mechatronic Engineering, The University of Sydney(澳大利亚机器人中心,航空航天、机械与机电工程学院,悉尼大学);Google DeepMind(谷歌DeepMind)
MoR: Mixture of Ranks for Low-Rank Adaptation Tuning
Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);School of Cyber Security, University of Chinese Academy of Sciences(中国科学院大学网络空间安全学院);Baidu Inc.(百度公司)
Joint Tensor-Train Parameterization for Efficient and Expressive Low-Rank Adaptation
Georgia Institute of Technology(佐治亚理工学院);National Taiwan University(国立台湾大学);University of Palermo(帕尔米奥大学);NVIDIA Research(NVIDIA研究);Hon-Hai Quantum Computing Research Center(鸿海量子计算研究中心)
University of Science and Technology of China(中国科学技术大学);Institute of Intelligent Machines, HFIPS, Chinese Academy of Sciences(中国科学院智能机械研究所);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);Fudan University(复旦大学);The Chinese University of Hong Kong(香港中文大学);University of Sydney(悉尼大学);Anhui Disaster Warning & Agrometeorological Information Center(安徽省灾害预警与农业气象信息中心)
QuanTA: Efficient High-Rank Fine-Tuning of LLMs with Quantum-Informed Tensor Adaptation
NSF AI Institute for Artificial Intelligence and Fundamental Interactions(NSF人工智能与基础交互AI研究所);Massachusetts Institute of Technology(麻省理工学院);Harvard University(哈佛大学)
DeLoRA: Decoupling Angles and Strength in Low-rank Adaptation
University of Tübingen(图宾根大学);Tübingen AI Center(图宾根人工智能中心);Helmholtz Munich(海德堡-穆恩医疗中心);Technical University of Munich(慕尼黑技术大学);Munich Center for Machine Learning(慕尼黑机器学习中心);MDSI(慕尼黑数据科学研究所)