arXivDaily arXiv每日学术速递 周一至周五更新
北美退伍军人节(Veterans Day),12 日无更新
arXiv 2412.16144cs.LGcs.DC
收藏

FedGAT:一种用于图注意力网络的隐私保护联邦近似算法

FedGAT: A Privacy-Preserving Federated Approximation Algorithm for Graph Attention Networks

Carnegie Mellon University(卡内基梅隆大学)

Siddharth Ambekar, Yuhang Yao, Ryan Li, Carlee Joe-Wong

AI总结:

针对图注意力网络联邦训练中跨客户端边的隐私与通信开销问题,提出FedGAT算法,仅需一轮预训练通信即可近似GAT行为,在保证近似误差界限的同时实现与集中式GAT相当的准确率。

摘要:

联邦训练方法在图学习中越来越受欢迎,其应用包括社交媒体网站的友谊图和大型在线市场的客户-商家交互图。然而,隐私法规通常要求本地生成的数据存储在本地客户端上。因此,图自然地被划分在各个客户端上,不允许任何客户端访问存储在另一客户端上的信息。在这种情况下,跨客户端边自然产生,并对联邦训练方法提出了有趣的挑战,因为在一个客户端训练图模型需要跨客户端边另一端节点的特征信息。试图保留此类边通常会产生显著的通信开销,而完全丢弃它们会降低模型性能。在图卷积网络等较简单的模型中,这可以通过在训练前跨客户端通信少量特征信息来解决,但GAT(图注意力网络)需要额外的、无法预先通信的信息,因为这些信息在每轮训练中都会发生变化。我们引入了用于半监督节点分类的Federated Graph Attention Network (FedGAT) 算法,该算法近似GAT的行为,并具有可证明的近似误差界限。FedGAT仅需一轮预训练通信,显著减少了联邦GAT训练的通信开销。然后,我们分析了近似误差,并检查了该算法的通信开销和计算复杂度。实验表明,FedGAT在集中式设置中实现了与GAT模型几乎相同的准确率,并且其性能对客户端数量和数据分布具有鲁棒性。

英文摘要:

Federated training methods have gained popularity for graph learning with applications including friendship graphs of social media sites and customer-merchant interaction graphs of huge online marketplaces. However, privacy regulations often require locally generated data to be stored on local clients. The graph is then naturally partitioned across clients, with no client permitted access to information stored on another. Cross-client edges arise naturally in such cases and present an interesting challenge to federated training methods, as training a graph model at one client requires feature information of nodes on the other end of cross-client edges. Attempting to retain such edges often incurs significant communication overhead, and dropping them altogether reduces model performance. In simpler models such as Graph Convolutional Networks, this can be fixed by communicating a limited amount of feature information across clients before training, but GATs (Graph Attention Networks) require additional information that cannot be pre-communicated, as it changes from training round to round. We introduce the Federated Graph Attention Network (FedGAT) algorithm for semi-supervised node classification, which approximates the behavior of GATs with provable bounds on the approximation error. FedGAT requires only one pre-training communication round, significantly reducing the communication overhead for federated GAT training. We then analyze the error in the approximation and examine the communication overhead and computational complexity of the algorithm. Experiments show that FedGAT achieves nearly the same accuracy as a GAT model in a centralised setting, and its performance is robust to the number of clients as well as data distribution.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑