arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2604.18237cs.LGcs.AI
收藏

基于语义的分布式学习以获得多样化和判别性的表示

Semantic-based Distributed Learning for Diverse and Discriminative Representations

Center for Wireless Communications, University of Oulu(奥卢大学无线通信中心)

Zhuojun Tian, Chaouki Ben Issaid, Mehdi Bennis

AI总结:

本文提出一种新型分布式学习框架,通过引入表示方差约束,解决传统方法导致的非结构化嵌入问题,确保表示的多样性和判别性,并在MNIST、CIFAR-10和CIFAR-100上验证了算法的有效性。

摘要:

在大规模分布式场景中,日益复杂的任务需要网络间的更智能协作,要求从数据样本中联合提取结构表示。然而,传统任务特定方法常导致非结构化嵌入,使得同一类数据样本之间的变异性坍塌,特别是在分类任务中。为了解决这一问题并充分利用数据的内在结构以用于下游应用,我们提出了一种新的分布式学习框架,确保既多样化又判别性的表示。对于独立同分布(i.i.d.)数据,我们通过引入对表示方差的约束来重新公式化并解耦全局优化函数。然后通过对偶方法推导并简化更新规则。对于非i.i.d.数据分布,我们通过聚类和虚拟复制节点来解决该问题,允许在每个聚类内使用块坐标下降法进行模型更新。在两种情况下,所得最优解都被理论证明保持判别性和多样性,并且对于i.i.d.条件有保证的收敛性。此外,从表示中获得的语义信息在节点之间共享,减少了对常见神经网络架构的需求。最后,在MNIST、CIFAR-10和CIFAR-100上的大量模拟证实了所提出算法在捕获全局结构表示方面的有效性。

英文摘要:

In large-scale distributed scenarios, increasingly complex tasks demand more intelligent collaboration across networks, requiring the joint extraction of structural representations from data samples. However, conventional task-specific approaches often result in nonstructural embeddings, leading to collapsed variability among data samples within the same class, particularly in classification tasks. To address this issue and fully leverage the intrinsic structure of data for downstream applications, we propose a novel distributed learning framework that ensures both diverse and discriminative representations. For independent and identically distributed (i.i.d.) data, we reformulate and decouple the global optimization function by introducing constraints on representation variance. The update rules are then derived and simplified using a primal-dual approach. For non-i.i.d. data distributions, we tackle the problem by clustering and virtually replicating nodes, allowing model updates within each cluster using block coordinate descent. In both cases, the resulting optimal solutions are theoretically proven to maintain discriminative and diverse properties, with a guaranteed convergence for i.i.d. conditions. Additionally, semantic information from representations is shared among nodes, reducing the need for common neural network architectures. Finally, extensive simulations on MNIST, CIFAR-10 and CIFAR-100 confirm the effectiveness of the proposed algorithms in capturing global structural representations.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑