发表机构
INRIA; Université de la Polynésie française; University of Sheffield; Princeton University（法国国家信息与自动化研究所; 法属波利尼西亚大学; 谢菲尔德大学; 普林斯顿大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文研究吉布斯测度的三种运算，证明嵌套运算与对数线性组合在特定参数下等价，并展示其在联邦学习中的应用，实现与集中训练相同的性能。
AI 中文摘要
本文研究了吉布斯概率测度上的三种运算。第一种运算通常称为重整化，它取一个吉布斯概率测度，通过对其密度的幂进行归一化来生成新的吉布斯测度。这种归一化具有双重效果：它改变了正则化因子，并将支撑集集中到原始支撑集的子集内。有趣的是，这些效果可以通过不同的参数独立控制。第二种运算包括吉布斯概率测度密度的归一化对数线性组合。第三种运算取两个吉布斯概率测度，并用前者改变后者的参考测度。因此，前者被称为“嵌套”在后者中，产生新的吉布斯概率测度。由第二种和第三种运算得到的测度也是吉布斯概率测度，并且分别被证明可以解决涉及给定测度目标函数线性组合期望的优化问题，并受相对熵正则化约束。这些优化问题仅在线性组合的系数上有所不同。这导致结论：存在一组参数，使得将一个吉布斯概率测度嵌套到另一个中的效果与对数线性组合它们的效果相同。这些运算在统计学习中具有相关应用。例如，一个一次性联邦学习系统中，客户端将其本地训练的吉布斯算法发送到服务器进行对数线性组合，被证明可以达到与在所有本地训练数据集的聚合上训练的吉布斯算法相同的性能。
英文摘要
In this paper, three operations on Gibbs probability measures are studied. The first operation, often referred to as renormalization, takes one Gibbs probability measure and generates a new Gibbs measure by normalizing a power of its density. This normalization has a twofold effect: it changes the regularization factor and concentrates the support within a subset of the original support. Interestingly, these effects can be independently controlled by different parameters. The second operation consists of a normalized log-linear combination of the densities of Gibbs probability measures. The third operation takes two Gibbs probability measures and changes the reference measure of the latter with the former. Hence, the former is said to be "nested" within the latter, yielding a new Gibbs probability measure. The resulting measures from the second and third operations are also Gibbs probability measures and are shown, respectively, to solve optimization problems involving the expectations of linear combinations of the objective functions of the given measures, subject to a relative entropy regularization. These optimization problems differ exclusively in the coefficients of the linear combinations. This leads to the conclusion that there exists a set of parameters for which nesting one Gibbs probability measure into another has the same effect as log-linearly combining them. These operations have relevant applications in statistical learning. As an example, a one-shot federated learning system in which clients send their locally trained Gibbs algorithms to the server for log-linear combination is shown to achieve the same performance as a Gibbs algorithm trained upon the aggregation of all local training datasets.
CommentsIn Proceedings of the IEEE Information Theory Workshop (ITW), 2026
