最近，许多神经模型已经被提出来解决组合谜题，通过使用它们的解决实例隐式地学习底层约束，例如数独或图着色（GCP）。通常基于图神经网络（GNN）的所提出的架构的一个缺点是，它们不能在输出空间的大小上进行泛化，从输出空间中为变量分配值，例如，GCP中的颜色集，或数独中的板大小。我们将变量的输出空间称为“value-set”。虽然许多工作已经证明了GNN在图大小上的泛化，但还没有研究如何设计GNN来实现来自同一域的问题的值集不变性。例如，学习解决16 x 16的数独后，只有9 x 9的数独训练。在这项工作中，我们提出了新的方法来扩展基于GNN的架构，以实现值集不变性。具体来说，我们的模型建立在最近提出的循环关系网络。我们的第一种方法通过将多类节点分类问题转换为二进制节点分类问题来利用GNN的图大小不变性。我们的第二种方法通过添加多个与值集中的值对应的节点，然后根据问题的初始化将变量节点连接到值节点来直接处理多个类。我们对三个不同的组合问题的实验评估表明，我们的模型在我们的新问题上表现良好，与通用神经推理机相比。在我们的两个模型之间，我们观察到一个内在的权衡：虽然二值化模型在较小的值集上训练时提供了更好的性能，但多值模型的内存效率要高得多，从而在较大的值集上训练时提高了性能，而二值化模型无法训练。
英文摘要：
Recently many neural models have been proposed to solve combinatorial puzzles by implicitly learning underlying constraints using their solved instances, such as sudoku or graph coloring (GCP). One drawback of the proposed architectures, which are often based on Graph Neural Networks (GNN), is that they cannot generalize across the size of the output space from which variables are assigned a value, for example, set of colors in a GCP, or board-size in sudoku. We call the output space for the variables as 'value-set'. While many works have demonstrated generalization of GNNs across graph size, there has been no study on how to design a GNN for achieving value-set invariance for problems that come from the same domain. For example, learning to solve 16 x 16 sudoku after being trained on only 9 x 9 sudokus. In this work, we propose novel methods to extend GNN based architectures to achieve value-set invariance. Specifically, our model builds on recently proposed Recurrent Relational Networks. Our first approach exploits the graph-size invariance of GNNs by converting a multi-class node classification problem into a binary node classification problem. Our second approach works directly with multiple classes by adding multiple nodes corresponding to the values in the value-set, and then connecting variable nodes to value nodes depending on the problem initialization. Our experimental evaluation on three different combinatorial problems demonstrates that both our models perform well on our novel problem, compared to a generic neural reasoner. Between two of our models, we observe an inherent trade-off: while the binarized model gives better performance when trained on smaller value-sets, multi-valued model is much more memory efficient, resulting in improved performance when trained on larger value-sets, where binarized model fails to train.
Constructing 3D Rotational Invariance and Equivariance with Symmetric Tensor Networks
Institute of Artificial Intelligence, Hefei Comprehensive National Science Center（人工智能研究所）；School of Information Science and Technology, University of Science and Technology of China（信息科学与技术学院）；CAS Key Laboratory of Quantum Information, University of Science and Technology of China（中国科学技术大学量子信息关键实验室）；Hefei National Laboratory, University of Science and Technology of China（合肥国家实验室）
Binarizing Physics-Inspired GNNs for Combinatorial Optimization
Department of Computer Science, Faculty of Electrical Engineering, Czech Technical University（计算机科学系，电气工程学院，捷克技术大学）；Quantum Technologies Group, HSBC Labs, Singapore（量子技术组，汇丰实验室，新加坡）；Archimedes Research Unit on AI, Data Science and Algorithms, Athena Research Center（人工智能、数据科学与算法研究单元，阿提卡研究中心）
VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning
Facebook AI Research（脸书人工智能研究院）；Inria, École normale supérieure, CNRS, PSL Research University（法国国家信息与自动化研究所、巴黎高等师范学院、法国国家科学研究中心、巴黎文理研究大学）；Courant Institute, New York University（纽约大学库朗数学科学研究所）；Center for Data Science, New York University（纽约大学数据科学中心）
Combinatorial Bandit Bayesian Optimization for Tensor Outputs
Department of Industrial Engineering, Tsinghua University, Beijing 100084, China（清华大学工业工程系）；College of Economics and Management, Nanjing University of Aeronautics and Astronautics, Nanjing 211106, China（南京航空航天大学经济管理学院）
Detecting Post-generation Edits to Watermarked LLM Outputs via Combinatorial Watermarking
Department of Industrial and Systems Engineering, University of Minnesota（工业与系统工程系，明尼苏达大学）；Google（谷歌）；Department of Electrical and Computer Engineering, Princeton University（电气工程与计算机科学系，普林斯顿大学）；Stony Brook University（石溪大学）
Geometry of the Loss Landscape in Overparameterized Neural Networks: Symmetries and Invariances
École Polytechnique Fédérale de Lausanne（洛桑联邦理工学院）
我们研究如何置换对称性在过参数化的多层神经网络产生的“神经诱导”的临界点。假设一个网络的$ L $层的最小宽度$ r_1^*, \ldots, r_{L-1}^* $达到一个零损失最小值在$ r_1^*! \cdots r_{L-1}^*! $孤立的点，是彼此的排列，我们表明，增加一个额外的神经元，每一层是足够的连接所有这些以前离散的最小值到一个单一的流形。对于宽度为$ r^*+ h =: m $的两层超参数化网络，我们明确地描述了全局极小点的流形：它由维数至少为$ h $的$ T(r^*, m) $仿射子空间相互连接组成。对于一个网络的宽度$m$，我们确定的数量$G(r,m)$的仿射子空间只包含与一个较小的网络的宽度$r<r^*$的临界点相关的临界点的诱导。通过组合分析，我们推导出$ T $和$ G $的封闭形式的公式，并表明，在轻度overparameterized制度（小$ h $），反之亦然，在大大overparameterized制度（$h \gg r^*$）的仿射子空间的数量占主导地位的临界子空间形成的全球极小流形的数量。我们的研究结果为超参数化神经网络的非凸损失函数的最小化提供了新的见解。
