首页/论文/1811.01900

arXiv 1811.019002019-02-25cs.LGstat.ML

Janossy Pooling: Learning Deep Permutation-Invariant Functions for Variable-Size Inputs

Purdue University（普渡大学）

Ryan L. Murphy, Balasubramaniam Srinivasan, Vinayak Rao, Bruno Ribeiro

备注：: This version clarifies and adds detail to some of the arguments
发表信息：: ICLR 2019

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

英文摘要相关论文

英文摘要：

We consider a simple and overarching representation for permutation-invariant functions of sequences (or multiset functions). Our approach, which we call Janossy pooling, expresses a permutation-invariant function as the average of a permutation-sensitive function applied to all reorderings of the input sequence. This allows us to leverage the rich and mature literature on permutation-sensitive functions to construct novel and flexible permutation-invariant functions. If carried out naively, Janossy pooling can be computationally prohibitive. To allow computational tractability, we consider three kinds of approximations: canonical orderings of sequences, functions with $k$-order interactions, and stochastic optimization algorithms with random permutations. Our framework unifies a variety of existing work in the literature, and suggests possible modeling and algorithmic extensions. We explore a few in our experiments, which demonstrate improved performance over current state-of-the-art methods.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

阅读原文

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

本文内容

英文摘要相关论文

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

继续探索

相关论文

深度集合和Janossy池化通用性的嵌入维数下界

Embedding Dimension Lower Bounds for Universality of Deep Sets and Janossy Pooling

Department of Mathematics（数学系）；Texas A&M University（德克萨斯大学）

研究深度集合和Janossy池化架构的通用性，确定嵌入维数下界以保证其通用性，为Deep Sets提供最小维数，对k-ary Janossy池化证明首次非平凡下界。

2026-05-08cs.LGstat.ML

论分段线性Janossy池化的（非）单射性

On the (Non) Injectivity of Piecewise Linear Janossy Pooling

Faculty of Computer Science（计算机科学系）；Technion - Israel Institute of Technology（技术ion-以色列理工学院）

本文研究多重集函数的单射性与双Lipschitz性，证明分段线性Janossy池化无法单射，但简单深度集模型在不含重数时足以满足这些性质。

2025-05-26cs.AIcs.LG

Universal Approximation of Functions on Sets

University of Oxford（牛津大学）

Modelling functions of sets, or equivalently, permutation-invariant functions, is a long-standing challenge in machine learning. Deep Sets is a popular method which is known to be a universal approximator for continuous set functions. We provide a theoretical analysis of Deep Sets which shows that this universal approximation property is only guaranteed if the model's latent space is sufficiently high-dimensional. If the latent space is even one dimension lower than necessary, there exist piecewise-affine functions for which Deep Sets performs no better than a naïve constant baseline, as judge…

2021-07-05cs.LGstat.ML

Learning the Multiple Traveling Salesmen Problem with Permutation Invariant Pooling Networks

The School of Computer Science（计算机科学学院）；Tel Aviv University（特拉维夫大学）；Facebook AI Research（脸书人工智能研究院）

While there are optimal TSP solvers, as well as recent learning-based approaches, the generalization of the TSP to the Multiple Traveling Salesmen Problem is much less studied. Here, we design a neural network solution that treats the salesmen, cities and depot as three different sets of varying cardinalities. We apply a novel technique that combines elements from recent architectures that were developed for sets, as well as elements from graph networks. Coupled with new constraint enforcing output layers, a dedicated loss, and a search method, our solution is shown to outperform all the meta-…

2019-02-15cs.LGstat.ML

Learning Functions over Sets via Permutation Adversarial Networks

Microsoft Research（微软研究院）

In this paper, we consider the problem of learning functions over sets, i.e., functions that are invariant to permutations of input set items. Recent approaches of pooling individual element embeddings can necessitate extremely large embedding sizes for challenging functions. We address this challenge by allowing standard neural networks like LSTMs to succinctly capture the function over the set. However, to ensure invariance with respect to permutations of set elements, we propose a novel architecture called SPAN that simultaneously learns the function as well as adversarial or worst-case per…

2020-01-10cs.LGstat.ML

等变层与不变层：点云分类的骨干网络与池化比较研究

Equivariant vs. Invariant Layers: A Comparison of Backbone and Pooling for Point Cloud Classification

Vanderbilt University（范德堡大学）

该研究针对点云分类任务，探究置换等变骨干网络与置换不变全局池化的相互作用，发现池化层选择对性能影响显著，为设计更优的置换不变集合神经网络提供参考。

2024-07-12cs.CVcs.LG

更多相关论文收起更多论文

掩码注意力池化对填充和排列具有不变性：Lean中的形式证明

Masked Attention Pooling Is Invariant to Padding and Permutation: A Formal Proof in Lean

The Ohio State University（俄亥俄州立大学）

该研究在Lean 4中形式证明了带受保护归一化器的掩码注意力池化对填充和排列具有不变性，并通过jax-lean流水线将该结果关联到JAX内核的Jaxpr。

2026-10-14cs.LG

群不变全局池化

Group Invariant Global Pooling

University of Cambridge（剑桥大学）

本文提出群不变全局池化（GIGP）层，可证明表达一大类不变函数，并在旋转MNIST和QM9上验证了其有效性。

2023-05-30cs.AIcs.CGcs.CV

PiNet: A Permutation Invariant Graph Neural Network for Graph Classification

University College London（伦敦大学学院）；Braintree Ltd.（Braintree公司）

We propose an end-to-end deep learning learning model for graph classification and representation learning that is invariant to permutation of the nodes of the input graphs. We address the challenge of learning a fixed size graph representation for graphs of varying dimensions through a differentiable node attention pooling mechanism. In addition to a theoretical proof of its invariance to permutation, we provide empirical evidence demonstrating the statistically significant gain in accuracy when faced with an isomorphic graph classification task given only a small number of training examples.…

2019-05-08cs.CVcs.LGcs.NE

打破 GNN 读出阶段的对称性瓶颈

Breaking Symmetry Bottlenecks in GNN Readouts

Department of Computing, Imperial College London, UK（伦敦帝国学院计算机系）；Department of Mathematics, Imperial College London, UK（伦敦帝国学院数学系）；London School of Geometry（几何与数论研究所）

本文提出基于投影器的不变读出方法，解决 GNN 读出阶段的对称性瓶颈问题，通过保留非平凡对称性信息提升模型表达能力。

2026-02-05cs.LG

Robust Attentional Aggregation of Deep Feature Sets for Multi-view 3D Reconstruction

Heriot-Watt University（赫瑞瓦特大学）；University of Oxford（牛津大学）

We study the problem of recovering an underlying 3D shape from a set of images. Existing learning based approaches usually resort to recurrent neural nets, e.g., GRU, or intuitive pooling operations, e.g., max/mean poolings, to fuse multiple deep features encoded from input images. However, GRU based approaches are unable to consistently estimate 3D shapes given different permutations of the same set of input images as the recurrent unit is permutation variant. It is also unlikely to refine the 3D shape given more images due to the long-term memory loss of GRU. Commonly used pooling approaches…

2019-08-18cs.AIcs.CVcs.LG

分层与置换不变的特征变换学习：基于策略引导的嵌入搜索

Hierarchical and Permutation-Invariant Feature Transformation Learning via Policy-Guided Embedding Search

University of Kansas（堪萨斯大学）；Southwest University of Finance and Economics（西南财经大学）；University of Wisconsin–Madison（威斯康星大学麦迪逊分校）；University of Auckland（奥克兰大学）；Arizona State University（亚利桑那州立大学）；Northeast Normal University（东北师范大学）

针对表格数据特征变换中的层次关系缺失、置换不变性偏差及梯度搜索不适配问题，提出置换不变分层模块与策略引导多目标强化学习框架，提升预测性能与效率。

2026-09-10cs.AIcs.LG

EDEN: Evolutionary Deep Networks for Efficient Machine Learning

African Institute for Mathematical Sciences（非洲数学科学研究所）；University of Cape Town（开普敦大学）；South African Astronomical Observatory（南非天文台）

Deep neural networks continue to show improved performance with increasing depth, an encouraging trend that implies an explosion in the possible permutations of network architectures and hyperparameters for which there is little intuitive guidance. To address this increasing complexity, we propose Evolutionary DEep Networks (EDEN), a computationally efficient neuro-evolutionary algorithm which interfaces to any deep neural network platform, such as TensorFlow. We show that EDEN evolves simple yet successful architectures built from embedding, 1D and 2D convolutional, max pooling and fully conn…

2017-09-26cs.LGcs.NEstat.ML

Pb-Hash：分块 b-bit 哈希

Pb-Hash: Partitioned b-bit Hashing

LinkedIn Ads（领英广告）

本文提出 Pb-Hash 算法，通过将哈希值分块重复使用以大幅减小模型规模，理论分析与实验表明其在准确率损失可控的前提下，对线性 SVM 和深度学习模型均有效。

2023-06-28cs.DScs.IRcs.LG

SetONet：一种基于集合的运算网络，用于通过变量输入采样求解偏微分方程

SetONet: A Set-Based Operator Network for Solving PDEs with Variable-Input Sampling

organization= Maseeh Department of Civil, Architectural；Environmental Engineering , addressline= University of Texas at Austin , city= Austin , postcode= 78712 , state= TX , country= USA；organization= Oden Institute for Computational Engineering；Sciences , addressline= University of Texas at Austin , city= Austin , postcode= 78712 , state= TX , country= USA

SetONet通过将运算输入重构为无序坐标-值观测集，解决了传统神经运算代理对固定传感器布局的依赖问题，提出SetONet-Key通过可学习查询令牌和位置-only键路径解耦采样几何与传感器值，适用于固定和变量布局及传感器丢失场景。

2026-04-01cs.LG