首页/论文/2008.06952

arXiv 2008.069522020-08cs.LGstat.ML

A Functional Perspective on Learning Symmetric Functions with Neural Networks

Courant Institute of Mathematical Sciences, New York University（纽约大学库朗数学科学研究所）；Center for Data Science, New York University（纽约大学数据科学中心）

Aaron Zweig, Joan Bruna

备注：: Accepted to ICML 2021

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

英文摘要相关论文

英文摘要：

Symmetric functions, which take as input an unordered, fixed-size set, are known to be universally representable by neural networks that enforce permutation invariance. These architectures only give guarantees for fixed input sizes, yet in many practical applications, including point clouds and particle physics, a relevant notion of generalization should include varying the input size. In this work we treat symmetric functions (of any size) as functions over probability measures, and study the learning and representation of neural networks defined on measures. By focusing on shallow architectures, we establish approximation and generalization bounds under different choices of regularization (such as RKHS and variation norms), that capture a hierarchy of functional spaces with increasing degree of non-linear learning. The resulting models can be learned efficiently and enjoy generalization guarantees that extend across input sizes, as we verify empirically.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

阅读原文

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

本文内容

英文摘要相关论文

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

继续探索

相关论文

Local moment matching: A unified methodology for symmetric functional estimation and distribution estimation under Wasserstein distance

Stanford University（斯坦福大学）

We present \emph{Local Moment Matching (LMM)}, a unified methodology for symmetric functional estimation and distribution estimation under Wasserstein distance. We construct an efficiently computable estimator that achieves the minimax rates in estimating the distribution up to permutation, and show that the plug-in approach of our unlabeled distribution estimator is "universal" in estimating symmetric functionals of discrete distributions. Instead of doing best polynomial approximation explicitly as in existing literature of functional estimation, the plug-in approach conducts polynomial appr…

2018-06-26cs.ITcs.LGmath.IT

Probability Functional Descent: A Unifying Perspective on GANs, Variational Inference, and Reinforcement Learning

Institute for Computational & Mathematical Engineering, Stanford University（斯坦福大学计算与数学工程研究所）；Management Science & Engineering, Stanford University（斯坦福大学管理科学与工程系）

This paper provides a unifying view of a wide range of problems of interest in machine learning by framing them as the minimization of functionals defined on the space of probability measures. In particular, we show that generative adversarial networks, variational inference, and actor-critic methods in reinforcement learning can all be seen through the lens of our framework. We then discuss a generic optimization algorithm for our formulation, called probability functional descent (PFD), and show how this algorithm recovers existing methods developed independently in the settings mentioned ea…

2019-05-19cs.LGstat.ML

可靠机器学习的对称损失视角

A Symmetric Loss Perspective of Reliable Machine Learning

The University of Tokyo（东京大学）；RIKEN AIP（理化学研究所人工智能研究中心）

本文综述对称损失在损坏标签二分类中的鲁棒作用，涵盖BER最小化、AUC最大化及关键词与未标注文档学习，并展望可靠机器学习中的损失设计。

2023-06-05cs.LGstat.ML

A geometric perspective on functional outlier detection

Ludwig-Maximilians-University（慕尼黑大学）

We consider functional outlier detection from a geometric perspective, specifically: for functional data sets drawn from a functional manifold which is defined by the data's modes of variation in amplitude and phase. Based on this manifold, we develop a conceptualization of functional outlier detection that is more widely applicable and realistic than previously proposed. Our theoretical and experimental analyses demonstrate several important advantages of this perspective: It considerably improves theoretical understanding and allows to describe and analyse complex functional outlier scenario…

2021-09-14cs.LGstat.COstat.ML

Robust and Scalable SDE Learning: A Functional Perspective

Oxford University（牛津大学）；InstaDeep Ltd.（InstaDeep公司）；Discovery Insure（Discovery Insure公司）

Stochastic differential equations provide a rich class of flexible generative models, capable of describing a wide range of spatio-temporal processes. A host of recent work looks to learn data-representing SDEs, using neural networks and other flexible function approximators. Despite these advances, learning remains computationally expensive due to the sequential nature of SDE integrators. In this work, we propose an importance-sampling estimator for probabilities of observations of SDEs for the purposes of learning. Crucially, the approach we suggest does not rely on such integrators. The pro…

2021-10-11cs.LGstat.ML

对称正定流形上函数型数据的几何特征学习

Geometric Feature Learning for Functional Data Valued on the Symmetric Positive Definite Manifold

Trinity College Dublin（都柏林圣三一学院）

本文提出函数型神经网络MatFAE，用于学习SPD流形上的轨迹，通过内在层和函数层编码动态信息，并在fMRI数据上验证了其高效性与可解释性。

2026-09-28cs.LGstat.ML

更多相关论文收起更多论文

Functional Regularization for Representation Learning: A Unified Theoretical Perspective

Amazon Alexa AI Search（亚马逊Alexa AI搜索）；Department of Computer Sciences（计算机科学系）；University of Wisconsin-Madison（威斯康星大学麦迪逊分校）；Manhattan Beach, CA, USA（美国加利福尼亚州曼哈顿海滩（此处为地址，按要求不提取））

Unsupervised and self-supervised learning approaches have become a crucial tool to learn representations for downstream prediction tasks. While these approaches are widely used in practice and achieve impressive empirical gains, their theoretical understanding largely lags behind. Towards bridging this gap, we present a unifying perspective where several such approaches can be viewed as imposing a regularization on the representation via a learnable function using unlabeled data. We propose a discriminative theoretical framework for analyzing the sample complexity of these approaches, which ge…

2020-10-22cs.LGstat.ML

A Functional Information Perspective on Model Interpretation

Technion - Israel Institute of Technology（以色列理工学院）

Contemporary predictive models are hard to interpret as their deep nets exploit numerous complex relations between input elements. This work suggests a theoretical framework for model interpretability by measuring the contribution of relevant features to the functional entropy of the network with respect to the input. We rely on the log-Sobolev inequality that bounds the functional entropy by the functional Fisher information with respect to the covariance of the data. This provides a principled way to measure the amount of information contribution of a subset of features to the decision funct…

2022-06-14cs.LG

Sparse Symmetric Tensor Regression for Functional Connectivity Analysis

Division of Biostatistics, University of California, Berkeley（加州大学伯克利分校生物统计学系）

Tensor regression models, such as CP regression and Tucker regression, have many successful applications in neuroimaging analysis where the covariates are of ultrahigh dimensionality and possess complex spatial structures. The high-dimensional covariate arrays, also known as tensors, can be approximated by low-rank structures and fit into the generalized linear models. The resulting tensor regression achieves a significant reduction in dimensionality while remaining efficient in estimation and prediction. Brain functional connectivity is an essential measure of brain activity and has shown sig…

2020-10-28cs.LGstat.ML

On the Hardness of Robustness Transfer: A Perspective from Rademacher Complexity over Symmetric Difference Hypothesis Space

The Pennsylvania State University（宾夕法尼亚州立大学）；Michigan State University（密歇根州立大学）；Sony AI（索尼人工智能研究院）

Recent studies demonstrated that the adversarially robust learning under $\ell_\infty$ attack is harder to generalize to different domains than standard domain adaptation. How to transfer robustness across different domains has been a key question in domain adaptation field. To investigate the fundamental difficulty behind adversarially robust domain adaptation (or robustness transfer), we propose to analyze a key complexity measure that controls the cross-domain generalization: the adversarial Rademacher complexity over {\em symmetric difference hypothesis space} $\mathcal{H} Δ\mathcal{H}$. F…

2023-02-23cs.LGstat.ML

重思对称矩阵分解：一种更通用且更优的聚类视角

Rethinking Symmetric Matrix Factorization: A More General and Better Clustering Perspective

Clemson University（克莱姆森大学）

针对现有对称非负矩阵分解算法约束单一、缺乏有效正则项的问题，本文提出带正则项的无强制非负约束的高效对称矩阵分解算法，并构建可适配因子矩阵不同约束的通用框架，提升聚类性能。

2023-11-03cs.AIcs.LG

通过对称注意力分解平衡扩散模型中的保真度与多样性：Hopfield视角

Balancing Fidelity and Diversity in Diffusion Models via Symmetric Attention Decomposition: Hopfield Perspective

Department of Electrical Engineering, Korea University, Seoul, South Korea（韩国大学电子工程系，首尔，韩国）

本文通过将Transformer中的注意力矩阵分解为对称和反对称部分，从Hopfield网络视角解释并调控扩散模型生成中的保真度-多样性权衡。

2026-05-26cs.AIcs.LG

功能单元：材料科学研究范式的全新视角

Functional Unit: A New Perspective on Materials Science Research Paradigms

本文提出功能单元概念，旨在解决材料科学从传统范式向数据驱动AI范式转变过程中结构-性质相关性理解的不足，并探讨其在多尺度材料设计中的应用与挑战。

2025-03-11cond-mat.mtrl-scics.LG

超越裁剪的梯度塑形：更新幅度控制的函数视角

Gradient Shaping Beyond Clipping: A Functional Perspective on Update Magnitude Control

Columbia University（哥伦比亚大学）；Hebei Institute of Communications（河北通信学院）

针对梯度裁剪固定阈值限制灵活性、忽略梯度动态的问题，提出SPAMP框架将裁剪推广为可微的逐层梯度塑形，重定义裁剪与热身为更新规模控制双重机制，在图像和语言任务上提升了训练稳定性、收敛性与鲁棒性。

2025-10-02cs.LG

论评估大语言模型作为函数逼近器的能力：贝叶斯视角

On Evaluating LLMs' Capabilities as Functional Approximators: A Bayesian Perspective

University of Cambridge（剑桥大学）；Microsoft（微软公司）；The Alan Turing Institute（艾伦·图灵研究所）

本研究从贝叶斯视角出发提出新评估框架，发现大语言模型函数建模时弱于原始数据模式理解、长于利用领域先验知识掌握底层函数，明确了其在该任务中的优劣势。

2024-10-06cs.AIcs.LG

从Eckart-Young-Schmidt视角看深度对称自编码器

Deep Symmetric Autoencoders from the Eckart-Young-Schmidt Perspective

该研究从Eckart-Young-Schmidt定理视角分析对称自编码器的数学特性，提出基于迭代奇异值分解的EYS初始化策略，并通过实验验证其相较传统深度自编码器的优势，完善了深度自编码器的表达能力理论基础。

2026-10-02cs.LGcs.NAmath.NA

多层分布外检测的函数型数据视角与基线

A Functional Data Perspective and Baseline On Multi-Layer Out-of-Distribution Detection

Laboratoire des signaux et systèmes (L2S)（信号与系统实验室（L2S））；Université Paris-Saclay（巴黎萨克雷大学）；CNRS（法国国家科学研究中心）；CentraleSupélec（巴黎中央理工-高等电力学院）；MICS（数学与计算机科学实验室（MICS））；Inria（法国国家信息与自动化研究所）；CEA（法国原子能和替代能源委员会）；Owkin（Owkin公司）；International Laboratory on Learning Systems (ILLS)（国际学习系统实验室（ILLS））

本文从函数型数据视角利用样本跨层轨迹及统计依赖进行分布外检测，提出函数型异常检测基线，并在视觉基准上验证其优于强基线。

2023-06-06cs.CVcs.LGstat.ML

从功能角度审视神经网络中的知识蒸馏

A Functional Perspective on Knowledge Distillation in Neural Networks

本文从功能角度研究知识蒸馏的压缩能力与知识传递，通过实验协议分析不同数据模态下的知识转移机制，发现知识蒸馏更像数据依赖的正则化方法，存在负向非对称转移风险。

2026-03-16cs.AIcs.LG

穿越非紧对称空间：Cartan Neural Networks的数学视角

Navigation through Non-Compact Symmetric Spaces: a mathematical perspective on Cartan Neural Networks

Dipartimento di Fisica, Università di Torino（都灵大学物理系）；Additati&Partners Consulting s.r.l（Additati&Partners咨询公司）；SISSA (Scuola Internazionale Superiore di Studi Avanzati)（国际先进研究高等学院）；INFN, Sezione di Torino（意大利国家核物理研究所，都灵分部）

本文从数学角度阐释 Cartan Neural Networks，分析非紧对称空间上网络层的几何性质及层间映射，说明其协变性与几何可解释性。

2025-07-22cs.LGhep-th