首页/论文/2002.11519

arXiv 2002.115192020-02-26cs.LGcs.LOstat.ML

Decidability of Sample Complexity of PAC Learning in finite setting

NYU Abu Dhabi（纽约大学阿布扎比）

Alberto Gandolfi

英文摘要：

In this short note we observe that the sample complexity of PAC machine learning of various concepts, including learning the maximum (EMX), can be exactly determined when the support of the probability measures considered as models satisfies an a-priori bound. This result contrasts with the recently discovered undecidability of EMX within ZFC for finitely supported probabilities (with no a priori bound). Unfortunately, the decision procedure is at present, at least doubly exponential in the number of points times the uniform bound on the support size.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

阅读原文

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

本文内容

英文摘要相关论文

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

继续探索

相关论文

PAC-learning is Undecidable

The problem of attempting to learn the mapping between data and labels is the crux of any machine learning task. It is, therefore, of interest to the machine learning community on practical as well as theoretical counts to consider the existence of a test or criterion for deciding the feasibility of attempting to learn. We investigate the existence of such a criterion in the setting of PAC-learning, basing the feasibility solely on whether the mapping to be learnt lends itself to approximation by a given class of hypothesis functions. We show that no such criterion exists, exposing a fundament…

2022-10-20cs.LGstat.ML

The Optimal Sample Complexity of PAC Learning

This work establishes a new upper bound on the number of samples sufficient for PAC learning in the realizable case. The bound matches known lower bounds up to numerical constant factors. This solves a long-standing open problem on the sample complexity of PAC learning. The technique and analysis build on a recent breakthrough by Hans Simon.

2016-02-07cs.LGstat.ML

A Tight Excess Risk Bound via a Unified PAC-Bayesian-Rademacher-Shtarkov-MDL Complexity

Centrum Wiskunde & Informatica（荷兰数学与计算机科学中心）；Leiden University（莱顿大学）；University of Victoria（维多利亚大学）

We present a novel notion of complexity that interpolates between and generalizes some classic existing complexity notions in learning theory: for estimators like empirical risk minimization (ERM) with arbitrary bounded losses, it is upper bounded in terms of data-independent Rademacher complexity; for generalized Bayesian estimators, it is upper bounded by the data-dependent information complexity (also known as stochastic or PAC-Bayesian, $\mathrm{KL}(\text{posterior} \operatorname{\|} \text{prior})$ complexity. For (penalized) ERM, the new complexity reduces to (generalized) normalized maxi…

2017-10-21cs.LGstat.ML

From PAC to Instance-Optimal Sample Complexity in the Plackett-Luce Model

Indian Institute of Science（印度科学研究所）

We consider PAC-learning a good item from $k$-subsetwise feedback information sampled from a Plackett-Luce probability model, with instance-dependent sample complexity performance. In the setting where subsets of a fixed size can be tested and top-ranked feedback is made available to the learner, we give an algorithm with optimal instance-dependent sample complexity, for PAC best arm identification, of $O\bigg(\frac{θ_{[k]}}{k}\sum_{i = 2}^n\max\Big(1,\frac{1}{Δ_i^2}\Big) \ln\frac{k}δ\Big(\ln \frac{1}{Δ_i}\Big)\bigg)$, $Δ_i$ being the Plackett-Luce parameter gap between the best and the $i^{th…

2020-02-26cs.LGstat.ML

利用PAC-Bayes理论与Gibbs分布推导带复杂度度量的泛化界

Leveraging PAC-Bayes Theory and Gibbs Distributions for Generalization Bounds with Complexity Measures

Univ Rennes（雷恩大学）；Inria（法国国家信息与自动化研究所）；CNRS（法国国家科学研究中心）；IRISA（雷恩信息与自动化研究所）；Université Jean Monnet Saint-Etienne（圣艾蒂安让·莫奈大学）；Institut d Optique Graduate School（光学研究所研究生院）；Laboratoire Hubert Curien（于贝尔·居里实验室）；Institut Universitaire de France (IUF)（法国大学研究院）；ServiceNow Research（ServiceNow研究院）

本文利用分解PAC-Bayes界与Gibbs分布，提出可适配任意复杂度度量的通用泛化界，使复杂度能随假设类与任务灵活调整。

2024-02-19cs.LGstat.ML

Fast-rate PAC-Bayes Generalization Bounds via Shifted Rademacher Processes

University of Toronto（多伦多大学）；Vector Institute（向量研究所）

The developments of Rademacher complexity and PAC-Bayesian theory have been largely independent. One exception is the PAC-Bayes theorem of Kakade, Sridharan, and Tewari (2008), which is established via Rademacher complexity theory by viewing Gibbs classifiers as linear operators. The goal of this paper is to extend this bridge between Rademacher complexity and state-of-the-art PAC-Bayesian theory. We first demonstrate that one can match the fast rate of Catoni's PAC-Bayes bounds (Catoni, 2007) using shifted Rademacher processes (Wegkamp, 2003; Lecué and Mitchell, 2012; Zhivotovskiy and Hanneke…

2019-12-01cs.LGmath.STstat.ML

更多相关论文收起更多论文

Improved Algorithms for Collaborative PAC Learning

Northeastern University（东北大学）

We study a recent model of collaborative PAC learning where $k$ players with $k$ different tasks collaborate to learn a single classifier that works for all tasks. Previous work showed that when there is a classifier that has very small error on all tasks, there is a collaborative algorithm that finds a single classifier for all tasks and has $O((\ln (k))^2)$ times the worst-case sample complexity for learning a single task. In this work, we design new algorithms for both the realizable and the non-realizable setting, having sample complexity only $O(\ln (k))$ times the worst-case sample compl…

2018-10-30cs.DScs.LGstat.ML

On the complexity of PAC learning in Hilbert spaces

Bosch Center for Artificial Intelligence（博世人工智能中心）

We study the problem of binary classification from the point of view of learning convex polyhedra in Hilbert spaces, to which one can reduce any binary classification problem. The problem of learning convex polyhedra in finite-dimensional spaces is sufficiently well studied in the literature. We generalize this problem to that in a Hilbert space and propose an algorithm for learning a polyhedron which correctly classifies at least $1- \varepsilon$ of the distribution, with a probability of at least $1 - δ,$ where $\varepsilon$ and $δ$ are given parameters. Also, as a corollary, we improve some…

2023-03-03cs.LGmath.FAstat.ML

可复制可实现PAC学习的样本复杂性

The Sample Complexity of Replicable Realizable PAC Learning

Computer Science Department, Aarhus University（奥胡斯大学计算机科学系）；Stanford University（斯坦福大学）

本文研究了可复制可实现PAC学习的样本复杂性，提出了一个下界并展示了几乎匹配的上界，通过分析Cayley图的谱性质得出结果。

2026-02-23cs.CCcs.DScs.LG

超越参数空间的PAC-贝叶斯：行为等价性、Z-信息与精确复杂度分解

PAC-Bayes Beyond Parameter Space: Behavioral Equivalence, Z-Information, and Exact Complexity Decomposition

The Pennsylvania State University（宾夕法尼亚州立大学）；Artificial Intelligence Research Laboratory（人工智能研究实验室）

该研究将PAC-贝叶斯复杂度的对象从参数空间扩展到预测行为，通过测度分解实现经典PAC-贝叶斯KL散度的精确分解，定义了Z-信息并揭示了行为选择项的变分特性。

2026-08-13cs.AIcs.LG

强化学习的PAC保证：样本复杂性、覆盖和结构

PAC Guarantees for Reinforcement Learning: Sample Complexity, Coverage, and Structure

Independent Researcher（独立研究者）

本文提出CSO框架，用于分析强化学习中PAC保证的样本复杂性、覆盖和结构因素，提供实践工具和开放问题列表。

2026-03-01cs.LGstat.ML

Low-complexity modular policies: learning to play Pac-Man and a new framework beyond MDPs

In this paper we propose a method that learns to play Pac-Man. We define a set of high-level observation and action modules. Actions are temporally extended, and multiple action modules may be in effect concurrently. A decision of the agent is represented as a rule-based policy. For learning, we apply the cross-entropy method, a recent global optimization algorithm. The learned policies reached better score than the hand-crafted policy, and neared the score of average human players. We argue that learning is successful mainly because (i) the policy space includes the combination of individual …

2006-10-30cs.AIcs.LG