首页/论文/2002.03123

arXiv 2002.031232020-02-08cs.LGstat.ML

Towards a combinatorial characterization of bounded memory learning

University of California San Diego（加利福尼亚大学圣地亚哥分校）

Alon Gonen, Shachar Lovett, Michal Moshkovitz

英文摘要：

Combinatorial dimensions play an important role in the theory of machine learning. For example, VC dimension characterizes PAC learning, SQ dimension characterizes weak learning with statistical queries, and Littlestone dimension characterizes online learning. In this paper we aim to develop combinatorial dimensions that characterize bounded memory learning. We propose a candidate solution for the case of realizable strong learning under a known distribution, based on the SQ dimension of neighboring distributions. We prove both upper and lower bounds for our candidate solution, that match in some regime of parameters. In this parameter regime there is an equivalence between bounded memory and SQ learning. We conjecture that our characterization holds in a much wider regime of parameters.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

阅读原文

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

本文内容

英文摘要相关论文

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

继续探索

相关论文

Combinatorial Multi-armed Bandit with Probabilistically Triggered Arms: A Case with Bounded Regret

Bilkent University（比尔肯特大学）

In this paper, we study the combinatorial multi-armed bandit problem (CMAB) with probabilistically triggered arms (PTAs). Under the assumption that the arm triggering probabilities (ATPs) are positive for all arms, we prove that a class of upper confidence bound (UCB) policies, named Combinatorial UCB with exploration rate $κ$ (CUCB-$κ$), and Combinatorial Thompson Sampling (CTS), which estimates the expected states of the arms via Thompson sampling, achieve bounded regret. In addition, we prove that CUCB-$0$ and CTS incur $O(\sqrt{T})$ gap-independent regret. These results improve the results…

2017-07-24cs.LGstat.ML

Bounded Memory Adversarial Bandits with Composite Anonymous Delayed Feedback

Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）

We study the adversarial bandit problem with composite anonymous delayed feedback. In this setting, losses of an action are split into $d$ components, spreading over consecutive rounds after the action is chosen. And in each round, the algorithm observes the aggregation of losses that come from the latest $d$ rounds. Previous works focus on oblivious adversarial setting, while we investigate the harder non-oblivious setting. We show non-oblivious setting incurs $Ω(T)$ pseudo regret even when the loss sequence is bounded memory. However, we propose a wrapper algorithm which enjoys $o(T)$ policy…

2022-04-28cs.LGstat.ML

监督在线可学习性的组合刻画

A Combinatorial Characterization of Supervised Online Learnability

University of Michigan（密歇根大学）

本文提出序列极小极大维度，紧致刻画了有界损失下假设类的在线可学习性，并统一了现有组合维度。

2024-02-09cs.LG

有界记忆下的极限语言生成

On Language Generation in the Limit with Bounded Memory

Cornell University（康奈尔大学）；Stanford University（斯坦福大学）；Google Research（谷歌研究）

研究有界记忆下语言生成的极限问题，通过组合界和滑动窗口分析记忆约束对可生成性、密度和识别的影响。

2026-05-28cs.AIcs.CLcs.DS

基于Frank-Wolfe优化的带记忆高效在线学习：具有有界动态遗憾的算法及控制应用

Efficient Online Learning with Memory via Frank-Wolfe Optimization: Algorithms with Bounded Dynamic Regret and Applications to Control

University of Michigan（密歇根大学）

针对在线学习中投影操作的计算瓶颈，提出基于OFW和Hedge的无投影带记忆元-基学习算法，实现有界动态遗憾，并成功应用于线性时变系统的在线控制。

2023-03-31cs.LGcs.SYeess.SY

A General Memory-Bounded Learning Algorithm

Edmond and Lily Safra Center for Brain Sciences（埃德蒙与莉莉·萨夫拉脑科学中心）；The Hebrew University（希伯来大学）

Designing bounded-memory algorithms is becoming increasingly important nowadays. Previous works studying bounded-memory algorithms focused on proving impossibility results, while the design of bounded-memory algorithms was left relatively unexplored. To remedy this situation, in this work we design a general bounded-memory learning algorithm, when the underlying distribution is known. The core idea of the algorithm is not to save the exact example received, but only a few important bits that give sufficient information. This algorithm applies to any hypothesis class that has an "anti-mixing" p…

2019-10-11cs.LG

更多相关论文收起更多论文

MiLoop：面向神经组合优化的选择性记忆传播

MiLoop: Selective Memory Propagation for Neural Combinatorial Optimization

Southern University of Science and Technology（南方科技大学）；Guangdong Provincial Key Laboratory of Fully Actuated System Control Theory and Technology（广东省全驱系统控制理论与技术重点实验室）；Pengcheng Laboratory（鹏城实验室）

MiLoop提出纯强化学习的构造性框架，通过选择性记忆传播实现动态嵌入，无需解标签或剪枝，在100至千万节点实例上展现强泛化能力。

2026-10-02cs.LG

A Note on Bounding Regret of the C$^2$UCB Contextual Combinatorial Bandit

University of Melbourne（墨尔本大学）

We revisit the proof by Qin et al. (2014) of bounded regret of the C$^2$UCB contextual combinatorial bandit. We demonstrate an error in the proof of volumetric expansion of the moment matrix, used in upper bounding a function of context vector norms. We prove a relaxed inequality that yields the originally-stated regret bound.

2019-02-20cs.LGstat.ML

Bounded Memory Active Learning through Enriched Queries

UCSD（加州大学圣地亚哥分校）；Qualcomm Institute（高通研究所）

The explosive growth of easily-accessible unlabeled data has lead to growing interest in active learning, a paradigm in which data-hungry learning algorithms adaptively select informative examples in order to lower prohibitively expensive labeling costs. Unfortunately, in standard worst-case models of learning, the active setting often provides no improvement over non-adaptive algorithms. To combat this, a series of recent works have considered a model in which the learner may ask enriched queries beyond labels. While such models have seen success in drastically lowering label costs, they tend…

2021-02-09cs.LGstat.ML

A Combinatorial Perspective on Transfer Learning

DeepMind

Human intelligence is characterized not only by the capacity to learn complex skills, but the ability to rapidly adapt and acquire new skills within an ever-changing environment. In this work we study how the learning of modular solutions can allow for effective generalization to both unseen and potentially differently distributed data. Our main postulate is that the combination of task segmentation, modular learning and memory-based ensembling can give rise to generalization on an exponentially growing number of unseen tasks. We provide a concrete instantiation of this idea using a combinatio…

2020-10-23cs.LGstat.ML

MawForge：用于局部专家混合推理的内存受限专家实例化

MawForge: Memory-Bounded Expert Materialization for Local Mixture-of-Experts Inference

Holocron Security, Inc（全息安全公司）

研究局部MoE推理在受限内存机器上的可行性，提出MawForge方法，通过磁盘存储模型、按需实例化专家张量等实现，发现其可作为有界执行机制，但非缓存最大化策略，性能受多种因素影响。

2026-07-14cs.LG

Learning to Ignore: Long Document Coreference with Bounded Memory Neural Networks

Toyota Technological Institute at Chicago（芝加哥丰田技术研究所）；University of Chicago（芝加哥大学）

Long document coreference resolution remains a challenging task due to the large memory and runtime requirements of current models. Recent work doing incremental coreference resolution using just the global representation of entities shows practical benefits but requires keeping all entities in memory, which can be impractical for long documents. We argue that keeping all entities in memory is unnecessary, and we propose a memory-augmented neural network that tracks only a small bounded number of entities at a time, thus guaranteeing a linear runtime in length of document. We show that (a) the…

2020-11-17cs.CLcs.LG

重新思考神经组合优化中的效率：基于Mamba的批量偏好优化

Rethinking Efficiency in Neural Combinatorial Optimization: Batched Preference Optimization with Mamba

National Key Laboratory of Big Data and Decision, National University of Defense Technology（大数据与决策国家重点实验室，国防科技大学）；School of Computer Science, South China University of Technology（华南理工大学计算机科学学院）；School of Computing and Information Systems, Singapore Management University（新加坡管理学院计算机与信息系统学院）

本文提出ECO框架，通过分阶段的监督预热和批量偏好优化，提升神经组合优化的效率，实现更优的内存使用和吞吐量。

2026-07-21cs.LG

面向持续异常检测的内存受限贪心采样的延续方法

Memory-Bounded Continuation of Greedy Sampling for Continual Anomaly Detection

Northeastern University（东北大学）；AIVEX Inc.（AIVEX公司）；Mitsubishi Electric Research Laboratories (MERL)（三菱电机研究实验室）

该研究针对持续异常检测提出ContCore方法，通过延续贪心采样构建固定内存的核心集，在MVTecAD、VisA数据集的11种任务调度及在线设置下实现最优性能。

2026-08-18cs.CVcs.LG

Learning Languages in the Limit from Positive Information with Finitely Many Memory Changes

We investigate learning collections of languages from texts by an inductive inference machine with access to the current datum and a bounded memory in form of states. Such a bounded memory states (BMS) learner is considered successful in case it eventually settles on a correct hypothesis while exploiting only finitely many different states. We give the complete map of all pairwise relations for an established collection of criteria of successfull learning. Most prominently, we show that non-U-shapedness is not restrictive, while conservativeness and (strong) monotonicity are. Some results carr…

2021-06-17cs.FLcs.LG

带界内存门的稳定自调制量子快速权重编程器

Stable Self-Modulating Quantum Fast-Weight Programmers with Bounded Memory Gates

National Taiwan University（国立台湾大学）；Stevens Institute of Technology（史蒂文斯理工学院）；Seoul National University（首尔国立大学）；Brookhaven National Laboratory（布鲁克海文国家实验室）；Imperial College London（伦敦帝国理工学院）；Wells Fargo（沃尔士·法戈）

提出带界旧态调制规则的自调制量子快速权重编程器，消除长序列发散并提升鲁棒性，在量子动力学预测和电信活动预测任务上验证了其有效性。

2026-07-03cs.AIcs.ETcs.LG