arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2601.08271cs.AI
收藏

稀疏性是必要的:大规模动作空间中代理LLM的多项式时间稳定性

Sparsity Is Necessary: Polynomial-Time Stability for Agentic LLMs in Large Action Spaces

Angshul Majumdar(独立研究者)

Angshul Majumdar

AI总结:

研究提出稀疏代理控制框架,通过多项式时间稳定性理论解决大规模动作空间中LLM的稀疏性问题,证明稀疏表示在策略学习中的有效性及密集策略类的样本需求。

摘要:

工具增强的LLM系统暴露了一个学习理论长期忽视的控制领域:在具有大量离散动作宇宙(工具、API、文档)的序列决策中,只有小的、未知的子集对任何固定任务分布相关。我们正式将此设定为稀疏代理控制(SAC),其中策略允许在M>>1个动作上具有块稀疏表示,奖励取决于稀疏主效应(可选地稀疏协同效应)。我们通过凸替代研究了ell_{1,2}-正则化策略学习,建立了尖锐的压缩感知式结果:(i)估计和价值次优程度在Policy-RSC条件下按k(log M/T)^{1/2}缩放;(ii)当T>k log M且不相干和beta-min时,通过对偶见证论证可以精确恢复工具支持;(iii)任何密集策略类需要Omega(M)样本,解释了仅提示控制器的不稳定性。我们进一步表明,在部分可观测性下,LLM仅通过信念/表示误差epsilon_b起作用,导致附加的O(epsilon_b)退化,同时保持对M的对数依赖性。扩展涵盖无调优、在线、鲁棒、组稀疏和交互感知的SAC。

英文摘要:

Tool-augmented LLM systems expose a control regime that learning theory has largely ignored: sequential decision-making with a massive discrete action universe (tools, APIs, documents) in which only a small, unknown subset is relevant for any fixed task distribution. We formalize this setting as Sparse Agentic Control (SAC), where policies admit block-sparse representations over M >> 1 actions and rewards depend on sparse main effects and (optionally) sparse synergies. We study ell_{1,2}-regularized policy learning through a convex surrogate and establish sharp, compressed-sensing-style results: (i) estimation and value suboptimality scale as k (log M / T)^{1/2} under a Policy-RSC condition; (ii) exact tool-support recovery holds via primal-dual witness arguments when T > k log M under incoherence and beta-min; and (iii) any dense policy class requires Omega(M) samples, explaining the instability of prompt-only controllers. We further show that under partial observability, LLMs matter only through a belief/representation error epsilon_b, yielding an additive O(epsilon_b) degradation while preserving logarithmic dependence on M. Extensions cover tuning-free, online, robust, group-sparse, and interaction-aware SAC.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑