通过无奖励自我微调代理实现自适应RAN切片控制
Adaptive RAN Slicing Control via Reward-Free Self-Finetuning Agents
AI总结：
本文提出一种无奖励自我微调框架，通过内化经验实现自适应RAN切片控制，优于传统RL和LLM基agent。
摘要：
生成式AI模型融入AI原生网络系统为实现自主和自适应控制提供了变革性的路径。然而，将此类模型应用于连续控制任务受到内在架构限制的阻碍，包括有限的上下文窗口、缺乏显式的奖励信号以及长期上下文的退化。本文认为，解锁稳健连续控制的关键是让智能体通过将经验内化为参数来学习，而不是依赖基于提示的记忆。为此，我们提出了一种新的自我微调框架，使智能体系统能够通过直接与环境互动持续学习，从而无需人工设计的奖励。我们的框架实现了双视角反思机制，生成自主语言反馈，从交互历史中构建偏好数据集。随后的基于偏好的微调过程将长期经验内化到模型参数中。我们评估了我们的方法在动态无线电接入网络（RAN）切片任务上的表现，这是一个具有挑战性的多目标控制问题，需要在波动的网络条件下解决频谱效率、服务质量与重构稳定性之间的尖锐权衡。实验结果表明，我们的框架在样本效率、稳定性和多指标优化方面均优于标准强化学习（RL）基线和现有大语言模型（LLM）基 agent。这些发现展示了自我改进的生成智能体在连续控制任务中的潜力，为未来的AI原生网络基础设施铺平了道路。
英文摘要：
The integration of Generative AI models into AI-native network systems offers a transformative path toward achieving autonomous and adaptive control. However, the application of such models to continuous control tasks is impeded by intrinsic architectural limitations, including finite context windows, the lack of explicit reward signals, and the degradation of the long context. This paper posits that the key to unlocking robust continuous control is enabling agents to internalize experience by distilling it into their parameters, rather than relying on prompt-based memory. To this end, we propose a novel self-finetuning framework that enables agentic systems to learn continuously through direct interaction with the environment, bypassing the need for handcrafted rewards. Our framework implements a bi-perspective reflection mechanism that generates autonomous linguistic feedback to construct preference datasets from interaction history. A subsequent preference-based fine-tuning process distills long-horizon experiences into the model's parameters. We evaluate our approach on a dynamic Radio Access Network (RAN) slicing task, a challenging multi-objective control problem that requires the resolution of acute trade-offs between spectrum efficiency, service quality, and reconfiguration stability under volatile network conditions. Experimental results show that our framework outperforms standard Reinforcement Learning (RL) baselines and existing Large Language Model (LLM)-based agents in sample efficiency, stability, and multi-metric optimization. These findings demonstrate the potential of self-improving generative agents for continuous control tasks, paving the way for future AI-native network infrastructure.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
