arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2409.09653cs.LGcs.AI
收藏

离线强化学习场景下KAN与MLP的对比研究

KAN v.s. MLP for Offline Reinforcement Learning

School of Information, Renmin University of China(中国人民大学信息学院);Key Laboratory of Data Engineering and Knowledge Engineering, Ministry of Education, China(教育部数据工程与知识工程重点实验室);School of Economics and Management, Tsinghua University, China(清华大学经济管理学院)

Haihong Guo, Fengxin Li, Jiao Li, Hongyan Liu

AI总结:

该研究探索将KAN用于离线强化学习的策略与价值网络,在D4RL基准上对比基于KAN和MLP的CQL,证实KAN可用更少参数达到接近MLP的性能,为离线RL基础网络选择提供新选项。

摘要:

Kolmogorov-Arnold Networks(KAN,柯尔莫哥洛夫-阿诺德网络)是机器学习领域新兴的神经网络架构,其能否成为常用的Multi-Layer Perceptions(MLP,多层感知机)的可行替代方案引发了研究界的广泛关注。多个领域的实验表明,基于KAN的机器学习方法即使性能不优于基于MLP的方法,也能达到与之相当的水平,且参数量规模小得多、可解释性更强。本文探究将KAN融入离线强化学习(RL)的策略网络与价值网络的方案,在经典的离线RL基准D4RL上,评估了多种基于KAN和MLP的conservative Q-learning(CQL,保守Q学习)的性能、参数量规模与训练效率。研究表明,KAN能用显著更少的参数达到与常用MLP接近的性能,为我们根据离线RL任务需求选择基础网络提供了新选项。

英文摘要:

Kolmogorov-Arnold Networks (KAN) is an emerging neural network architecture in machine learning. It has greatly interested the research community about whether KAN can be a promising alternative of the commonly used Multi-Layer Perceptions (MLP). Experiments in various fields demonstrated that KAN-based machine learning can achieve comparable if not better performance than MLP-based methods, but with much smaller parameter scales and are more explainable. In this paper, we explore the incorporation of KAN into the actor and critic networks for offline reinforcement learning (RL). We evaluated the performance, parameter scales, and training efficiency of various KAN and MLP based conservative Q-learning (CQL) on the the classical D4RL benchmark for offline RL. Our study demonstrates that KAN can achieve performance close to the commonly used MLP with significantly fewer parameters. This provides us an option to choose the base networks according to the requirements of the offline RL tasks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑