PyTupli：面向协作式离线强化学习项目的可扩展基础设施
PyTupli: A Scalable Infrastructure for Collaborative Offline Reinforcement Learning Projects
AI总结：
PyTupli是一个Python基础设施，用于创建、存储和共享离线强化学习基准及元组数据集，支持细粒度过滤与安全部署，促进协作、可复现和可扩展的研究。
摘要：
离线强化学习（RL）作为一种从预先收集的数据中学习控制策略的强大范式已获得广泛关注，它消除了对昂贵或高风险在线交互的需求。尽管许多开源库提供了离线强化学习算法的稳健实现，但它们都依赖于由状态、动作、下一状态和奖励组成经验元组的数据集。管理、整理和分发此类数据集需要合适的基础设施。虽然已有针对既定基准问题的静态数据集，但尚无标准化或可扩展的解决方案来支持开发和共享针对新颖或用户自定义基准的数据集。为弥补这一空白，我们引入了PyTupli，一个基于Python的工具，旨在简化基准环境及其对应元组数据集的创建、存储和传播。PyTupli包含一个轻量级客户端库，具有用于上传和检索基准及数据的定义接口。它支持在回合级和元组级进行细粒度过滤，使研究人员能够策划高质量、任务特定的数据集。一个容器化的服务器组件支持生产级部署，具备认证、访问控制和自动化证书配置以确保安全使用。通过解决数据集基础设施中的关键障碍，PyTupli促进了更具协作性、可复现性和可扩展性的离线强化学习研究。
英文摘要：
Offline reinforcement learning (RL) has gained traction as a powerful paradigm for learning control policies from pre-collected data, eliminating the need for costly or risky online interactions. While many open-source libraries offer robust implementations of offline RL algorithms, they all rely on datasets composed of experience tuples consisting of state, action, next state, and reward. Managing, curating, and distributing such datasets requires suitable infrastructure. Although static datasets exist for established benchmark problems, no standardized or scalable solution supports developing and sharing datasets for novel or user-defined benchmarks. To address this gap, we introduce PyTupli, a Python-based tool to streamline the creation, storage, and dissemination of benchmark environments and their corresponding tuple datasets. PyTupli includes a lightweight client library with defined interfaces for uploading and retrieving benchmarks and data. It supports fine-grained filtering at both the episode and tuple level, allowing researchers to curate high-quality, task-specific datasets. A containerized server component enables production-ready deployment with authentication, access control, and automated certificate provisioning for secure use. By addressing key barriers in dataset infrastructure, PyTupli facilitates more collaborative, reproducible, and scalable offline RL research.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
