OpenTinker：在智能体强化学习中分离关注点
OpenTinker: Separating Concerns in Agentic Reinforcement Learning
AI总结：
介绍OpenTinker，一种用于训练大型语言模型智能体的开放基础设施。通过共享执行资源和多LoRA策略，围绕策略生命周期运行，支持SFT和RL，实现多用户等共享基础模型并隔离更新等。
摘要：
我们引入了OpenTinker，这是一个用于在共享执行资源上训练具有许多LoRA支持策略的大型语言模型（LLM）智能体的开放基础设施。现代智能体工作负载混合了监督微调（SFT）、在线强化学习（RL）、展开生成、验证和多轮环境交互。在这样的工作负载中，LoRA适配器不是静态推理工件……
英文摘要：
We introduce \textsc{OpenTinker}, an open infrastructure for training large language model (LLM) agents with many LoRA-backed policies over shared execution resources. Modern agent workloads mix supervised fine-tuning (SFT), online reinforcement learning (RL), rollout generation, validation, and multi-turn environment interaction. In such workloads, LoRA adapters are not static inference artifacts: they are frequently updated policy states whose optimizer state, rollout snapshot, and training data attribution must remain consistent. \textsc{OpenTinker} centers the runtime around this policy lifecycle. Users define environments, agents, and learning objectives, while the system manages training clients, rollout samplers, checkpoint handles, and policy-version refresh. The same data path supports SFT and RL by converting trajectories into token sequences with explicit masks: context and environment observations condition the model, while generated action tokens carry supervised weights or RL advantages. This design enables multi-LoRA SFT/RL training in which many users, tasks, or agents can share a base model while keeping adapter updates, checkpoints, and rollout snapshots isolated. We describe the system architecture, the adapter lifecycle, the service-backed snapshot handoff used by the current implementation, the backend contract for mixed-adapter rollout kernels, and the training scheduler that isolates adapter-local gradients and optimizer state. Representative validation tasks exercise single-turn, multi-turn, LoRA, and multi-agent agentic training.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
