无交互逆强化学习：一种数据驱动的对齐框架
Interactionless Inverse Reinforcement Learning: A Data-Centric Framework for Durable Alignment
AI总结：
本文提出无交互逆强化学习框架，旨在通过数据驱动的方法学习可检查、可编辑和可重用的奖励特征，从而将对齐问题从一次性训练成本转变为可验证的工程资产。
摘要：
AI对齐问题日益重要，但许多现有方法通过直接修改策略参数来学习安全行为，将规范约束与底层策略纠缠在一起。这通常导致不透明、难以编辑的对齐特征，并减少其在不同模型或部署中的再利用，我们称之为对齐浪费。我们提出了无交互逆强化学习，一种用于学习可检查、可编辑和可重用的奖励特征的框架，这些特征与策略优化分离。我们进一步引入了对齐飞轮，一种人机协作的生命周期，通过自动化评估和细化，迭代审计、修补和强化这些特征。共同，这些想法将对齐从一次性训练成本转变为可验证的工程资产。
英文摘要：
AI alignment is growing in importance, yet many current approaches learn safety behavior by directly modifying policy parameters, entangling normative constraints with the underlying policy. This often yields opaque, difficult-to-edit alignment artifacts and reduces their reuse across models or deployments, a failure mode we term Alignment Waste. We propose Interactionless Inverse Reinforcement Learning, a framework for learning inspectable, editable, and reusable reward artifacts separately from policy optimization. We further introduce the Alignment Flywheel, a human-in-the-loop lifecycle for iteratively auditing, patching, and hardening these artifacts through automated evaluation and refinement. Together, these ideas recast alignment from a disposable training expense into a durable, verifiable engineering asset.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
