可控用户模拟
Controllable User Simulation
AI总结：
本文提出将可控模拟视为因果推断问题，通过改进模拟器训练方法消除 lookahead 偏差，实现更稳定的评估性能。
摘要：
使用离线数据集评估对话代理往往无法覆盖罕见场景或支持新政策测试。为此，本文将可控模拟形式化为因果推断问题，通过将自然语言评估与 off-policy 评估方法结合，证明传统通过监督微调生成轨迹标签训练模拟器会引入结构偏差。进一步证明在策略转移下，这种失败会导致评估指标方差几何级爆炸，即可控性崩溃。为恢复因果一致性，本文建立了准确模拟的理论条件，并提出实用训练缓解措施：事前控制、分步动态控制和直接策略条件学习。实验证明，尽管传统全局控制会扭曲对话分布并破坏行为多样性，我们的因果基础模拟器能消除 lookahead 偏差，保留自然方差，并在未见代理行为下表现出鲁棒的零样本泛化能力。
英文摘要：
Using offline datasets to evaluate conversational agents often fails to cover rare scenarios or to support testing new policies. This has motivated the use of controllable user simulators for targeted, counterfactual evaluation, typically implemented by prompting or fine-tuning large language models. In this work, we formalize controllable simulation as a causal inference problem. By bridging natural language evaluation with off-policy evaluation methodology, we show that the standard practice of training simulators via supervised fine-tuning on post-hoc trajectory labels yields a structurally biased model. Specifically, these labels are inextricably coupled to the data-generating behavior policy, injecting a look-ahead bias that breaks causal consistency. Furthermore, we prove that under policy shift this failure causes the variance of evaluation metrics to explode geometrically, a phenomenon we term controllability collapse. To restore causal consistency, we establish theoretical conditions for accurate simulation and propose practical training mitigations: a priori controls, step-wise dynamic controls, and direct policy-conditioned learning. Empirical evaluation confirms that while standard global controls distort conversational distributions and collapse behavioral diversity, our causally grounded simulators eliminate look-ahead bias, preserve natural variance, and exhibit robust zero-shot generalization to unseen agent behaviors.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
