FAIL: 流匹配对抗模仿学习用于图像生成
FAIL: Flow Matching Adversarial Imitation Learning for Image Generation
AI总结：
本文提出FAIL方法，通过对抗训练最小化策略-专家差异，无需显式奖励或配对比较，在图像生成中实现高效训练与泛化。
摘要：
流匹配模型的后训练过程——将输出分布对齐高质量目标——在数学上等价于模仿学习。尽管监督微调能有效模仿专家演示，但无法纠正未见状态中的策略漂移。偏好优化方法对此问题有所解决，但需要昂贵的偏好配对或奖励建模。我们提出流匹配对抗模仿学习（FAIL），通过对抗训练最小化策略-专家差异，无需显式奖励或配对比较。我们推导出两种算法：FAIL-PD利用可微ODE求解器实现低方差路径梯度，而FAIL-PG为离散或计算受限场景提供黑盒替代方案。使用仅13,000个来自Nano Banana Pro的演示对FLUX进行微调，FAIL在提示跟随和美学基准上实现了竞争力表现。此外，该框架有效泛化到离散图像和视频生成，并作为稳健正则化器缓解基于奖励的优化中的奖励黑客问题。代码和数据可在https://github.com/HansPolo113/FAIL获取。
英文摘要：
Post-training of flow matching models-aligning the output distribution with a high-quality target-is mathematically equivalent to imitation learning. While Supervised Fine-Tuning mimics expert demonstrations effectively, it cannot correct policy drift in unseen states. Preference optimization methods address this but require costly preference pairs or reward modeling. We propose Flow Matching Adversarial Imitation Learning (FAIL), which minimizes policy-expert divergence through adversarial training without explicit rewards or pairwise comparisons. We derive two algorithms: FAIL-PD exploits differentiable ODE solvers for low-variance pathwise gradients, while FAIL-PG provides a black-box alternative for discrete or computationally constrained settings. Fine-tuning FLUX with only 13,000 demonstrations from Nano Banana pro, FAIL achieves competitive performance on prompt following and aesthetic benchmarks. Furthermore, the framework generalizes effectively to discrete image and video generation, and functions as a robust regularizer to mitigate reward hacking in reward-based optimization. Code and data are available at https://github.com/HansPolo113/FAIL.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
