arXiv 2603.019732026-03-02cs.CLcs.AIcs.SI

CharacterFlywheel: 在生产环境中迭代改进社交聊天应用中吸引人且可操控的大型语言模型

CharacterFlywheel: Scaling Iterative Improvement of Engaging and Steerable LLMs in Production

Meta Superintelligence Labs（Meta超智能实验室）；FAIR at Meta（Meta的FAIR部门）；OpenAI（开放人工智能研究所）

Yixin Nie, Lin Guan, Zhongyao Ma, Anchit Gupta, Yipin Zhou, Xiao Li, Zhengping Zhou, Raymond Zeng, Gelin Zhou, Shigan Chu, Ajay Thampi, Wancen Mu, Nathan Shuste… 展开作者

Yixin Nie, Lin Guan, Zhongyao Ma, Anchit Gupta, Yipin Zhou, Xiao Li, Zhengping Zhou, Raymond Zeng, Gelin Zhou, Shigan Chu, Ajay Thampi, Wancen Mu, Nathan Shuster, Ketong Wang, Lin Chen, Jason Brewer, Derek Hao Hu, Alexander McCauley, Jason Weston, Sem Park, Na Zhang, Kevin Tang

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

CharacterFlywheel通过迭代优化提升社交聊天应用中LLMs的参与度和可操控性，实现显著的性能提升。

摘要：

本报告介绍了CharacterFlywheel，一种用于在Instagram、WhatsApp和Messenger等社交聊天应用中改进大型语言模型（LLMs）的迭代飞轮过程。从LLaMA 3.1开始，我们利用内部和外部真实用户流量的数据，通过15代模型的优化，持续部署从2024年7月到2025年4月。我们进行了受控的7天A/B测试，显示持续的参与度提升：8个新部署的模型中有7个在基准线上表现出积极的提升，最强的模型在参与度广度上提升了高达8.8%，在参与度深度上提升了19.4%。我们还观察到在可操控性方面有显著的提升，指令遵循从59.2%提升到84.8%，而指令违规率从26.6%降至5.8%。我们详细介绍了CharacterFlywheel过程，该过程整合了数据整理、奖励建模以估计和插值参与度指标的景观、监督微调（SFT）、强化学习（RL）以及离线和在线评估，以确保在每一步优化中都能取得可靠的进展。我们还讨论了我们的方法，以防止过拟合并导航大规模生产动态。这些贡献推动了社交应用中服务于数百万用户的LLMs的科学严谨性和理解。

英文摘要：

This report presents CharacterFlywheel, an iterative flywheel process for improving large language models (LLMs) in production social chat applications across Instagram, WhatsApp, and Messenger. Starting from LLaMA 3.1, we refined models across 15 generations using data from both internal and external real-user traffic. Through continuous deployments from July 2024 to April 2025, we conducted controlled 7-day A/B tests showing consistent engagement improvements: 7 of 8 newly deployed models demonstrated positive lift over the baseline, with the strongest performers achieving up to 8.8% improvement in engagement breadth and 19.4% in engagement depth. We also observed substantial gains in steerability, with instruction following increasing from 59.2% to 84.8% and instruction violations decreasing from 26.6% to 5.8%. We detail the CharacterFlywheel process which integrates data curation, reward modeling to estimate and interpolate the landscape of engagement metrics, supervised fine-tuning (SFT), reinforcement learning (RL), and both offline and online evaluation to ensure reliable progress at each optimization step. We also discuss our methods for overfitting prevention and navigating production dynamics at scale. These contributions advance the scientific rigor and understanding of LLMs in social applications serving millions of users.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。