arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.06669cs.CLcs.AI
收藏

compar:IA:法国政府的LLM竞技场,用于收集法语人类提示和偏好数据

compar:IA: The French Government's LLM arena to collect French-language human prompts and preference data

The French Government(法国政府)

Lucie Termignon, Simonas Zilinskas, Hadrien Pélissier, Aurélien Barrot, Nicolas Chesnais, Elie Gavoty

AI总结:

compar:IA由法国政府开发,旨在通过收集法语人类提示和偏好数据,提升非英语语言中LLM的性能和文化契合度,同时推动多语言模型训练和评估的国际发展。

摘要:

大型语言模型(LLMs)在非英语语言中常常表现出性能下降、文化契合度降低和安全鲁棒性不足的问题,这在一定程度上是因为英语在预训练数据和人类偏好对齐数据集上占据主导地位。训练方法如基于人类反馈的强化学习(RLHF)和直接偏好优化(DPO)需要人类偏好数据,而这种数据在许多非英语语言中仍然稀缺且大多不公开。为了解决这一差距,我们介绍了compar:IA,这是一个由法国政府开发的开源数字公共服务平台,旨在从以法语为主的普通受众中收集大规模的人类偏好数据。该平台使用盲目的成对比较界面,以捕捉不受约束的真实世界提示和用户判断,涵盖多样化的语言模型。同时,该平台保持了低参与摩擦和隐私保护的自动过滤机制。截至2026年2月7日,compar:IA已收集了超过60万条自由形式提示和25万张偏好投票,其中大约89%的数据为法语。我们以开放许可证发布了三个互补的数据集——对话、投票和反应,并展示了初步分析,包括法语语言模型排行榜和用户交互模式。除了法国语境外,compar:IA正朝着国际数字公共产品方向发展,提供可重用的多语言模型训练、评估和人类与AI交互研究的基础设施。

英文摘要:

Large Language Models (LLMs) often show reduced performance, cultural alignment, and safety robustness in non-English languages, partly because English dominates both pre-training data and human preference alignment datasets. Training methods like Reinforcement Learning from Human Feedback (RLHF) and Direct Preference Optimization (DPO) require human preference data, which remains scarce and largely non-public for many languages beyond English. To address this gap, we introduce compar:IA, an open-source digital public service developed inside the French government and designed to collect large-scale human preference data from a predominantly French-speaking general audience. The platform uses a blind pairwise comparison interface to capture unconstrained, real-world prompts and user judgments across a diverse set of language models, while maintaining low participation friction and privacy-preserving automated filtering. As of 2026-02-07, compar:IA has collected over 600,000 free-form prompts and 250,000 preference votes, with approximately 89% of the data in French. We release three complementary datasets -- conversations, votes, and reactions -- under open licenses, and present initial analyses, including a French-language model leaderboard and user interaction patterns. Beyond the French context, compar:IA is evolving toward an international digital public good, offering reusable infrastructure for multilingual model training, evaluation, and the study of human-AI interaction.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑