Feedback Forensics：一种衡量AI人格的工具包
Feedback Forensics: A Toolkit to Measure AI Personality
AI总结：
针对传统基准难以衡量AI人格、现有反馈式评估方法不透明且存在过拟合等问题，研究人员推出开源工具包Feedback Forensics，支持通过Python API和浏览器应用分析反馈数据集引导的人格特质及主流模型的人格表现，并同步发布网页应用与标注数据。
摘要：
构成“优秀”AI模型的某些特质很难预先描述。例如，回复应该更礼貌还是更随意？这类特质有时被概括为模型性格或人格。由于缺乏明确的目标，基于自动验证的传统基准难以衡量这类特质。使用人类反馈的评估方法（如Chatbot Arena）已成为一种流行的替代方案。这些方法通过对多个模型的回复进行相对排名，隐式推断出“更好的”人格及其他理想特质。近期模型发布中出现的问题凸显了这些现有不透明评估方法的局限性：一款主流模型因谄媚型人格问题被回滚，还有模型被发现对这类基于反馈的排行榜存在过拟合。尽管存在这些已知问题，但用于显式评估模型人格的公开工具仍然有限。我们推出了Feedback Forensics：一款开源工具包，用于追踪AI人格的变化，既包括由人类（或AI）反馈引导产生的人格变化，也包括在这类反馈上训练和评估的AI模型所展现出的人格特征。借助AI标注者，我们的工具包支持通过Python API和浏览器应用开展人格研究。我们分两步证明了该工具包的实用性：（A）首先分析了Chatbot Arena、MultiPref和PRISM等主流人类反馈数据集中所引导的人格特质；（B）随后使用我们的工具包分析了主流模型展现出这类特质的程度。我们在https://github.com/rdnfn/feedback-forensics发布了三项内容：（1）Feedback Forensics工具包；（2）一个追踪主流模型和反馈数据集中AI人格的网页应用；（3）底层标注数据。
英文摘要：
Some traits making a "good" AI model are hard to describe upfront. For example, should responses be more polite or more casual? Such traits are sometimes summarized as model character or personality. Without a clear objective, conventional benchmarks based on automatic validation struggle to measure such traits. Evaluation methods using human feedback such as Chatbot Arena have emerged as a popular alternative. These methods infer "better" personality and other desirable traits implicitly by ranking multiple model responses relative to each other. Recent issues with model releases highlight limitations of these existing opaque evaluation approaches: a major model was rolled back over sycophantic personality issues, models were observed overfitting to such feedback-based leaderboards. Despite these known issues, limited public tooling exists to explicitly evaluate model personality. We introduce Feedback Forensics: an open-source toolkit to track AI personality changes, both those encouraged by human (or AI) feedback, and those exhibited across AI models trained and evaluated on such feedback. Leveraging AI annotators, our toolkit enables investigating personality via Python API and browser app. We demonstrate the toolkit's usefulness in two steps: (A) first we analyse the personality traits encouraged in popular human feedback datasets including Chatbot Arena, MultiPref and PRISM; and (B) then use our toolkit to analyse how much popular models exhibit such traits. We release (1) our Feedback Forensics toolkit alongside (2) a web app tracking AI personality in popular models and feedback datasets as well as (3) the underlying annotation data at https://github.com/rdnfn/feedback-forensics.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
