超新星事件数据集：通过关键事件分析解读大语言模型的个性
Supernova Event Dataset: Interpreting Large Language Models' Personality through Critical Event Analysis
AI总结：
本文提出超新星事件数据集，通过让LLM提取和排序关键事件，并由裁判模型推断其个性，揭示不同模型的决策风格，提升可解释性。
摘要：
大语言模型（LLMs）正日益融入日常应用。随着其影响力的增长，理解它们的决策过程和潜在个性变得至关重要。在本工作中，我们使用我们提出的超新星事件数据集（Supernova Event Dataset）来解读模型个性，这是一个新颖的数据集，包含涵盖传记、历史事件、新闻和科学发现的多篇多样化文章。我们使用该数据集对LLMs进行基准测试，任务是从文本中提取并排序关键事件，这是一个主观且复杂的挑战，需要对长程上下文进行推理并建模因果链。我们评估了Phi-4、Orca 2和Qwen 2.5等小模型，以及Claude 3.7、Gemini 2.5和OpenAI o3等大型更强模型，并提出一个框架，其中另一个LLM充当裁判，根据模型对事件的选择和分类来推断每个模型的个性。我们的分析显示出不同的个性特征：例如，Orca 2表现出情感推理，关注人际动态，而Qwen 2.5则展现出更具战略性、分析性的风格。在分析科学发现事件时，Claude Sonnet 3.7强调概念框架，Gemini 2.5 Pro优先考虑实证验证，而o3则偏好逐步因果推理。该分析提高了模型的可解释性，使其对广泛的多样化应用更加用户友好。项目页面 - https://www.supernova-event.ai/
英文摘要：
Large Language Models (LLMs) are increasingly integrated into everyday applications. As their influence grows, understanding their decision making and underlying personality becomes essential. In this work, we interpret model personality using our proposed Supernova Event Dataset, a novel dataset with diverse articles spanning biographies, historical events, news, and scientific discoveries. We use this dataset to benchmark LLMs on extracting and ranking key events from text, a subjective and complex challenge that requires reasoning over long-range context and modeling causal chains. We evaluate small models like Phi-4, Orca 2, and Qwen 2.5, and large, stronger models such as Claude 3.7, Gemini 2.5, and OpenAI o3, and propose a framework where another LLM acts as a judge to infer each model's personality based on its selection and classification of events. Our analysis shows distinct personality traits: for instance, Orca 2 demonstrates emotional reasoning focusing on interpersonal dynamics, while Qwen 2.5 displays a more strategic, analytical style. When analyzing scientific discovery events, Claude Sonnet 3.7 emphasizes conceptual framing, Gemini 2.5 Pro prioritizes empirical validation, and o3 favors step-by-step causal reasoning. This analysis improves model interpretability, making them user-friendly for a wide range of diverse applications. Project Page - https://www.supernova-event.ai/
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
