SportsMetrics：融合文本与数值数据以理解 LLM 中的信息融合
SportsMetrics: Blending Text and Numerical Data to Understand Information Fusion in LLMs
AI总结：
本文提出 SportsMetrics 基准，通过四项基于逐回合 NBA、NFL 比赛描述及对抗性场景的任务，评估 LLM 融合文本与数值数据并进行复杂推理的能力。
摘要：
大型语言模型在整合文本文档和数据库记录等多种数据类型以进行高级分析方面具有巨大潜力。然而，融合文本与数值数据带来了重大挑战。LLM 需要处理并交叉引用实体和数字，应对数据不一致和冗余，并发展规划能力，例如构建用于管理复杂数据查询的工作记忆。本文提出四项以体育数据分析为中心的新任务，以评估 LLM 的数值推理和信息融合能力。这些任务向 LLM 提供详细的、逐回合的体育比赛描述，随后用对抗性场景对其提出挑战，例如新比赛规则、更长持续时间、打乱的叙述，以及分析比赛摘要中的关键统计数据。我们在 NBA 和 NFL 比赛上开展大量实验，以评估 LLM 在这些任务上的表现。我们的基准 SportsMetrics 引入了一种评估 LLM 数值推理与融合技能的新机制。
英文摘要：
Large language models hold significant potential for integrating various data types, such as text documents and database records, for advanced analytics. However, blending text and numerical data presents substantial challenges. LLMs need to process and cross-reference entities and numbers, handle data inconsistencies and redundancies, and develop planning capabilities such as building a working memory for managing complex data queries. In this paper, we introduce four novel tasks centered around sports data analytics to evaluate the numerical reasoning and information fusion capabilities of LLMs. These tasks involve providing LLMs with detailed, play-by-play sports game descriptions, then challenging them with adversarial scenarios such as new game rules, longer durations, scrambled narratives, and analyzing key statistics in game summaries. We conduct extensive experiments on NBA and NFL games to assess the performance of LLMs on these tasks. Our benchmark, SportsMetrics, introduces a new mechanism for assessing LLMs' numerical reasoning and fusion skills.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
