体育智能：通过从文本到视频的问答评估语言模型的体育理解能力
Sports Intelligence: Assessing the Sports Understanding Capabilities of Language Models through Question Answering from Text to Video
AI总结：
针对现有NLP体育理解基准的不足，研究全面评估了主流大语言模型与视频语言模型在多类体育任务上的表现，提出新基准并开展错误分析，为该领域后续研究指明方向。
摘要：
由于体育领域的复杂性与动态性，理解体育对自然语言处理（NLP）的发展至关重要。对复杂体育场景进行推理给当前需要高级认知能力的NLP技术带来了重大挑战。为解决NLP领域现有体育理解基准的局限性，我们针对多种体育任务对主流大语言模型开展了全面评估。我们的评估覆盖从基础规则、历史事实的简单查询到复杂的特定上下文推理，采用了从零样本到少样本学习以及思维链技术的策略。除单模态分析外，我们还进一步评估了主流视频语言模型的体育推理能力，以填补多模态体育理解基准的空白。我们的研究结果凸显了体育理解对NLP的关键挑战。我们在全面梳理现有体育数据集的基础上提出了一个新基准，并提供了详尽的错误分析，希望能助力明确该领域未来的研究重点。
英文摘要：
Understanding sports is crucial for the advancement of Natural Language Processing (NLP) due to its intricate and dynamic nature. Reasoning over complex sports scenarios has posed significant challenges to current NLP technologies which require advanced cognitive capabilities. Toward addressing the limitations of existing benchmarks on sports understanding in the NLP field, we extensively evaluated mainstream large language models for various sports tasks. Our evaluation spans from simple queries on basic rules and historical facts to complex, context-specific reasoning, leveraging strategies from zero-shot to few-shot learning, and chain-of-thought techniques. In addition to unimodal analysis, we further assessed the sports reasoning capabilities of mainstream video language models to bridge the gap in multimodal sports understanding benchmarking. Our findings highlighted the critical challenges of sports understanding for NLP. We proposed a new benchmark based on a comprehensive overview of existing sports datasets and provided extensive error analysis which we hope can help identify future research priorities in this field.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
