基于大型视觉语言模型的风格化体育图像描述生成
Large VLM-based Stylized Sports Captioning
AI总结：
本文针对现有大型视觉语言模型在体育图像描述中缺乏领域术语和风格化能力的问题，提出一种两级微调LVLM流水线，在F1和BERT分数上显著提升，并在超级碗直播中验证了其高效实用性。
摘要：
大型（视觉）语言模型（LLM / LVLM）的出现，已在社交媒体内容生成、搜索与推荐、医疗预后、面向认知任务的AI助手等多个领域催生了大量自动化类人系统。尽管这些系统已成功集成到生产环境中，但针对体育领域，尤其是对比赛过程的准确识别与自然语言描述，关注仍然非常有限。现有的大多数LLM/LVLM能够解释一般性体育活动，但缺乏足够的领域中心化体育术语，难以生成自然（类人）的描述。本研究突出了现有最先进（SoTA）LLM/LVLM在从图像生成符合期望风格化格式的生产级体育描述方面的局限性，并提出了一种两级微调LVLM流水线来解决该问题。与替代方法相比，所提出的流水线在F1分数上实现了超过8-10%的提升，在BERT分数上实现了超过2-10%的提升。此外，该方法具有较小的运行时内存占用和快速的执行时间。在第五十九届超级碗（Super Bowl LIX）期间，该流水线证明了其在专业体育直播新闻中的实际应用价值；在比赛过程中为超过1000张图像以每3-5秒6张图像的速率生成了高度准确且风格化的描述。
英文摘要：
The advent of large (visual) language models (LLM / LVLM) have led to a deluge of automated human-like systems in several domains including social media content generation, search and recommendation, healthcare prognosis, AI assistants for cognitive tasks etc. Although these systems have been successfully integrated in production; very little focus has been placed on sports, particularly accurate identification and natural language description of the game play. Most existing LLM/LVLMs can explain generic sports activities, but lack sufficient domain-centric sports' jargon to create natural (human-like) descriptions. This work highlights the limitations of existing SoTA LLM/LVLMs for generating production-grade sports captions from images in a desired stylized format, and proposes a two-level fine-tuned LVLM pipeline to address that. The proposed pipeline yields an improvement > 8-10% in the F1, and > 2-10% in BERT score compared to alternative approaches. In addition, it has a small runtime memory footprint and fast execution time. During Super Bowl LIX the pipeline proved its practical application for live professional sports journalism; generating highly accurate and stylized captions at the rate of 6 images per 3-5 seconds for over 1000 images during the game play.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
