FinCap：面向金融类 YouTube 短视频的主题对齐字幕
FinCap: Topic-Aligned Captions for Short-Form Financial YouTube Videos
AI总结：
本文构建 FinCap 数据集，系统评估 MLLM 在金融短视频主题对齐字幕生成中的七种模态组合，发现单视频及选择性双模态常优于全模态。
摘要：
我们评估多模态大语言模型（MLLM）在金融短视频（SV）主题对齐字幕生成中的表现，测试其对字幕文本（T）、音频（A）和视频（V）的联合推理能力。利用624个带标注的 YouTube 短视频，我们在主要推荐、情绪分析、视频目的、视觉分析和金融实体识别五个主题上，评估全部七种模态组合（T、A、V、TA、TV、AV、TAV）。仅使用视频在五个主题中的四个上表现强劲，凸显了其在捕捉视觉情境以及情绪、手势和肢体语言等有效线索方面的价值。TV或AV等选择性模态组合通常优于TAV，表明过多模态可能引入噪声。这些结果建立了金融短视频字幕生成的首批基线，并阐明了在该领域中锚定复杂视觉线索的潜力与挑战。所有代码和数据均可在我们的 Github 上获取，并采用 CC-BY-NC-SA 4.0 许可证。
英文摘要：
We evaluate multimodal large language models (MLLMs) for topic-aligned captioning in financial short-form videos (SVs) by testing joint reasoning over transcripts (T), audio (A), and video (V). Using 624 annotated YouTube SVs, we assess all seven modality combinations (T, A, V, TA, TV, AV, TAV) across five topics: main recommendation, sentiment analysis, video purpose, visual analysis, and financial entity recognition. Video alone performs strongly on four of five topics, underscoring its value for capturing visual context and effective cues such as emotions, gestures, and body language. Selective pairs such as TV or AV often surpass TAV, implying that too many modalities may introduce noise. These results establish the first baselines for financial short-form video captioning and illustrate the potential and challenges of grounding complex visual cues in this domain. All code and data can be found on our Github under the CC-BY-NC-SA 4.0 license.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
