MultiFinBen：面向多语言和多模态金融应用的大型语言模型基准测试
MultiFinBen: Benchmarking Large Language Models for Multilingual and Multimodal Financial Application
AI总结：
提出首个专家标注的多语言多模态金融基准MultiFinBen，包含多语言金融推理和金融OCR任务，评估21个领先LLM发现前沿模型在多语言与多模态金融推理上仍存在显著局限。
摘要：
现实世界的金融分析涉及跨多种语言和模态的信息，从报告、新闻到扫描的文件和会议录音。然而，现有对金融领域LLMs的评估大多仍局限于纯文本、单语言，且基本已被当前模型所饱和。为填补这些空白，我们提出了MultiFinBen，这是首个专家标注的多语言（五种语言）和多模态（文本、视觉、音频）基准，用于在真实金融语境中评估LLMs。MultiFinBen引入了两个新任务族：多语言金融推理，测试从文件和新闻中整合跨语言证据的能力；以及金融OCR，从包含表格和图表的扫描文档中提取结构化文本。我们没有聚合所有可用数据集，而是基于先进模型的表现应用了结构化、难度感知的筛选，确保挑战均衡并移除冗余任务。对21个领先LLMs的评估表明，即使是GPT-4o等前沿多模态模型也仅达到46.01%的总体得分，它们在视觉和音频上表现较强，但在多语言环境下急剧下降。这些发现揭示了多语言、多模态及专家级金融推理方面持续存在的局限性。所有数据集、评估脚本和排行榜均已公开发布。
英文摘要：
Real-world financial analysis involves information across multiple languages and modalities, from reports and news to scanned filings and meeting recordings. Yet most existing evaluations of LLMs in finance remain text-only, monolingual, and largely saturated by current models. To bridge these gaps, we present MultiFinBen, the first expert-annotated multilingual (five languages) and multimodal (text, vision, audio) benchmark for evaluating LLMs in realistic financial contexts. MultiFinBen introduces two new task families: multilingual financial reasoning, which tests cross-lingual evidence integration from filings and news, and financial OCR, which extracts structured text from scanned documents containing tables and charts. Rather than aggregating all available datasets, we apply a structured, difficulty-aware selection based on advanced model performance, ensuring balanced challenge and removing redundant tasks. Evaluating 21 leading LLMs shows that even frontier multimodal models like GPT-4o achieve only 46.01% overall, stronger on vision and audio but dropping sharply in multilingual settings. These findings expose persistent limitations in multilingual, multimodal, and expert-level financial reasoning. All datasets, evaluation scripts, and leaderboards are publicly released.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
