Plutus：低资源希腊金融领域大语言模型的基准测试
Plutus: Benchmarking Large Language Models in Low-Resource Greek Finance
AI总结：
针对希腊金融领域缺乏基准和模型的问题，提出首个希腊金融评估基准Plutus-ben和金融大语言模型Plutus-8B，涵盖五项核心NLP任务，评估22个LLM，揭示跨语言迁移局限，推动希腊金融NLP发展。
摘要：
尽管希腊在全球经济中扮演着关键角色，但由于希腊语的复杂性和领域特定数据集的稀缺，大语言模型（LLM）在希腊金融语境中的应用仍未得到充分探索。以往在多语言金融自然语言处理（NLP）方面的努力暴露了显著的性能差异，但直到现在，还没有专门的希腊金融基准或希腊特定的金融大语言模型被开发出来。为了弥合这一差距，我们推出了Plutus-ben，这是首个希腊金融评估基准，以及Plutus-8B，这是开创性的希腊金融大语言模型，使用希腊领域特定数据进行了微调。Plutus-ben涵盖了希腊语中的五项核心金融NLP任务：数字和文本命名实体识别、问答、抽象式摘要生成以及主题分类，从而促进了系统化和可复现的LLM评估。为了支撑这些任务，我们提出了三个新颖的高质量希腊金融数据集，由希腊母语专家彻底标注，并辅以两个现有资源。我们在Plutus-ben上对22个LLM进行的全面评估表明，由于语言复杂性、领域特定术语和金融推理差距，希腊金融NLP仍然具有挑战性。这些发现强调了跨语言迁移的局限性、在希腊语训练模型中金融专业知识的必要性，以及将金融LLM适应希腊文本的挑战。我们公开发布Plutus-ben、Plutus-8B以及所有相关数据集，以促进可复现的研究并推进希腊金融NLP，从而在金融领域促进更广泛的多语言包容性。
英文摘要：
Despite Greece's pivotal role in the global economy, large language models (LLMs) remain underexplored for Greek financial context due to the linguistic complexity of Greek and the scarcity of domain-specific datasets. Previous efforts in multilingual financial natural language processing (NLP) have exposed considerable performance disparities, yet no dedicated Greek financial benchmarks or Greek-specific financial LLMs have been developed until now. To bridge this gap, we introduce Plutus-ben, the first Greek Financial Evaluation Benchmark, and Plutus-8B, the pioneering Greek Financial LLM, fine-tuned with Greek domain-specific data. Plutus-ben addresses five core financial NLP tasks in Greek: numeric and textual named entity recognition, question answering, abstractive summarization, and topic classification, thereby facilitating systematic and reproducible LLM assessments. To underpin these tasks, we present three novel, high-quality Greek financial datasets, thoroughly annotated by expert native Greek speakers, augmented by two existing resources. Our comprehensive evaluation of 22 LLMs on Plutus-ben reveals that Greek financial NLP remains challenging due to linguistic complexity, domain-specific terminology, and financial reasoning gaps. These findings underscore the limitations of cross-lingual transfer, the necessity for financial expertise in Greek-trained models, and the challenges of adapting financial LLMs to Greek text. We release Plutus-ben, Plutus-8B, and all associated datasets publicly to promote reproducible research and advance Greek financial NLP, fostering broader multilingual inclusivity in finance.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
