arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.07405cs.CL
收藏

Dólares还是Dollars?揭示金融LLM在西班牙语与英语间的双语能力

Dólares or Dollars? Unraveling the Bilingual Prowess of Financial LLMs Between Spanish and English

The Fin AI;University of Manchester(曼彻斯特大学);School of Computer Science Sichuan University(四川大学计算机学院);School of Computer Science Wuhan University(武汉大学计算机学院);University of Florida(佛罗里达大学);Columbia University(哥伦比亚大学)

Xiao Zhang, Ruoyu Xiang, Chenhan Yuan, Duanyu Feng, Weiguang Han, Alejandro Lopez-Lira, Xiao-Yang Liu, Sophia Ananiadou, Min Peng, Jimin Huang, Qianqian Xie

AI总结:

针对西班牙语金融NLP研究不足的问题,提出首个双语框架Toisón de Oro,构建指令数据集与评估基准,并推出FinMA-ES模型,在西班牙语金融任务中超越GPT-4等SOTA模型。

摘要:

尽管西班牙语在全球金融业中扮演着关键角色,但与英语相比,西班牙语在金融自然语言处理(NLP)和应用研究方面存在显著差距,尤其是在大语言模型(LLM)时代。为了弥合这一差距,我们推出了Toisón de Oro,这是首个双语框架,为西班牙语与英语结合的金融LLM建立了指令数据集、微调LLM和评估基准。我们构建了一个经过严格筛选的双语指令数据集,包含来自15个数据集、覆盖7项任务的超过144K个西班牙语和英语样本。利用该数据集,我们引入了FinMA-ES,一种专为双语金融应用设计的LLM。我们使用FLARE-ES评估我们的模型和现有LLM,这是首个包含21个数据集、覆盖9项任务的综合双语评估基准。FLARE-ES基准结果显示,现有LLM存在显著的多语言性能差距和偏差。得益于策略性指令调优和利用来自多样语言资源的数据,FinMA-ES模型在西班牙语金融任务中超越了GPT-4等SOTA LLM,突显了跨语言迁移的积极影响。我们所有的数据集、模型和基准均已发布。

英文摘要:

Despite Spanish's pivotal role in the global finance industry, a pronounced gap exists in Spanish financial natural language processing (NLP) and application studies compared to English, especially in the era of large language models (LLMs). To bridge this gap, we unveil Toisón de Oro, the first bilingual framework that establishes instruction datasets, finetuned LLMs, and evaluation benchmark for financial LLMs in Spanish joint with English. We construct a rigorously curated bilingual instruction dataset including over 144K Spanish and English samples from 15 datasets covering 7 tasks. Harnessing this, we introduce FinMA-ES, an LLM designed for bilingual financial applications. We evaluate our model and existing LLMs using FLARE-ES, the first comprehensive bilingual evaluation benchmark with 21 datasets covering 9 tasks. The FLARE-ES benchmark results reveal a significant multilingual performance gap and bias in existing LLMs. FinMA-ES models surpass SOTA LLMs such as GPT-4 in Spanish financial tasks, due to strategic instruction tuning and leveraging data from diverse linguistic resources, highlighting the positive impact of cross-linguistic transfer. All our datasets, models, and benchmarks have been released.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑