While Large Language Models (LLMs) have shown impressive capabilities in numerous Natural Language Processing (NLP) tasks, they still struggle with financial question answering (QA), particularly when numerical reasoning is required. Recently, LLM-based multi-agent frameworks have demonstrated remarkable effectiveness in multi-step reasoning, which is crucial for financial QA tasks as it involves extracting relevant information from tables and text and then performing numerical reasoning on the extracted data to infer answers. In this study, we propose a multi-agent framework incorporating a critic agent that reflects on the reasoning steps and final answers for each question. Additionally, we enhance our system by adding multiple critic agents, each focusing on a specific aspect of the answer. Our results indicate that this framework significantly improves performance compared to single-agent reasoning, with an average performance increase of 15% for the LLaMA3-8B model and 5% for the LLaMA3-70B model. Furthermore, our framework performs on par with, and in some cases surpasses, larger single-agent LLMs such as LLaMA3.1-405B and GPT-4o-mini, though it falls slightly short compared to Claude-3.5 Sonnet. Overall, our framework presents an effective solution to enhance open-source LLMs for financial QA tasks, offering a cost-effective alternative to larger models like Claude-3.5 Sonnet.
Automated Regulatory Compliance Question Answering in Financial Services with Domain-Adapted Retrieval-Augmented Generation
University of Bonn(波恩大学);Lamarr-Institute for Machine Learning and Artificial Intelligence(拉马尔机器学习和人工智能研究所);Universitat Rovira i Virgili(罗维拉-维尔吉利大学);Fraunhofer IAIS(弗劳恩霍夫智能分析和信息系统研究所)
Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学);The Fin AI(金融人工智能公司);FMI, Sofia University “St. Kliment Ohridski”(索非亚大学“圣克莱门特·奥赫里德斯基”金融数学与信息学系);INSAIT, Sofia University “St. Kliment Ohridski”(索非亚大学“圣克莱门特·奥赫里德斯基”信息技术与自动化研究所);University of Arizona(亚利桑那大学);The University of Tokyo(东京大学);Linköping University(林雪平大学);McGill University(麦吉尔大学);Mila, Quebec AI Institute(魁北克人工智能研究所);Meiji Gakuin University(明治学院大学)
Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学);INSAIT, Sofia University “St. Kliment Ohridski”(索非亚大学“圣克莱门特·奥赫里德斯基”INSAIT学院);University of Arizona(亚利桑那大学);FMI, Sofia University “St. Kliment Ohridski”(索非亚大学“圣克莱门特·奥赫里德斯基”FMI学院);The Fin AI(The Fin AI公司);The University of Tokyo(东京大学);Linköping University(林雪平大学);McGill University(麦吉尔大学);Mila, Quebec AI Institute(魁北克Mila人工智能研究所);Meiji Gakuin University(明治学院大学)
Adaptive Memory and Reflection Multi-Agent System for Medical Question Answering
School of Computing and Mathematics, Manchester Metropolitan University(曼彻斯特城市大学计算与数学学院);Electrical and Computer Engineering, Technion – Israel Institute of Technology(以色列理工学院电气与计算机工程系);School of Science and Technology, Hong Kong Metropolitan University(香港都会大学科学与科技学院)
Enhancing Financial Report Question-Answering: A Retrieval-Augmented Generation System with Reranking Analysis
School of Engineering Stanford University Stanford, CA, USA;Simon Business School University of Rochester Rochester, NY, USA;Accounting \& Information Systems Rutgers University Newark, NJ, USA;Institute for Social;Economic Research;Policy Columbia University New York, NY, USA;Department of Economics Northeastern University Boston, MA, USA
MTBench: A Multimodal Time Series Benchmark for Temporal Reasoning and Question Answering
Yale University, New Haven, CT, USA(耶鲁大学);McGill University, Montreal, QC, Canada(麦吉尔大学);University of Texas Rio Grande Valley, TX, USA(德克萨斯大学里奥格兰德谷分校)
EvidenceLens: A Claim-Evidence Matrix for Auditing Financial Question Answering
1 School of AI;Advanced Computing, XJTLU Entrepreneur College (Taicang), Xi’an Jiaotong-Liverpool University, Suzhou, Jiangsu, China 2 ETSI de Telecomunicaci\' o n, Universidad Polit\' e cnica de Madrid, Madrid, Spain IEEE VIS 2026 conditionally accepted version
DSC-IITISM at FinCausal 2021: Combining POS tagging with Attention-based Contextual Representations for Identifying Causal Relationships in Financial Documents
Indian Institute of Technology (ISM), Dhanbad(印度理工学院(ISM)达巴德分校)
FinRank: An Evidence-Grounded Benchmark for Financial Question Answering and Retrieval over SEC Filings
University of Groningen(格罗宁根大学);Goethe University Frankfurt(法兰克福大学);DataNXT GmbH(DataNXT有限公司);Halle Institute for Economic Research (IWH)(哈雷经济研究所);Martin Luther University Halle-Wittenberg(马丁路德大学哈雷-维滕贝格分校)
FinSage: A Multi-aspect RAG System for Financial Filings Question Answering
1SimpleWay.AI 2McGill University 3University of Toronto 4University of California, Los Angeles 5The Chinese University of Hong Kong 6Duke University 7Universit\'e de Montr\'eal 8Mila - Quebec AI Institute 9Noah's Ark Lab 10CG Matrix Technology Limited;1SimpleWay.AI 2McGill University 3University of Toronto 4University of California, Los Angeles;5The Chinese University of Hong Kong 6Duke University 7Universit\'e de Montr\'eal 8Mila - Quebec AI Institute;9Noah's Ark Lab 10CG Matrix Technology Limited
VeritasFi: An Adaptable, Multi-tiered RAG Framework for Multi-modal Financial Question Answering
University of Toronto(多伦多大学);McMaster University(麦马斯特大学);McGill University(麦吉尔大学);University of Manitoba(曼尼托巴大学);University of California, Los Angeles(加州大学洛杉矶分校);University of Montreal(蒙特利尔大学);Mila;CUHK(香港中文大学);HKUST(GZ)(香港理工大学(广州));Nanyang Technological University(南洋理工大学);Stanford University(斯坦福大学);CG Matrix Technology Limited(CG矩阵科技有限公司)
Bypassing Document Ingestion: An MCP Approach to Financial Q&A
University of Groningen(Groningen大学);Goethe University Frankfurt(法兰克福歌德大学);Martin Luther University Halle-Wittenberg(哈雷-维滕贝格马丁路德大学);Halle Institute for Economic Research (IWH)(哈雷经济研究所)
Can Language Models Analyze Data? Evaluating Large Language Models for Question Answering over Datasets
School of Production Engineering and Management, Technical University of Crete(生产工程与管理学院,希腊克里特技术大学);Institute of Computer Science, Foundation for Research and Technology - Hellas(计算机科学研究所,希腊基础研究与技术研究院)
Fine-tuning of lightweight large language models for sentiment classification on heterogeneous financial textual data
International Business School, Zhejiang University(浙江大学国际商务学院);Center for Data Science, Zhejiang University(浙江大学数据科学中心);Centre for Statistics, Georg-August-Universität Göttingen(哥廷根大学统计中心)