PyFi: 通过对抗代理实现金字塔式金融图像理解的愿景语言模型
PyFi: Toward Pyramid-like Financial Image Understanding for VLMs via Adversarial Agents
AI总结：
PyFi通过对抗代理生成的金字塔结构数据集，提升VLM在金融图像理解中的推理能力，实验显示模型在复杂金融问题上的准确率提升19.52%。
摘要：
本文提出PyFi，一种用于金字塔式金融图像理解的新框架，使视觉语言模型（VLMs）能够通过问题链逐步进行推理。PyFi的核心是PyFi-600K数据集，包含600,000个金融问答对，按推理金字塔组织：基础问题只需基本感知，而顶端问题需要逐步增加的金融视觉理解和专业知识。该数据集可扩展，因为其通过PyFi-adv多代理对抗机制生成，该机制基于蒙特卡洛树搜索（MCTS）范式，在每个图像中，挑战者代理与求解者代理竞争，生成逐步深入的金融视觉推理问题链。利用该数据集，我们对先进VLM在金融领域的细粒度、分层和全面进行了评估。此外，对Qwen2.5-VL-3B和Qwen2.5-VL-7B进行微调，使这些模型能够通过将复杂金融问题分解为逐步增加推理需求的子问题来回答复杂问题，在数据集上分别实现了19.52%和8.06%的平均准确率提升。所有资源包括代码、数据集和模型均可在https://github.com/AgenticFinLab/PyFi获取。
英文摘要：
This paper proposes PyFi, a novel framework for pyramid-like financial image understanding that enables vision language models (VLMs) to reason through question chains in a progressive, simple-to-complex manner. At the core of PyFi is PyFi-600K, a dataset comprising 600K financial question-answer pairs organized into a reasoning pyramid: questions at the base require only basic perception, while those toward the apex demand increasing levels of capability in financial visual understanding and expertise. This data is scalable because it is synthesized without human annotations, using PyFi-adv, a multi-agent adversarial mechanism under the Monte Carlo Tree Search (MCTS) paradigm, in which, for each image, a challenger agent competes with a solver agent by generating question chains that progressively probe deeper capability levels in financial visual reasoning. Leveraging this dataset, we present fine-grained, hierarchical, and comprehensive evaluations of advanced VLMs in the financial domain. Moreover, fine-tuning Qwen2.5-VL-3B and Qwen2.5-VL-7B on the pyramid-structured question chains enables these models to answer complex financial questions by decomposing them into sub-questions with gradually increasing reasoning demands, yielding average accuracy improvements of 19.52% and 8.06%, respectively, on the dataset. All resources of code, dataset and models are available at: https://github.com/AgenticFinLab/PyFi .
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
