微调视觉语言模型以实现马来西亚审计财务报告中财务表格的Markdown转换
Fine-Tuning Vision-Language Models for Markdown Conversion of Financial Tables in Malaysian Audited Financial Reports
AI总结：
针对马来西亚审计财务报告中复杂表格的Markdown转换问题，提出基于Qwen2.5-VL-7B和LoRA微调的视觉语言模型，在准确率与推理速度上超越GPT-4o等大模型，实现高效文档自动化。
摘要：
从财务文档中准确提取并表示表格数据的结构仍然是文档理解中的一项关键挑战，特别是对于监管和分析用例。本研究解决了将马来西亚审计财务报告中的财务表格转换为Markdown格式的复杂性，该任务因旋转布局、多级表头和隐式结构线索而变得复杂。我们提出了一种基于Qwen2.5-VL-7B微调的视觉语言模型（VLM），针对从文档图像生成高保真Markdown进行了优化。我们的方法包括一个包含2,152个图文对的精选数据集（带有数据增强）以及一种使用LoRA的监督微调策略。为了评估性能，我们在100个样本外表格上使用双重框架评估了模型：基于标准的LLM-as-a-judge用于细粒度准确率，以及我们新颖的基于Markdown树编辑距离的相似度（TEDS）指标用于整体结构保真度。我们的模型在基于标准的评估上达到了92.20%的总体准确率，Markdown TEDS得分为96.53%。这一性能显著超越了其Qwen2.5-VL-7B基础模型、更大规模的VLM以及支持推理的专用模型。与这些自托管替代方案相比，它还显著减少了推理时间。此外，其准确率超过了广泛使用的专有模型，如OpenAI的GPT-4o和Gemini 2.5 Flash。这些结果表明，特定领域的微调提供了一种有效且高效的方法来弥合非结构化财务文档与下游自动化之间的差距，在无需计算开销的情况下媲美更大且更通用的模型。
英文摘要：
Accurately extracting and representing the structure of tabular data from financial documents remains a critical challenge in document understanding, particularly for regulatory and analytical use cases. This study addresses the complexity of converting financial tables from Malaysian audited financial reports into Markdown format, a task complicated by rotated layouts, multi-level headers, and implicit structural cues. We propose a fine-tuned vision-language model (VLM), based on Qwen2.5-VL-7B, optimized for high-fidelity Markdown generation from document images. Our approach includes a curated dataset of 2,152 image-text pairs with augmentations and a supervised fine-tuning strategy using LoRA. To assess performance, we evaluated our model on 100 out-of-sample tables using a dual framework: a criteria-based LLM-as-a-judge for fine-grained accuracy and our novel Markdown Tree-Edit-Distance-based Similarity (TEDS) metric for holistic structural fidelity. Our model achieves a 92.20% overall accuracy on the criteria-based assessment and a 96.53% Markdown TEDS score. This performance significantly surpasses its Qwen2.5-VL-7B base model, larger-scale VLMs, and specialized reasoning-enabled models. Compared to these self-hosted alternatives, it also significantly reduces inference time. Furthermore, its accuracy exceeds that of widely used proprietary models such as OpenAI's GPT-4o and Gemini 2.5 Flash. These results demonstrate that domain-specific fine-tuning provides an effective and efficient method to bridge the gap between unstructured financial documents and downstream automation, rivalling much larger and more general models without their computational overhead.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
