FlipVQA: 通过教科书到知识合成实现多模态指令微调的扩展
FlipVQA: Scaling Multi-modal Instruction Tuning via Textbook-to-Knowledge Synthesis
AI总结：
本文提出FlipVQA-Miner自动化流程，解决OCR文档中的长距离逻辑依赖和跨页断续问题，构建了包含83K问答对的FlipVQA-83K数据集，在保持高结构保真度的同时节省50倍成本，提升了模型推理能力和跨领域泛化能力。
摘要：
教科书是人类验证的推理知识最丰富的存储库，但其复杂的布局包含多列排版、跨页问答分离和交织的图表，使自动化提取结构化的问答对和视觉问答对极为困难。现有方法要么从头合成数据，缺乏真实问题上下文，要么依赖昂贵的专家标注无法扩展。我们提出FlipVQA-Miner，一个自动化流程，解决OCR解析文档中的长距离逻辑依赖和跨页断续问题，恢复连贯的问题-答案-图表关联，即使答案位于不同的配套卷中。随后的多阶段整理流程将这些原始提取转换为AI-ready的监督信号。使用FlipVQA-Miner，我们构建了包含83K问答对和视觉问答对的FlipVQA-83K数据集，覆盖11个学术领域，在比手动标注节省50倍成本的同时保持高结构保真度（F1>0.96）。在FlipVQA-83K上微调的模型表现出显著改进的推理能力和跨领域泛化能力，建立了人类知识引导的数据整理可扩展范式。我们的数据集和完整的数据生成和整理方法可在https://github.com/OpenDCAI/DataFlow-VQA找到。
英文摘要：
Textbooks are among the richest repositories of human-verified reasoning knowledge, yet their complex layouts contain multi-column typesetting, cross-page question answer separation, and interleaved figures, make automated extraction of structured QA and VQA pairs extremely challenging. Existing alternatives either synthesize data from scratch, which lacks authentic problem contexts, or rely on costly expert annotation that cannot scale. We propose $\textbf{FlipVQA-Miner}$, an automated pipeline that resolves long-range logical dependencies and cross-page discontinuities in OCR-parsed documents, recovering coherent question--answer--figure associations even when answers reside in separate companion volumes. A subsequent multi-stage curation pipeline transforms these raw extractions into AI-ready supervision signals. Using FlipVQA-Miner, we construct $\textbf{FlipVQA-83K}$, comprising 83K QA and VQA pairs spanning 11 academic disciplines, at a $\textbf{50$\times$}$ cost saving compared to manual annotation while maintaining high structural fidelity ($F_1 > 0.96$). Models fine-tuned on FlipVQA-83K demonstrate significantly improved reasoning ability and cross-domain generalization, establishing a scalable paradigm for human-knowledge-grounded data curation. Our dataset and the complete data generating and curating methods can be found in https://github.com/OpenDCAI/DataFlow-VQA .
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
