arXiv 2506.075532026-01-13cs.AIq-bio.QM

GTR-CoT：图遍历作为视觉推理链用于分子结构识别

GTR-CoT: Graph Traversal as Visual Chain of Thought for Molecular Structure Recognition

Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；East China Normal University（华东师范大学）；Peking University（北京大学）；Shanghai Jiaotong University（上海交通大学）；Tongji University（同济大学）；Fudan University（复旦大学）；Northwestern Polytechnical University（西北工业大学）

Jingchao Wang, Yifan He, Haote Yang, Jiang Wu, Lingli Ge, Xingjian Wei, Yinfan Wang, Linye Li, Huijie Ao, Chengjin Liu, Bin Wang, Lijun Wu, Conghui He

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

GTR-CoT通过图遍历机制和强化学习提升手绘分子结构识别性能，构建首个细粒度评估基准。

摘要：

光学化学结构识别（OCSR）对于将分子图像转换为机器可读格式至关重要。尽管最近的视觉-语言模型（VLMs）显示出潜力，但其图像-描述方法在复杂分子结构和不一致注释方面往往力不从心。为了解决这些问题，我们引入了GTR-VL，其有两个关键创新：（1）图遍历作为视觉推理链机制，通过顺序的原子-键预测逐步解析分子图，模拟人类推理；（2）数据导向的“忠实识别所见”原则，将图像中的缩略结构与扩展注释对齐。对于手绘OCSR任务，其中数据集缺乏图注释且仅提供最终SMILES，我们应用强化学习使用GRPO方法，引入格式奖励、图奖励和SMILES奖励等机制。这种方法通过弱监督显著提升了手绘识别任务的性能。我们开发了GTR-1.3M，一个大规模指令微调数据集，具有修正的注释，并构建了MolRec-Bench，首个用于评估OCSR中图解析精度的细粒度基准。我们的两阶段训练方案包括对印刷图像的SFT训练和使用GRPO方法将能力转移到手绘任务。实验表明，GTR-VL在印刷和手绘数据集上均优于专用模型、化学领域VLMs和商业VLMs。

英文摘要：

Optical Chemical Structure Recognition (OCSR) is essential for converting molecular images into machine-readable formats. While recent vision-language models (VLMs) have shown promise, their image-captioning approach often struggles with complex molecular structures and inconsistent annotations. To address these issues, we introduce GTR-VL, featuring two key innovations: (1) the \textit{Graph Traversal as Visual Chain of Thought} mechanism that emulates human reasoning by incrementally parsing molecular graphs through sequential atom-bond predictions, and (2) the data-centric \textit{Faithfully Recognize What You've Seen} principle, which aligns abbreviated structures in images with their expanded annotations. For hand-drawn OCSR tasks, where datasets lack graph annotations and only provide final SMILES, we apply reinforcement learning using the GRPO method, introducing reward mechanisms like format reward, graph reward, and SMILES reward. This approach significantly enhances performance in hand-drawn recognition tasks through weak supervision. We developed GTR-1.3M, a large-scale instruction-tuning dataset with corrected annotations, and MolRec-Bench, the first benchmark for fine-grained evaluation of graph-parsing accuracy in OCSR. Our two-stage training scheme involves SFT training for printed images and the GRPO method for transferring capabilities to hand-drawn tasks. Experiments show that GTR-VL outperforms specialist models, chemistry-domain VLMs, and commercial VLMs on both printed and hand-drawn datasets.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。