arXiv 2504.123342026-05-10cs.CL

QM-ToT: 一种用于量化模型的医学树状思维推理框架

QM-ToT: A Medical Tree of Thoughts Reasoning Framework for Quantized Model

City University of Hong Kong(Dongguan)（香港城市大学（东莞））；Hong Kong Polytechnic University（香港理工大学）；City University of Hong Kong（香港城市大学）

Zongxian Yang, Jiayu Qian, Kay Chen Tan, Hau-San Wong, Yulong Chen, Haoyu Zhang, Zhi-An Huang

备注：: Accepted by ICIC 2026 Poster

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出QM-ToT框架，通过树状思维方法分解复杂医疗问题，提升INT4量化模型在MedQAUSMLE数据集上的性能，使LLaMA2-70b模型准确率从34%提升至50%。

摘要：

大型语言模型（LLMs）在专门的生物医学任务中面临显著挑战，由于医疗推理的复杂性和临床数据的敏感性。现有LLMs在处理复杂的医学术语和需要准确的临床见解时表现不佳，导致在资源受限部署中量化后性能下降。为此，我们提出了量化医学树状思维（QM-ToT），一种基于路径的推理框架。QM-ToT利用树状思维（ToT）推理方法将复杂医疗问题分解为可管理的子任务，并结合评估者评估层。该框架在具有挑战性的MedQAUSMLE数据集上显著提升了INT4量化模型的性能。具体而言，我们展示了LLaMA2-70b模型的准确率从34%提升至50%，LLaMA-3.1-8b模型的准确率从58.77%提升至69.49%。此外，我们还提出了一种基于ToT的数据蒸馏方法。与传统蒸馏方法相比，我们仅使用3.9%的数据，实现了86.27%的提升。本文首次展示了ToT在复杂生物医学任务中显著提升性能的潜力，为未来在资源受限医疗环境中部署高性能量化LLM奠定了重要基础。

英文摘要：

Large language models (LLMs) face significant challenges in specialized biomedical tasks due to the inherent complexity of medical reasoning and the sensitive nature of clinical data. Existing LLMs often struggle with intricate medical terminology and the need for accurate clinical insights, leading to performance reduction when quantized for resource-constrained deployment. To address these issues, we propose Quantized Medical Tree of Thought (QM-ToT), a path-based reasoning framework. QM-ToT leverages a Tree of Thought (ToT) reasoning approach to decompose complex medical problems into manageable subtasks, coupled with evaluator assessment layers. This framework facilitates substantial performance improvements in INT4-quantized models on the challenging MedQAUSMLE dataset. Specifically, we demonstrate a remarkable accuracy increase from 34% to 50% for the LLaMA2-70b model and from 58.77% to 69.49% for LLaMA-3.1-8b. Besides, we also proposed an effect data distillation method based on ToT. Compared to the traditional distillation method, we achieved an improvement of 86. 27% while using only 3.9% of the data.This work, for the first time, showcases the potential of ToT to significantly enhance performance on complex biomedical tasks, establishing a crucial foundation for future advances in deploying high-performing quantized LLM in resource-limited medical settings.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。