小型视觉-语言模型在定性力学问题上的评估
Evaluation of Small Vision-Language Models on Qualitative Mechanical Problems
浏览论文内容
中文总结 AI 辅助
本研究评估Gemma-3和Qwen-VL两个多模态模型解读力学问题图像的能力，通过思维链评估其推理过程，对比答案测准确率，为小型视觉-语言模型在定性力学问题上的应用提供评估依据。
中文摘要 AI 辅助
定性力学问题求解（QMPS）指解决力学领域的定性问题，这类问题仅需极少的学科特定信息，无需任何可靠的定量计算，通常通过定性推理和常识知识来求解。QMPS是人类智能的重要方面，能让我们处理从打开水龙头这类简单日常任务，到急诊医学、管道工程、驾驶等多个领域高要求、高收入工作中的复杂任务。雇主常使用贝内特机械理解测验（BMCT）评估求职者解决此类问题的能力。本研究评估两个最先进的多模态模型Gemma-3和Qwen-VL，通过引导生成逐步思维链（CoT）及最终答案，考察它们解读力学问题图像的能力。每张图像固有地编码了真实的定性事实，如齿轮的接触点、支撑关系和相对重量，我们用这些事实评估每个模型的空间和常识推理能力。我们评估每条思维链的连贯性、完整性和逻辑推进性以考察模型的思维过程，并将最终答案与验证过的解决方案对比以测量准确率。
英文摘要
Qualitative mechanical problem-solving (QMPS) refers to solving qualitative problems from the mechanical domain. Qualitative problems can be solved with minimal discipline-specific information, without any robust quantitative calculation, generally by using qualitative reasoning and commonsense knowledge. QMPS is a vital aspect of human intelligence that allows us to tackle a wide range of tasks, from simple everyday ones such as turning on a tap to complex tasks in highly demanding and well-paying jobs in various fields, e.g., emergency medicine, plumbing, driving, etc. Employers often use the Bennett Mechanical Comprehension Test (BMCT) to evaluate job candidates' ability to solve such problems. In this work, we assess two state-of-the-art multimodal models, Gemma-3 and Qwen-VL, on their ability to interpret mechanical problem images by eliciting a step-by-step chain of thought (CoT) and a final answer. Each image inherently encodes ground-truth qualitative facts, such as contact points in gears, support relations, and relative weights, which we use to evaluate each model's spatial and commonsense reasoning capabilities. We assess each chain for coherence, completeness, and logical progression to assess each model's thought process, and final answers are compared to verified solutions to measure accuracy.
发表机构
- Louisiana State University of New Orleans（新奥尔良路易斯安那州立大学）
机构由 AI 辅助整理，请以论文原文为准。
