arXivDaily arXiv每日学术速递 周一至周五更新
北美退伍军人节(Veterans Day),12 日无更新
arXiv 2602.05059cs.AI
收藏

在图论中解决和未解决的问题上评估大语言模型:对计算教育的启示

Evaluating Large Language Models on Solved and Unsolved Problems in Graph Theory: Implications for Computing Education

Department of Computer Science(计算机科学系);Ball State University(巴尔的摩州立大学);School of Artificial Intelligence and Data Science(人工智能与数据科学学院);Jio Institute(乔研究所)

Adithya Kulkarni, Mohna Chakraborty, Jay Bagga

AI总结:

本研究评估了大语言模型在图论已解决和未解决问题上的表现,发现其在已有知识探索上有效,但在需要创新数学洞察的任务中存在局限,对计算教育有重要启示。

摘要:

大语言模型越来越多地被学生用来探索计算机科学中的高级内容,包括图论。随着这些工具被整合到本科生和研究生课程中,了解它们如何可靠地支持数学严谨性思考变得至关重要。本研究检验了一个LLM在两个相关的图论问题上的表现:一个已解决的问题,涉及线图的优雅性,以及一个目前尚无解决方案的开放性问题。我们使用了一个八阶段的评估协议,反映了真实的数学探究,包括解释、探索、策略形成和证明构建。模型在已解决的问题上表现良好,产生了正确的定义,识别了相关结构,回忆了适当的结论而没有幻觉,并构建了一个经过图论专家确认的有效证明。对于开放性问题,模型生成了连贯的解释和合理的探索策略,但并未接近解决方案。它没有伪造结果,而是承认了不确定性,这与明确的提示指令一致,这些指令要求模型避免编造定理或未经支持的声明。这些发现表明,LLM可以支持已确立内容的探索,但在需要新颖数学洞察或批判性结构推理的任务中仍有限制。对于计算教育,这一区别突显了引导学生使用LLM进行概念探索的重要性,同时依靠独立验证和严谨论证来解决正式问题。

英文摘要:

Large Language Models are increasingly used by students to explore advanced material in computer science, including graph theory. As these tools become integrated into undergraduate and graduate coursework, it is important to understand how reliably they support mathematically rigorous thinking. This study examines the performance of a LLM on two related graph theoretic problems: a solved problem concerning the gracefulness of line graphs and an open problem for which no solution is currently known. We use an eight stage evaluation protocol that reflects authentic mathematical inquiry, including interpretation, exploration, strategy formation, and proof construction. The model performed strongly on the solved problem, producing correct definitions, identifying relevant structures, recalling appropriate results without hallucination, and constructing a valid proof confirmed by a graph theory expert. For the open problem, the model generated coherent interpretations and plausible exploratory strategies but did not advance toward a solution. It did not fabricate results and instead acknowledged uncertainty, which is consistent with the explicit prompting instructions that directed the model to avoid inventing theorems or unsupported claims. These findings indicate that LLMs can support exploration of established material but remain limited in tasks requiring novel mathematical insight or critical structural reasoning. For computing education, this distinction highlights the importance of guiding students to use LLMs for conceptual exploration while relying on independent verification and rigorous argumentation for formal problem solving.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑