以眼还眼：利用计算机图形学问题评估 GPT-4o 的视觉感知技能与几何推理技能
An Eye for an AI: Evaluating GPT-4o's Visual Perception Skills and Geometric Reasoning Skills Using Computer Graphics Questions
AI总结：
本文构建两类计算机图形学问题数据集，评估 GPT-4o 的视觉感知与几何推理能力，发现其虽具独立解题潜力但准确性和质量仍有限，并提出将生成式人工智能融入 CG 教学的新方法。
摘要：
计算机图形学（CG）是计算机科学（CS）中的一个热门领域，但许多学生觉得这门课很难，因为它需要大量技能，例如数学、编程、几何推理和创造力。在过去几年中，研究人员一直在探索如何利用生成式人工智能（GenAI）的能力来改进教学。在计算机科学领域，大部分研究都集中在入门级计算课程上。最近一项评估大语言模型（LLM）GPT-4（仅文本）在计算机图形学问题上表现的研究表明，其性能较差，并且依赖于对图像内容的详细描述，而这些描述通常需要用户具备相当的洞察力才能返回合理结果。迄今为止，还没有研究调查大模态模型（LMM），即多模态大语言模型，解决计算机图形学问题的能力，以及如何利用这些能力改进教学。在本研究中，我们构建了两个计算机图形学问题数据集，这些问题需要不同程度的视觉感知技能和几何推理技能，并在两个数据集上评估当前最先进的大模态模型 GPT-4o。我们发现，尽管 GPT-4o 在独立解决包含视觉信息的问题方面展现出巨大潜力，但生成结果的准确性和质量仍存在重大局限。我们提出了几种新颖的方法，供计算机图形学教育者在这些局限下将生成式人工智能融入计算机图形学教学。我们希望我们的指导方针能进一步鼓励计算机图形学课堂中的学习和参与。
英文摘要：
CG (Computer Graphics) is a popular field of CS (Computer Science), but many students find this topic difficult due to it requiring a large number of skills, such as mathematics, programming, geometric reasoning, and creativity. Over the past few years, researchers have investigated ways to harness the power of GenAI (Generative Artificial Intelligence) to improve teaching. In CS, much of the research has focused on introductory computing. A recent study evaluating the performance of an LLM (Large Language Model), GPT-4 (text-only), on CG questions, indicated poor performance and reliance on detailed descriptions of image content, which often required considerable insight from the user to return reasonable results. So far, no studies have investigated the abilities of LMMs (Large Multimodal Models), or multimodal LLMs, to solve CG questions and how these abilities can be used to improve teaching. In this study, we construct two datasets of CG questions requiring varying degrees of visual perception skills and geometric reasoning skills, and evaluate the current state-of-the-art LMM, GPT-4o, on the two datasets. We find that although GPT-4o exhibits great potential in solving questions with visual information independently, major limitations still exist to the accuracy and quality of the generated results. We propose several novel approaches for CG educators to incorporate GenAI into CG teaching despite these limitations. We hope that our guidelines further encourage learning and engagement in CG classrooms.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
