在GMAT上评估大型语言模型：对商业教育未来的启示
Evaluating Large Language Models on the GMAT: Implications for the Future of Business Education
AI总结：
本研究引入首个评估七大主流LLM在GMAT考试中表现的基准，发现多数模型优于人类考生，其中GPT-4 Turbo表现最佳，凸显了AI在商业教育中的潜力及谨慎应用的必要性。
摘要：
人工智能（AI）的快速发展，特别是在大型语言模型（LLMs）和生成式AI领域，为各个领域的应用开辟了新途径，但其在商业教育中的作用仍未被充分探索。本研究引入了首个基准，用于评估七个主要LLM在GMAT（研究生商业项目录取过程中的关键考试）上的表现，包括OpenAI的模型（GPT-3.5 Turbo、GPT-4和GPT-4 Turbo）、Google的模型（PaLM 2、Gemini 1.0 Pro）和Anthropic的模型（Claude 2和Claude 2.1）。我们的分析表明，大多数LLM的表现优于人类考生，其中GPT-4 Turbo不仅优于其他模型，还超过了顶尖商学院研究生的平均分。通过案例研究，本研究考察了GPT-4 Turbo解释答案、评估回答、识别错误、定制指令和生成替代场景的能力。与前身相比，最新的LLM版本（GPT-4 Turbo、Claude 2.1和Gemini 1.0 Pro）在推理任务上表现出显著改进，突显了其在复杂问题解决中的潜力。尽管AI在教育、评估和辅导方面的前景明确，但仍存在挑战。我们的研究不仅揭示了LLM的学术潜力，还强调了在教育中谨慎开发和应用AI的必要性。随着AI技术的进步，建立AI交互的框架和协议、验证AI生成内容的准确性、确保全球多样化学习者的访问权限，以及创造AI支持人类专业知识的教育环境势在必行。本研究为进一步探索负责任地使用AI以丰富教育体验、改进考试准备和评估方法奠定了基础。
英文摘要：
The rapid evolution of artificial intelligence (AI), especially in the domain of Large Language Models (LLMs) and generative AI, has opened new avenues for application across various fields, yet its role in business education remains underexplored. This study introduces the first benchmark to assess the performance of seven major LLMs, OpenAI's models (GPT-3.5 Turbo, GPT-4, and GPT-4 Turbo), Google's models (PaLM 2, Gemini 1.0 Pro), and Anthropic's models (Claude 2 and Claude 2.1), on the GMAT, which is a key exam in the admission process for graduate business programs. Our analysis shows that most LLMs outperform human candidates, with GPT-4 Turbo not only outperforming the other models but also surpassing the average scores of graduate students at top business schools. Through a case study, this research examines GPT-4 Turbo's ability to explain answers, evaluate responses, identify errors, tailor instructions, and generate alternative scenarios. The latest LLM versions, GPT-4 Turbo, Claude 2.1, and Gemini 1.0 Pro, show marked improvements in reasoning tasks compared to their predecessors, underscoring their potential for complex problem-solving. While AI's promise in education, assessment, and tutoring is clear, challenges remain. Our study not only sheds light on LLMs' academic potential but also emphasizes the need for careful development and application of AI in education. As AI technology advances, it is imperative to establish frameworks and protocols for AI interaction, verify the accuracy of AI-generated content, ensure worldwide access for diverse learners, and create an educational environment where AI supports human expertise. This research sets the stage for further exploration into the responsible use of AI to enrich educational experiences and improve exam preparation and assessment methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
