arXiv 2407.095192024-06-19cs.AIcs.CL

《将GPT-4o置于考验之下：语言、视觉、语音及多模态能力的综合评估》

Putting GPT-4o to the Sword: A Comprehensive Evaluation of Language, Vision, Speech, and Multimodal Proficiency

University of Guelph（圭尔夫大学）；University of North Texas（北德克萨斯大学）；Iowa State University（爱荷华州立大学）；Zayed University（扎耶德大学）；University of Texas at Dallas（德克萨斯大学达拉斯分校）；National University of Computer & Emerging Sciences（国立计算机与新兴科学大学）

Sakib Shahriar, Brady Lund, Nishith Reddy Mannuru, Muhammad Arbab Arshad, Kadhim Hayawi, Ravi Varma Kumar Bevara, Aashrith Mannuru, Laiba Batool

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本研究综合评估GPT-4o的语言、视觉、语音及多模态能力，发现其在语言推理、少样本学习及多模态任务上表现优异，但在复杂模糊输入处理上存在局限，提出需完善评估体系并优化相关技术以适配真实场景。

摘要：

随着大语言模型（LLMs）持续发展，评估其综合能力对其在各领域的应用具有重要意义。本研究对GPT-4o的语言、视觉、语音及多模态能力开展了综合评估。研究采用标准化考试题、推理任务和翻译评估来检验模型的语言能力；此外，通过图像分类、目标识别任务以及口音分类测试GPT-4o的视觉与语音能力；多模态评估则考查模型整合视觉与语言数据的表现。研究结果显示，GPT-4o在语言与推理能力的多个领域都展现出高准确率与高效率，在需要少样本学习（few-shot learning）的任务中表现突出；与前代模型相比，GPT-4o在多模态任务上也有显著提升。但该模型在处理复杂、模糊输入时表现存在波动，且面临局限，尤其体现在音频与视觉能力上。本文强调，需要更全面的基准（benchmarks）与稳健的评估框架，涵盖涉及人类判断的定性评估以及错误分析。未来工作应聚焦于扩充数据集、探究基于提示的评估方法，以及优化少样本学习技术，以测试模型在真实场景中的实际适用性与表现。

英文摘要：

As large language models (LLMs) continue to advance, evaluating their comprehensive capabilities becomes significant for their application in various fields. This research study comprehensively evaluates the language, vision, speech, and multimodal capabilities of GPT-4o. The study employs standardized exam questions, reasoning tasks, and translation assessments to assess the model's language capability. Additionally, GPT-4o's vision and speech capabilities are tested through image classification and object recognition tasks, as well as accent classification. The multimodal evaluation assesses the model's performance in integrating visual and linguistic data. Our findings reveal that GPT-4o demonstrates high accuracy and efficiency across multiple domains in language and reasoning capabilities, excelling in tasks that require few-shot learning. GPT-4o also provides notable improvements in multimodal tasks compared to its predecessors. However, the model shows variability and faces limitations in handling complex and ambiguous inputs, particularly in audio and vision capabilities. This paper highlights the need for more comprehensive benchmarks and robust evaluation frameworks, encompassing qualitative assessments involving human judgment as well as error analysis. Future work should focus on expanding datasets, investigating prompt-based assessment, and enhancing few-shot learning techniques to test the model's practical applicability and performance in real-world scenarios.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。