arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.02547cs.AIcs.CL
收藏

将认知任务整合进面向大模型的通用人工智能测试

Integration of cognitive tasks into artificial general intelligence test for large models

Southern University of Science and Technology(南方科技大学);Shanghai AI Laboratory(上海人工智能实验室);Tencent AI lab(腾讯AI实验室);iFLYTEK AI Research(科大讯飞人工智能研究院);Beijing Institute of Technology(北京理工大学);Peking University(北京大学);University of Macau(澳门大学);Tsinghua University(清华大学)

Youzhi Qu, Chen Wei, Penghui Du, Wenxin Che, Chi Zhang, Wanli Ouyang, Yatao Bian, Feiyang Xu, Bin Hu, Kai Du, Haiyan Wu, Jia Liu, Quanying Liu

AI总结:

针对当前大模型评估缺乏统一多维度智能衡量框架的问题,提出受认知科学启发的AGI综合测试框架,覆盖多类智能维度,融入沉浸式虚拟社区,可引导大模型针对性优化并加速其融入人类社会。

摘要:

在大模型的发展演进过程中,必须开展性能评估以衡量其能力,并在实际应用前确保安全性。然而,当前的模型评估主要依赖特定任务与数据集,缺乏用于衡量大模型多维度智能的统一框架。基于这一视角,我们倡导构建一套受认知科学启发的通用人工智能(AGI)综合测试框架,旨在满足能力不断增强的大模型的测试需求。\n这套受认知科学启发的AGI测试覆盖智能的全维度范畴,包括晶体智力、流体智力、社会智力以及具身智能。为评估大模型的多维度智能,该AGI测试包含一系列从人类智力测试中借鉴而来、经过精心设计的认知测试,并将其自然融入一个沉浸式虚拟社区中。我们提出,应随着大模型的技术进步同步提升AGI测试任务的复杂度,同时强调必须对测试结果进行可解释性分析,以避免假阴性与假阳性问题。我们认为,受认知科学启发的AGI测试将有效引导大模型在特定智能维度上的针对性优化,并加速大模型融入人类社会的进程。

英文摘要:

During the evolution of large models, performance evaluation is necessarily performed to assess their capabilities and ensure safety before practical application. However, current model evaluations mainly rely on specific tasks and datasets, lacking a united framework for assessing the multidimensional intelligence of large models. In this perspective, we advocate for a comprehensive framework of cognitive science-inspired artificial general intelligence (AGI) tests, aimed at fulfilling the testing needs of large models with enhanced capabilities. The cognitive science-inspired AGI tests encompass the full spectrum of intelligence facets, including crystallized intelligence, fluid intelligence, social intelligence, and embodied intelligence. To assess the multidimensional intelligence of large models, the AGI tests consist of a battery of well-designed cognitive tests adopted from human intelligence tests, and then naturally encapsulates into an immersive virtual community. We propose increasing the complexity of AGI testing tasks commensurate with advancements in large models and emphasizing the necessity for the interpretation of test results to avoid false negatives and false positives. We believe that cognitive science-inspired AGI tests will effectively guide the targeted improvement of large models in specific dimensions of intelligence and accelerate the integration of large models into human society.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑