我思故我在：使用AwareBench对大型语言模型的意识进行基准测试
I Think, Therefore I am: Benchmarking Awareness of Large Language Models Using AwareBench
AI总结：
本文提出AwareBench基准及AwareEval数据集，从能力、使命、情感、文化和视角五个维度评估LLMs的自我意识与社会智能，实验表明多数模型难以完全识别自身能力与使命，强调其对AI对齐与安全的重要性。
摘要：
大型语言模型（LLMs）是否表现出类似人类的意识形式？本文介绍了AwareBench，一个旨在评估LLMs意识的基准。借鉴心理学和哲学理论，我们将LLMs的意识定义为其理解自身为AI模型并展现社会智能的能力。随后，我们将LLMs的意识划分为五个维度，包括能力、使命、情感、文化和视角。基于此分类，我们创建了名为AwareEval的数据集，包含二元、多选和开放式问题，以评估LLMs对特定意识维度的理解。我们在13个LLMs上进行的实验显示，多数模型难以完全识别自身能力与使命，但展现出良好的社会智能。最后，我们将LLMs意识与AI对齐及安全性相联系，强调其对LLMs可信与伦理发展的意义。数据集和代码可在https://github.com/HowieHwong/Awareness-in-LLM获取。
英文摘要：
Do large language models (LLMs) exhibit any forms of awareness similar to humans? In this paper, we introduce AwareBench, a benchmark designed to evaluate awareness in LLMs. Drawing from theories in psychology and philosophy, we define awareness in LLMs as the ability to understand themselves as AI models and to exhibit social intelligence. Subsequently, we categorize awareness in LLMs into five dimensions, including capability, mission, emotion, culture, and perspective. Based on this taxonomy, we create a dataset called AwareEval, which contains binary, multiple-choice, and open-ended questions to assess LLMs' understandings of specific awareness dimensions. Our experiments, conducted on 13 LLMs, reveal that the majority of them struggle to fully recognize their capabilities and missions while demonstrating decent social intelligence. We conclude by connecting awareness of LLMs with AI alignment and safety, emphasizing its significance to the trustworthy and ethical development of LLMs. Our dataset and code are available at https://github.com/HowieHwong/Awareness-in-LLM.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
