基于大语言模型的特征型AI智能体
Characteristic AI Agents via Large Language Models
AI总结：
本研究针对大语言模型构建特征型AI智能体的研究空白，创建了包含数据集、技术和评估指标的基准，并构建了Character100数据集，通过全面评估揭示了提升LLMs角色扮演能力的潜在方向。
摘要：
大语言模型（LLMs）的进步显著提升了聊天机器人系统的性能。许多研究者致力于为聊天机器人赋予个性特征。尽管已有利用LLMs开发角色驱动聊天机器人的商业产品，但值得注意的是，该领域的学术研究仍相对匮乏。我们的研究聚焦于通过在不同场景下模拟真实个体，探究LLMs在构建特征型AI智能体方面的表现。现有研究主要集中于基于简单档案的角色扮演。针对这一研究空白，我们为特征型AI智能体任务创建了一个基准，包括数据集、技术和评估指标。我们为该基准构建了一个名为“Character100”的数据集，包含维基百科上访问量最高的人物，供语言模型进行角色扮演。利用构建的数据集，我们对LLMs在各种场景下进行了全面评估。此外，我们设计了一套自动评估指标用于定量性能评估。实验结果凸显了进一步提升LLMs在构建特征型AI智能体方面能力的潜在方向。该基准可在https://github.com/nuaa-nlp/Character100获取。
英文摘要：
The advancement of Large Language Models (LLMs) has led to significant enhancements in the performance of chatbot systems. Many researchers have dedicated their efforts to the development of bringing characteristics to chatbots. While there have been commercial products for developing role-driven chatbots using LLMs, it is worth noting that academic research in this area remains relatively scarce. Our research focuses on investigating the performance of LLMs in constructing Characteristic AI Agents by simulating real-life individuals across different settings. Current investigations have primarily focused on act on roles with simple profiles. In response to this research gap, we create a benchmark for the characteristic AI agents task, including dataset, techniques, and evaluation metrics. A dataset called ``Character100'' is built for this benchmark, comprising the most-visited people on Wikipedia for language models to role-play. With the constructed dataset, we conduct comprehensive assessment of LLMs across various settings. In addition, we devise a set of automatic metrics for quantitative performance evaluation. The experimental results underscore the potential directions for further improvement in the capabilities of LLMs in constructing characteristic AI agents. The benchmark is available at https://github.com/nuaa-nlp/Character100.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
