我们能信任ChatGPT吗?ChatGPT可靠性的测量与特征刻画
In ChatGPT We Trust? Measuring and Characterizing the Reliability of ChatGPT
AI总结:
该研究通过10个数据集8个领域的5695个问题,首次大规模测量ChatGPT在通用问答场景的可靠性,发现其领域表现不均、易受系统角色和单字符对抗样本影响,为大语言模型的可靠性优化提供参考。
摘要:
随着ChatGPT的出现,用户获取信息的方式正经历范式转变。与传统搜索引擎不同,ChatGPT从模型自身检索知识并为用户生成答案。ChatGPT出色的问答(QA)能力在短时间内吸引了超1亿用户,但也引发了对其可靠性的担忧。本文在通用问答场景下对ChatGPT的可靠性开展了首次大规模测量,所用数据集经精心整理,包含10个数据集、8个领域的共5695个问题。我们发现ChatGPT的可靠性在不同领域存在差异,在法律和科学类问题上表现尤其不佳。我们还证明,OpenAI最初设计用于让用户调控ChatGPT行为的系统角色,会以难以察觉的方式影响ChatGPT的可靠性。我们进一步表明,ChatGPT易受对抗样本攻击,在某些情况下,哪怕单个字符的改动都会对其可靠性产生负面影响。我们认为本研究为ChatGPT的可靠性提供了宝贵洞见,并凸显了强化大语言模型(LLMs)可靠性与安全性的必要性。
英文摘要:
The way users acquire information is undergoing a paradigm shift with the advent of ChatGPT. Unlike conventional search engines, ChatGPT retrieves knowledge from the model itself and generates answers for users. ChatGPT's impressive question-answering (QA) capability has attracted more than 100 million users within a short period of time but has also raised concerns regarding its reliability. In this paper, we perform the first large-scale measurement of ChatGPT's reliability in the generic QA scenario with a carefully curated set of 5,695 questions across ten datasets and eight domains. We find that ChatGPT's reliability varies across different domains, especially underperforming in law and science questions. We also demonstrate that system roles, originally designed by OpenAI to allow users to steer ChatGPT's behavior, can impact ChatGPT's reliability in an imperceptible way. We further show that ChatGPT is vulnerable to adversarial examples, and even a single character change can negatively affect its reliability in certain cases. We believe that our study provides valuable insights into ChatGPT's reliability and underscores the need for strengthening the reliability and security of large language models (LLMs).
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。