arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2209.01515cs.CLcs.AI
收藏

大型语言模型知道人类所知道的东西吗?

Do Large Language Models know what humans know?

UC San Diego(加州大学圣地亚哥分校)

Sean Trott, Cameron Jones, Tyler Chang, James Michaelov, Benjamin Bergen

AI总结:

本文通过让 GPT-3 与人类完成语言版错误信念任务,检验语言接触能否解释心理状态推理能力,发现模型虽优于随机但不及人类,说明语言统计学习之外仍需其他机制。

摘要:

人类能够将信念归因于他人。然而,目前尚不清楚这种能力在多大程度上源于先天的生物禀赋,或源于儿童发展过程中积累的经验,尤其是接触描述他人心理状态的语言。我们通过评估接触大量人类语言的模型是否对书面段落中角色隐含的知识状态表现出敏感性,来检验语言接触假说的可行性。在预注册分析中,我们向人类参与者和大型语言模型 GPT-3 呈现了语言版本的错误信念任务。两者都对他人的信念表现出敏感性,但尽管该语言模型的表现显著优于随机水平,它却不如人类表现得好,也无法解释人类行为的全部范围——尽管它接触的语言比一个人一生中接触的还要多。这表明,虽然从语言接触中进行统计学习可能在一定程度上解释人类如何发展推理他人心理状态的能力,但其他机制也同样发挥作用。

英文摘要:

Humans can attribute beliefs to others. However, it is unknown to what extent this ability results from an innate biological endowment or from experience accrued through child development, particularly exposure to language describing others' mental states. We test the viability of the language exposure hypothesis by assessing whether models exposed to large quantities of human language display sensitivity to the implied knowledge states of characters in written passages. In pre-registered analyses, we present a linguistic version of the False Belief Task to both human participants and a Large Language Model, GPT-3. Both are sensitive to others' beliefs, but while the language model significantly exceeds chance behavior, it does not perform as well as the humans, nor does it explain the full extent of their behavior -- despite being exposed to more language than a human would in a lifetime. This suggests that while statistical learning from language exposure may in part explain how humans develop the ability to reason about the mental states of others, other mechanisms are also responsible.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑