arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2306.04563cs.AIcs.CLcs.HCcs.LG
收藏

ChatGPT 很有趣,但它并不好笑!幽默对大型语言模型仍具挑战

ChatGPT is fun, but it is not funny! Humor is still challenging Large Language Models

Institute for Software Technology, German Aerospace Center (DLR)(德国航空航天中心(DLR)软件技术研究所);Technical University Darmstadt(达姆施塔特工业大学);Centre for Cognitive Science, TU Darmstadt(达姆施塔特工业大学认知科学中心);Hessian Center for AI (hessian.AI)(黑森人工智能中心(hessian.AI))

Sophie Jentzsch, Kristian Kersting

AI总结:

本文通过基于提示的笑话生成、解释与检测实验评估 ChatGPT 的幽默感,发现其输出高度重复、会编造解释且易受笑话特征误导,表明计算幽默仍未解决。

摘要:

幽默是人类交流的核心方面,迄今为止人工智能体尚未解决这一问题。大型语言模型(LLM)越来越能够捕捉隐含和上下文信息。尤其是,OpenAI 的 ChatGPT 最近获得了巨大的公众关注。这个基于 GPT3 的模型几乎看起来能在人类层面上交流,甚至能讲笑话。幽默是人类交流的重要组成部分。但 ChatGPT 真的好笑吗?我们对 ChatGPT 的幽默感进行了测试。在围绕笑话的一系列探索性实验中,即生成、解释和检测,我们试图理解 ChatGPT 把握和再现人类幽默的能力。由于无法访问模型本身,我们采用了基于提示的实验。我们的实证证据表明,笑话并非硬编码的,但大多也不是由模型新生成的。在生成的 1008 个笑话中,超过 90% 都是相同的 25 个笑话。该系统能准确解释有效笑话,但也会为无效笑话编造虚构的解释。笑话的典型特征会在笑话分类中误导 ChatGPT。ChatGPT 尚未解决计算幽默问题,但它可能是迈向“有趣”机器的一大步。

英文摘要:

Humor is a central aspect of human communication that has not been solved for artificial agents so far. Large language models (LLMs) are increasingly able to capture implicit and contextual information. Especially, OpenAI's ChatGPT recently gained immense public attention. The GPT3-based model almost seems to communicate on a human level and can even tell jokes. Humor is an essential component of human communication. But is ChatGPT really funny? We put ChatGPT's sense of humor to the test. In a series of exploratory experiments around jokes, i.e., generation, explanation, and detection, we seek to understand ChatGPT's capability to grasp and reproduce human humor. Since the model itself is not accessible, we applied prompt-based experiments. Our empirical evidence indicates that jokes are not hard-coded but mostly also not newly generated by the model. Over 90% of 1008 generated jokes were the same 25 Jokes. The system accurately explains valid jokes but also comes up with fictional explanations for invalid jokes. Joke-typical characteristics can mislead ChatGPT in the classification of jokes. ChatGPT has not solved computational humor yet but it can be a big leap toward "funny" machines.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑