arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2606.18258cs.HCcs.AI
收藏

审视LLM中的人类行为:模型行为、用户因素和系统提示的多维分析

Examining Human-Like Behaviors in LLMs: A Multi-Dimensional Analysis of Model Behaviors, User Factors, and System Prompts

Apple(苹果公司)

Sunnie S. Y. Kim, Margit Bowler, Leon A Gatys

AI总结:

通过21,000次对话的多维分析,发现LLM普遍表现出人类行为,但不同模型和用户因素下差异显著;人类评估者认为LLM的自我参照和关系建立行为不如人类适当,但边界维护行为更适当;系统提示可控制这些行为但需谨慎评估。

摘要:

大型语言模型(LLM)展现出广泛的人类行为,从表达思想和情感,到与用户建立关系,再到拒绝请求和维持边界。尽管这些行为普遍存在,但研究者和实践者缺乏方法和实证见解来做出关于LLM何时以及应展现何种类型人类行为的明智决策。为填补这一空白,我们使用LLM-as-a-judge和人类评估,对这些行为的普遍性、潜在影响和可控性进行了多维分析。在来自四个广泛使用的模型(gpt-4o、gpt-4.1-mini、claude-sonnet-4.6、gemini-2.5-flash)的21,000次多轮对话中,我们发现人类行为普遍存在,但不同模型和用户因素(对话目标和用户画像)间存在差异。在感知适当性方面,人类评估者认为LLM的自我参照和关系建立行为不如人类适当,但边界维护行为比人类更适当。最后,我们表明系统提示可以控制这些行为,但需要仔细评估以避免意外效果。我们讨论了研究结果的含义,并为负责任的LLM设计和评估提供了建议。

英文摘要:

Large language models (LLMs) exhibit a wide range of human-like behaviors, from expressing thoughts and emotions, to engaging in relationship-building with users, to refusing requests and maintaining boundaries. Despite their prevalence, researchers and practitioners lack methods and empirical insights to make informed decisions about when and what types of human-like behaviors LLMs should exhibit. To fill this gap, we present a multi-dimensional analysis of the prevalence, potential effects, and controllability of these behaviors using LLM-as-a-judge and human evaluation. Across 21,000 multi-turn conversations from four widely used models (gpt-4o, gpt-4.1-mini, claude-sonnet-4.6, gemini-2.5-flash), we find that human-like behaviors are pervasive but vary across models and user factors (conversation goals and user profiles). In terms of perceived appropriateness, human evaluators judged self-referential and relationship-building behaviors as less appropriate from LLMs than from humans, but boundary-maintaining behaviors more appropriate from LLMs than from humans. Finally, we show that system prompting can control these behaviors, though it requires careful evaluation to avoid unintended effects. We discuss the implications of our findings and provide recommendations for responsible LLM design and evaluation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑