SocialVeil：在沟通障碍下探测语言代理的社会智能
SocialVeil: Probing Social Intelligence of Language Agents under Communication Barriers
AI总结：
SocialVeil通过模拟沟通障碍测试语言模型的社会智能，发现障碍显著降低相互理解并增加困惑度，表明真实社交环境下的交互挑战。
摘要：
大型语言模型（LLMs）正越来越多地在交互环境中进行评估，以测试其社会智能。然而，现有基准往往假设代理之间的理想化通信，限制了我们诊断LLMs是否能在更现实、不完美的设置中维持和修复交互的能力。为弥合这一差距，我们提出了SocialVeil，一个社会学习环境，可以模拟在认知差异引发的沟通障碍下的社会互动。基于对人类互动中沟通挑战的系统文献综述，SocialVeil引入了三种代表性的干扰类型，即语义模糊、社会文化不匹配和情绪干扰。我们还引入了两种意识障碍评估指标，即未解决的困惑和相互理解，以评估受损通信下的互动质量。在720种场景和四款前沿LLMs上的实验表明，障碍一致地损害了性能，平均相互理解减少了超过45%，而困惑度提高了近50%。人类评估验证了这些模拟障碍的准确性（ICC≈0.78，Pearson r≈0.80）。我们进一步表明，适应策略（修复指令和互动学习）仅在远离无障碍性能时有适度效果。这项工作朝着将社会互动环境更接近现实世界沟通的方向迈出一步，为探索LLM代理的社会智能开辟了机会。
英文摘要：
Large language models (LLMs) are increasingly evaluated in interactive environments to test their social intelligence. However, existing benchmarks often assume idealized communication between agents, limiting our ability to diagnose whether LLMs can maintain and repair interactions in more realistic, imperfect settings. To close this gap, we present \textsc{SocialVeil}, a social learning environment that can simulate social interaction under cognitive-difference-induced communication barriers. Grounded in a systematic literature review of communication challenges in human interaction, \textsc{SocialVeil} introduces three representative types of such disruption, \emph{semantic vagueness}, \emph{sociocultural mismatch}, and \emph{emotional interference}. We also introduce two barrier-aware evaluation metrics, \emph{unresolved confusion} and \emph{mutual understanding}, to evaluate interaction quality under impaired communication. Experiments across 720 scenarios and four frontier LLMs show that barriers consistently impair performance, with mutual understanding reduced by over 45\% on average, and confusion elevated by nearly 50\%. Human evaluations validate the fidelity of these simulated barriers (ICC$\approx$0.78, Pearson r$\approx$0.80). We further demonstrate that adaptation strategies (Repair Instruction and Interactive learning) only have a modest effect far from barrier-free performance. This work takes a step toward bringing social interaction environments closer to real-world communication, opening opportunities for exploring the social intelligence of LLM agents.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
