不是“什么”，而是“如何”：LLM 响应框架的沟通审计
Not What, But How: A Framework for Auditing LLM Responses across Positioning, Generalization, Anthropomorphism, and Maxims
AI总结：
提出 FRANZ 框架，从文化定位、概括性语言、拟人化线索和对话准则遵守四个维度审计 LLM 对主观问题的响应框架，并构建 SQUARE 语料库进行实证分析。
摘要：
大型语言模型（LLM）越来越多地被用于回答主观性、信息寻求型问题，在这些问题中，用户对响应的沟通方式敏感，而不仅仅是答案是否正确。现有的针对主观文化查询的 LLM 评估主要关注事实正确性，忽略了响应的框架方式。为此，我们引入了 FRANZ，一个自动化的响应特征化框架，用于沿四个维度对 LLM 响应进行沟通审计：文化定位、概括性语言的使用、拟人化线索以及对对话准则的遵守。为了支持这一评估，我们贡献了 SQUARE——一个包含来自 57 个子版块的 376k 个主观问题的语料库，并映射到 7 个国家和 19 个问题类别。我们通过评分三个开放权重 LLM 的响应来展示 FRANZ 的适用性。我们观察到，LLM 在采用每种响应特征的频率上显示出统计显著差异。与单维度审计不同，FRANZ 揭示了内部定位和拟人化是正相关的，且相关程度因国家而异，为识别框架差异提供了诊断视角。
英文摘要：
Large language models (LLMs) are being increasingly used to answer subjective, information-seeking questions, where users are sensitive to how responses are communicated, not just whether the answers are correct. Existing LLM evaluations for subjective cultural queries largely focus on factual correctness, ignoring how the response is framed. To this end, we introduce FRANZ, an automated FRAmework for respoNse characteriZation to conduct communicative audit of LLM responses along four dimensions: cultural positioning, use of generalizing language, anthropomorphic cues, and adherence to conversational maxims. To enable this evaluation, we contribute SQUARE - a corpus of 376k subjective questions sourced from 57 subreddits, and mapped to 7 countries and 19 question categories. We demonstrate FRANZ's applicability by scoring responses from three open-weight LLMs. We observe that LLMs show statistically significant differences in the frequency with which they employ each response characteristic. Unlike single-dimensional audits, FRANZ reveals that insider positioning and anthropomorphism are positively coupled, with the degree of coupling varying by country, providing a diagnostic lens for identifying framing divergences.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
