arXivDaily arXiv每日学术速递 周一至周五更新
​通知:联邦节假日调休:哥伦布日 / 原住民日(Columbus Day / Indigenous Peoples' Day),今日(13日)无更新。
arXiv 2505.08143cs.HCcs.AI

由大语言模型(LLM)与人类撰写的新冠疫情信息解释的沟通风格及读者偏好:一项案例研究

Communication styles and reader preferences of LLM- and human-authored COVID-19 information explanations: a case study

School of Interactive Computing, Georgia Institute of Technology(交互计算学院,佐治亚理工学院);Amazon(亚马逊);Siebel School of Computing and Data Science, University of Illinois Urbana-Champaign(Siebel计算与数据科学学院,伊利诺伊大学厄巴纳-香槟分校)

Jiawei Zhou, Kritika Venkatachalam, Minje Choi, Koustuv Saha, Munmun De Choudhury

AI总结:

本研究以健康事实核查为场景,对比LLM与人类新冠信息解释的沟通风格,发现LLM内容在说服性等质量指标上较弱但受读者偏好,揭示LLM在健康领域的潜力与局限。

摘要:

随着大语言模型(LLM)在信息辅助领域的广泛应用,考察其与人类沟通风格及价值观的契合度至关重要。本研究聚焦健康事实核查领域,该领域中有效沟通对于纠正误解、建立信任至关重要。尽管近期已有研究探索了LLM在事实核查与健康沟通中的应用,但LLM与人类的沟通风格差异及读者相关感知仍未得到充分研究。我们构建了包含1498条健康错误信息主张及权威事实核查机构解释的数据集,并针对不准确的健康信息生成了LLM响应。基于健康沟通理论,我们从信息语言特征、发送者说服策略、接收者价值观契合度三个维度评估沟通风格;同时通过99名参与者的盲评评估读者感知。结果显示,LLM生成的内容在说服策略、确定性表达及社会价值观与道德基础契合度方面得分显著更低,但参与者却更偏好LLM内容,超60%的反馈因LLM内容的清晰性、完整性与说服性而倾向于选择它。这种偏好与结构化呈现、清晰性及中立语气相关,尽管LLM内容的细微差别或严谨性有所降低,但仍能传递出完整性与专业性。这些发现凸显了LLM在健康沟通与事实核查中的潜力与局限,表明读者偏好未必与既定的沟通质量衡量标准相对应。

英文摘要:

With the wide adoption of large language models (LLMs) in information assistance, it is essential to examine their alignment with human communication styles and values. We situate this study within health fact-checking, where effective communication is critical for correcting misconceptions and building trust. Although recent studies have explored LLMs for fact-checking and health communication, differences between LLM and human communication styles and associated reader perceptions remain under-explored. We compiled a dataset of 1,498 health misinformation claims and explanations from authoritative fact-checking organizations and generated LLM responses to inaccurate health information. Drawing on health communication theories, we evaluated communication styles across three dimensions: information linguistic features, sender persuasive strategies, and receiver value alignments. We further assessed reader perceptions through a blinded evaluation with 99 participants. LLM-generated articles scored significantly lower in persuasive strategies, certainty expressions, and alignment with social values and moral foundations. However, participants strongly preferred LLM content, with over 60% of responses favoring LLM articles for clarity, completeness, and persuasiveness. This preference was associated with structured presentation, clarity, and neutral tone, which may convey completeness and professionalism despite reduced nuance or rigor. These findings highlight both the potential and limitations of LLMs in health communication and fact-checking, suggesting that reader preference may not necessarily correspond to established measures of communication quality.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑