面向癌症沟通的大语言模型：评估生成式AI的语言质量、安全性与可及性
Large Language Models for Cancer Communication: Evaluating Linguistic Quality, Safety, and Accessibility in Generative AI
AI总结：
本研究通过混合方法框架评估8款通用与医学大语言模型在癌症科普沟通中的表现，发现通用模型语言质量与情感性更优、医学模型可及性更强但安全风险更高，为优化AI健康内容与数字健康工具开发提供参考。
摘要：
针对乳腺癌与宫颈癌的有效科普沟通仍是一项长期存在的健康挑战，公众在癌症预防、筛查和治疗方面的认知存在显著缺口，可能导致诊断延误与治疗不足。本研究评估了大语言模型（LLMs）在生成准确、安全、可及的癌症相关信息以辅助患者理解方面的能力与局限性。我们采用混合方法评估框架，从语言质量、安全与可信度、沟通可及性与情感性三个维度，评估了5个通用型LLM和3个医学专用LLM。研究方法结合了量化指标、专家定性评级，并使用Welch's ANOVA、Games-Howell检验和Hedges' g进行统计分析。结果显示，通用型LLM生成的内容语言质量更高、情感表达更充分，而医学专用LLM的沟通可及性更强；但医学专用LLM往往存在更高水平的潜在危害、毒性与偏见，导致其安全与可信度表现下降。研究发现揭示了健康沟通领域中专属知识与安全性之间的二元矛盾。结果表明，需要开展有针对性的模型设计优化，尤其要在降低危害与偏见、提升安全性与情感性方面重点改进。本研究对用于癌症沟通的LLM进行了全面评估，为优化AI生成的健康内容提供了关键洞见，也为未来开发准确、安全、可及的数字健康工具提供了参考。
英文摘要：
Effective communication about breast and cervical cancers remains a persistent health challenge, with significant gaps in public understanding of cancer prevention, screening, and treatment, potentially leading to delayed diagnoses and inadequate treatments. This study evaluates the capabilities and limitations of Large Language Models (LLMs) in generating accurate, safe, and accessible cancer-related information to support patient understanding. We evaluated five general-purpose and three medical LLMs using a mixed-methods evaluation framework across linguistic quality, safety and trustworthiness, and communication accessibility and affectiveness. Our approach utilized quantitative metrics, qualitative expert ratings, and statistical analysis using Welch's ANOVA, Games-Howell, and Hedges' g. Our results show that general-purpose LLMs produced outputs of higher linguistic quality and affectiveness, while medical LLMs demonstrate greater communication accessibility. However, medical LLMs tend to exhibit higher levels of potential harm, toxicity, and bias, reducing their performance in safety and trustworthiness. Our findings indicate a duality between domain-specific knowledge and safety in health communications. The results highlight the need for intentional model design with targeted improvements, particularly in mitigating harm and bias, and improving safety and affectiveness. This study provides a comprehensive evaluation of LLMs for cancer communication, offering critical insights for improving AI-generated health content and informing future development of accurate, safe, and accessible digital health tools.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
