用于总结健康故事中所描述的实际医疗体验的主题感知大语言模型
Topic-aware Large Language Models for Summarizing the Lived Healthcare Experiences Described in Health Stories
AI总结：
本研究结合LDA主题识别与开源LLM分层总结方法，对50篇非裔美国人医疗体验故事开展主题感知总结，经GPT-4与专家验证总结质量优异，可为医疗研究与临床改进提供洞见。
摘要：
叙事是一种强有力的沟通形式，或可为揭示导致医疗结果差异的因素提供洞见。为确定大语言模型（Large Language Models, LLMs）能否识别潜在的深层因素与干预方向，我们对非裔美国人（African American, AA）讲述者的叙事文本开展了主题感知分层总结研究。\n研究使用50篇转录的非裔美国人体验故事，采用隐含狄利克雷分配（Latent Dirichlet Allocation, LDA）技术识别其体验中的主题。针对同一主题的故事，我们采用基于开源LLM的分层总结方法进行总结：先对每篇涉及该主题的故事生成单篇总结，再跨这些单篇总结生成对应主题的总结。\n我们通过GPT-4模型对生成的主题总结在虚构性、准确性、全面性和有用性四个维度进行评分，并由两位领域专家将模型评分结果与原始故事总结进行对比验证可靠性。\n实验结果显示，50篇非裔美国人故事中共识别出26个主题。GPT-4的评分表明，生成的主题总结无虚构内容，准确性高、全面性好且实用性强。与专家评估相比，GPT评分的可靠性呈现中到高度的一致性。\n我们的方法识别出了与非裔美国人体验相关的主题，包括健康行为、与医疗团队成员的互动、照护与症状管理等。这些洞见可帮助研究者通过高效利用非结构化叙事文本、发挥叙事的沟通力量，识别潜在影响因素与干预措施。将LDA与LLMs结合用于识别和总结非裔美国人群体的体验，为健康研究提供了多种可行方向，也为支持患者与照护者、最终改善医疗结果的临床改进提供了可能路径。
英文摘要：
Storytelling is a powerful form of communication and may provide insights into factors contributing to gaps in healthcare outcomes. To determine whether Large Language Models (LLMs) can identify potential underlying factors and avenues for intervention, we performed topic-aware hierarchical summarization of narratives from African American (AA) storytellers. Fifty transcribed stories of AA experiences were used to identify topics in their experience using the Latent Dirichlet Allocation (LDA) technique. Stories about a given topic were summarized using an open-source LLM-based hierarchical summarization approach. Topic summaries were generated by summarizing across story summaries for each story that addressed a given topic. Generated topic summaries were rated for fabrication, accuracy, comprehensiveness, and usefulness by the GPT4 model, and the model's reliability was validated against the original story summaries by two domain experts. 26 topics were identified in the fifty AA stories. The GPT4 ratings suggest that topic summaries were free from fabrication, highly accurate, comprehensive, and useful. The reliability of GPT ratings compared to expert assessments showed moderate to high agreement. Our approach identified AA experience-relevant topics such as health behaviors, interactions with medical team members, caregiving and symptom management, among others. Such insights could help researchers identify potential factors and interventions by learning from unstructured narratives in an efficient manner-leveraging the communicative power of storytelling. The use of LDA and LLMs to identify and summarize the experience of AA individuals suggests a variety of possible avenues for health research and possible clinical improvements to support patients and caregivers, thereby ultimately improving health outcomes.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
