arXivDaily arXiv每日学术速递 周一至周五更新
​通知:联邦节假日调休:哥伦布日 / 原住民日(Columbus Day / Indigenous Peoples' Day),今日(13日)无更新。
arXiv 2605.00468cs.CL

ReLay:为更好的理解而定制的LLM生成的通俗摘要,但代价是什么?

ReLay: Personalized LLM-Generated Plain-Language Summaries for Better Understanding, but at What Cost?

University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);University of Virginia(弗吉尼亚大学);University of Colorado Anschutz(科罗拉多大学安舒茨分校);Johns Hopkins University(约翰霍普金斯大学);University of Chicago Pritzker School of Medicine(芝加哥大学普ritzker法学院);National Library of Medicine, National Institutes of Health(国家医学图书馆,国家卫生研究院);University of Washington(华盛顿大学)

Joey Chan, Yikun Han, Jingyuan Chen, Samuel Fang, Lauren D. Gryboski, Alexandra Lee, Sheel Tanna, Qingqing Zhu, Zhiyong Lu, Lucy Lu Wang, Yue Guo

AI总结:

ReLay研究通过评估五种LLM在两种个性化方法中的表现,探讨了个性化通俗摘要对理解提升与安全性的平衡问题。

摘要:

摘要:Plain Language Summaries (PLS)旨在使研究内容对非专业读者更易于理解,但通常采用一种通用的写作风格,忽略了读者在信息需求和理解能力上的差异。在健康领域,这种局限性尤为重要,因为对科学信息的误解可能会影响现实中的决策。大型语言模型(LLMs)为个性化PLS提供了新的机会,但尚不清楚个性化是否有效,哪些策略最有效,以及如何在个性化与安全性之间取得平衡。我们引入了ReLay,这是一个包含300个参与者-PLS配对数据的数据库,来自50名非专业参与者,在静态(专家撰写)和互动(LLM个性化)两种设置中。ReLay包括用户特征、健康信息需求、信息寻求行为、理解结果、交互日志和质量评分。我们利用ReLay在两种个性化方法上评估了五个LLMs。个性化提高了理解和感知质量,但也增加了强化用户偏见和引入幻觉的风险,揭示了个性化与安全之间的权衡。这些发现突显了需要既有效又可信的个性化方法,以满足多样化非专业受众的需求。

英文摘要:

Plain Language Summaries (PLS) aim to make research accessible to lay readers, but they are typically written in a one-size-fits-all style that ignores differences in readers' information needs and comprehension. In health contexts, this limitation is particularly important because misunderstanding scientific information can affect real-world decisions. Large language models (LLMs) offer new opportunities for personalizing PLS, but it remains unclear whether personalization helps, which strategies are most effective, and how to balance personalization with safety. We introduce ReLay, a dataset of 300 participant--PLS pairs from 50 lay participants in both static (expert-written) and interactive (LLM-personalized) settings. ReLay includes user characteristics, health information needs, information-seeking behavior, comprehension outcomes, interaction logs, and quality ratings. We use ReLay to evaluate five LLMs across two personalization methods. Personalization improves comprehension and perceived quality, but it also raises the risk of reinforcing user biases and introducing hallucinations, revealing a trade-off between personalization and safety. These findings highlight the need for personalization methods that are both effective and trustworthy for diverse lay audiences.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑