DictLLM:利用大型语言模型处理键值数据结构以增强医疗诊断
DictLLM: Harnessing Key-Value Data Structures with Large Language Models for Enhanced Medical Diagnostics
AI总结:
提出DictLLM框架,通过分组位置编码、分层注意力偏置和最优传输对齐层处理键值结构化数据,在医疗诊断生成任务中显著优于基线及少样本GPT-4。
摘要:
结构化数据为信息组织提供了一种复杂的机制。在大型语言模型背景下,现有的结构化数据文本序列化方法未能充分解决键值结构化数据固有的异构性问题。这些方法并不理想,通常会导致输入规模变大且对输入变化的适应性差。在本文中,我们提出了DictLLM,这是一个旨在改进键值结构化数据(如医疗化验报告)建模以生成医疗诊断的创新框架。DictLLM集成了三个关键组件:(1)分组位置编码以保持排列不变性,(2)分层注意力偏置以捕捉结构化数据中的固有偏置,(3)最优传输对齐层,将字典编码器生成的嵌入与LLM对齐,从而产生一系列固定长度的虚拟标记。我们在一个综合的真实世界医疗化验报告数据集上,使用多种LLM模型进行了自动诊断生成实验。研究结果表明,在Rouge-L和Knowledge F1分数方面,DictLLM显著优于既有的基线方法和少样本GPT-4实现。此外,我们通过一系列实验评估了该框架的可扩展性和鲁棒性,突显了其在准确建模医疗字典数据复杂键值数据结构方面的卓越能力。
英文摘要:
Structured data offers a sophisticated mechanism for the organization of information. Existing methodologies for the text-serialization of structured data in the context of large language models fail to adequately address the heterogeneity inherent in key-value structured data. These methods are not ideal and frequently result in larger input sizes and poor adaptability to input changes. In this paper, we introduce DictLLM, an innovative framework designed to improve the modeling of key-value structured data, like medical laboratory reports, for generating medical diagnoses. DictLLM integrates three key components: (1) group positional encoding to maintain permutation invariance, (2) hierarchical attention bias to capture the inherent bias in structured data, and (3) an optimal transport alignment layer that aligns the embedding generated by the dictionary encoder with the LLM, thereby producing a sequence of fixed-length virtual tokens. We carry out experiments using various LLM models on a comprehensive real-world medical laboratory report dataset for automatic diagnosis generation, our findings illustrate that DictLLM significantly outperforms established baseline methods and few-shot GPT-4 implementations in terms of both Rouge-L and Knowledge F1 scores. Furthermore, our evaluation of the framework's scalability and robustness, through a series of experiments, underscores its exceptional capability in accurately modeling the complex key-value data structure of medical dictionary data.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。