arXiv 2407.009782024-12-09cs.AIcs.LG

面向医疗物联网安全数据管理的混合RAG赋能多模态大语言模型：一种基于扩散的合约方法

Hybrid RAG-empowered Multi-modal LLM for Secure Data Management in Internet of Medical Things: A Diffusion-based Contract Approach

Guangdong University of Technology（广东工业大学）；Nanjing University of Aeronautics and Astronautics（南京航空航天大学）；Guangzhou University of Traditional Chinese Medicine（广州中医药大学）；The Second Affiliated Hospital of Guangzhou University of Chinese Medicine（广州中医药大学第二附属医院）

Cheng Su, Jinbo Wen, Jiawen Kang, Yonghua Wang, Yuanjia Su, Hudan Pan, Zishao Zhong, M. Shamim Hossain

备注：: 13 pages, 7 figures

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对医疗物联网中医疗MLLM面临的数据安全与时效性问题，提出混合RAG赋能的医疗MLLM框架，结合分层跨链架构、合约理论与扩散模型强化学习，实现安全高效的医疗数据管理。

摘要：

在快速发展的医疗领域，尤其是随着医疗物联网（IoMT）的融合日益深入，安全的数据管理与高效的数据共享已变得至关重要。生成式人工智能的兴起进一步让多模态大语言模型（MLLMs）成为IoMT中管理和优化医疗数据的核心工具。MLLM可通过在海量多模态数据上进行大规模训练，支持多模态输入并生成多种类型的内容。然而，医疗MLLM的开发仍存在关键挑战，包括医疗数据的安全性与时效性问题，会影响MLLM的输出质量。针对这一问题，本文提出一种混合检索增强生成（RAG）赋能的医疗MLLM框架，用于医疗数据管理。该框架采用分层跨链架构以实现安全的数据训练；此外，它通过混合RAG提升MLLM的输出质量——混合RAG使用多模态指标筛选各类单模态RAG结果，并将这些检索结果作为额外输入接入MLLM。同时，我们采用信息年龄间接评估MLLM的数据时效性影响，并利用合约理论激励医疗数据持有者分享其最新数据，缓解数据共享过程中的信息不对称问题。最后，我们使用基于生成式扩散模型的深度强化学习算法，识别最优合约以实现高效数据共享。数值结果验证了所提方案的有效性，可实现安全高效的医疗数据管理。

英文摘要：

Secure data management and effective data sharing have become paramount in the rapidly evolving healthcare landscape, especially with the growing integration of the Internet of Medical Things (IoMT). The rise of generative artificial intelligence has further elevated Multi-modal Large Language Models (MLLMs) as essential tools for managing and optimizing healthcare data in IoMT. MLLMs can support multi-modal inputs and generate diverse types of content by leveraging large-scale training on vast amounts of multi-modal data. However, critical challenges persist in developing medical MLLMs, including security and freshness issues of healthcare data, affecting the output quality of MLLMs. To this end, in this paper, we propose a hybrid Retrieval-Augmented Generation (RAG)-empowered medical MLLM framework for healthcare data management. This framework leverages a hierarchical cross-chain architecture to facilitate secure data training. Moreover, it enhances the output quality of MLLMs through hybrid RAG, which employs multi-modal metrics to filter various unimodal RAG results and incorporates these retrieval results as additional inputs to MLLMs. Additionally, we employ age of information to indirectly evaluate the data freshness impact of MLLMs and utilize contract theory to incentivize healthcare data holders to share their fresh data, mitigating information asymmetry during data sharing. Finally, we utilize a generative diffusion model-based deep reinforcement learning algorithm to identify the optimal contract for efficient data sharing. Numerical results demonstrate the effectiveness of the proposed schemes, which achieve secure and efficient healthcare data management.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。