联邦信任链：区块链增强的大语言模型训练与遗忘
Federated TrustChain: Blockchain-Enhanced LLM Training and Unlearning
AI总结：
本文提出一种基于区块链的联邦学习框架，通过防篡改记录和可验证的遗忘机制，增强大语言模型协作训练的透明度、问责制与遗忘能力。
摘要：
大语言模型（LLMs）的发展面临一个重大挑战：公开可用新鲜数据的枯竭。这是因为训练大语言模型需要大量新数据。联邦学习作为一种有前景的解决方案应运而生，使协作模型能够将其私有数据贡献给大语言模型全局模型。然而，将联邦学习与大语言模型集成带来了新的挑战，包括缺乏透明度以及对有效遗忘机制的需求。透明度对于确保参与者之间的信任和公平至关重要，而问责制对于威慑恶意行为以及在必要时采取纠正措施至关重要。为应对这些挑战，我们提出了一种新颖的基于区块链的大语言模型联邦学习框架，该框架增强了透明度、问责制和遗忘能力。我们的框架利用区块链技术为每个模型的贡献创建防篡改记录，并引入了一种与联邦学习机制无缝集成的创新遗忘功能。我们研究了低秩适配（LoRA）超参数对遗忘性能的影响，并集成 Hyperledger Fabric 以确保遗忘过程的安全性、透明度和可验证性。通过全面的实验和分析，我们展示了所提出框架在使用联邦学习训练的大语言模型中实现高效遗忘方面的有效性。我们的研究结果凸显了将区块链技术集成到大语言模型联邦学习框架中的可行性。
英文摘要：
The development of Large Language Models (LLMs) faces a significant challenge: the exhausting of publicly available fresh data. This is because training a LLM needs a large demanding of new data. Federated learning emerges as a promising solution, enabling collaborative model to contribute their private data to LLM global model. However, integrating federated learning with LLMs introduces new challenges, including the lack of transparency and the need for effective unlearning mechanisms. Transparency is essential to ensuring trust and fairness among participants, while accountability is crucial for deterring malicious behaviour and enabling corrective actions when necessary. To address these challenges, we propose a novel blockchain-based federated learning framework for LLMs that enhances transparency, accountability, and unlearning capabilities. Our framework leverages blockchain technology to create a tamper-proof record of each model's contributions and introduces an innovative unlearning function that seamlessly integrates with the federated learning mechanism. We investigate the impact of Low-Rank Adaptation (LoRA) hyperparameters on unlearning performance and integrate Hyperledger Fabric to ensure the security, transparency, and verifiability of the unlearning process. Through comprehensive experiments and analysis, we showcase the effectiveness of our proposed framework in achieving highly effective unlearning in LLMs trained using federated learning. Our findings highlight the feasibility of integrating blockchain technology into federated learning frameworks for LLMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
