FedCoT：面向大型语言模型的通信高效联邦推理增强
FedCoT: Communication-Efficient Federated Reasoning Enhancement for Large Language Models
AI总结：
本文提出FedCoT框架，通过轻量级思维链增强机制和基于LoRA模块堆叠的改进聚合方法，在保护隐私和降低通信开销的前提下，显著提升了联邦学习环境下大型语言模型的推理准确性与可解释性。
摘要：
在联邦学习环境中高效增强大型语言模型（LLMs）的推理能力仍然具有挑战性，特别是在平衡性能提升与严格的计算、通信和隐私约束时。这一挑战在医疗保健领域尤为突出，因为涵盖临床、运营和面向患者场景的决策不仅要求准确的输出，还需要可解释、可追溯的推理依据，以确保安全性、问责制和法规合规性。传统的LLM联邦微调方法未能满足这一需求：它们主要针对答案正确性进行优化，却忽视了推理依据的质量，使得CoT能力依赖于模型固有的预训练能力。此外，现有的改进推理依据的方法通常依赖于从集中式模型进行侵犯隐私的知识蒸馏。另外，传统的LLM联邦微调中的通信开销依然很大。我们通过提出FedCoT来填补这一空白，这是一个专门设计用于在联邦环境中增强推理的新颖框架。FedCoT采用轻量级的思维链增强机制：本地模型生成多条推理路径，并由一个紧凑的判别器动态选择最有希望的一条。该方法在提升推理准确性和鲁棒性的同时，提供了宝贵的可解释性，这对于医疗应用尤为关键。为了高效处理客户端异构性，我们采用了一种基于先进LoRA模块堆叠的改进聚合方法，结合客户端分类器感知，以在多样化客户端之间实现无噪声聚合。在医疗推理任务上的全面实验表明，FedCoT在严格的资源预算下显著提升了客户端推理性能，同时完全保护了数据隐私。
英文摘要：
Enhancing LLM reasoning in federated settings is nontrivial due to stringent computational, communication, and privacy constraints, especially in healthcare, where clinically consequential decisions require not only accuracy but also interpretable, auditable rationales to meet safety, accountability, and regulatory requirements. Conventional federated fine-tuning largely imitates final answers rather than cultivating step-by-step reasoning, often relying on privacy-sensitive centralized distillation and still incurring substantial communication overhead. We address this gap with \textbf{\ours{}}, a federated reasoning framework that combines lightweight chain-of-thought resampling with a compact discriminator for selection, and client-aware LoRA stacking with weighted classifier aggregation to accommodate heterogeneity while reducing aggregation noise and communication; clients generate candidate chains and supervision locally, and only lightweight modules are aggregated on the server. Experiments on medical reasoning benchmarks show consistent gains under tight resource budgets while keeping data local and respecting privacy, offering an interpretable and resource-efficient solution. Our code is made publicly available at https://github.com/DIaacKr/FedCoT
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
