Personalized Pieces：通过协作努力实现高效的个性化大语言模型
Personalized Pieces: Efficient Personalized Large Language Models through Collaborative Efforts
AI总结：
本文提出 Per-Pcs 框架，将个性化 PEFT 拆分为可共享片段并通过门控组装，在六项任务上以更低资源达到接近 OPPU 的效果。
摘要：
个性化大语言模型（LLM）旨在使交互、内容和推荐适配个体用户偏好。尽管参数高效微调（PEFT）方法在性能和泛化方面表现出色，但它们成本高昂，且单独使用时限制了共享收益。为此，我们提出 Personalized Pieces（Per-Pcs），这是一个允许用户通过协作努力安全共享并高效组装个性化 PEFT 的框架。Per-Pcs 包括选择共享者、将其 PEFT 拆分为多个片段，以及为每个片段训练门控。这些片段被加入一个资源池，目标用户可利用其历史数据从中选择并组装个性化 PEFT。该方法保护隐私，并能实现细粒度用户建模，而无需过多的存储和计算需求。实验结果表明，Per-Pcs 优于非个性化和 PEFT 检索基线，在六项任务上提供与 OPPU 相当的性能，同时资源使用显著更低。进一步分析突出了 Per-Pcs 在共享者数量和选择策略、片段共享比例方面的鲁棒性，以及在计算时间和存储空间上的可扩展性。Per-Pcs 的模块化促进了安全共享，通过协作努力使 LLM 个性化更高效、有效且可广泛获取。
英文摘要：
Personalized large language models (LLMs) aim to tailor interactions, content, and recommendations to individual user preferences. While parameter-efficient fine-tuning (PEFT) methods excel in performance and generalization, they are costly and limit communal benefits when used individually. To this end, we introduce Personalized Pieces (Per-Pcs), a framework that allows users to safely share and assemble personalized PEFT efficiently with collaborative efforts. Per-Pcs involves selecting sharers, breaking their PEFT into pieces, and training gates for each piece. These pieces are added to a pool, from which target users can select and assemble personalized PEFT using their history data. This approach preserves privacy and enables fine-grained user modeling without excessive storage and computation demands. Experimental results show Per-Pcs outperforms non-personalized and PEFT retrieval baselines, offering performance comparable to OPPU with significantly lower resource use across six tasks. Further analysis highlights Per-Pcs's robustness concerning sharer count and selection strategy, pieces sharing ratio, and scalability in computation time and storage space. Per-Pcs's modularity promotes safe sharing, making LLM personalization more efficient, effective, and widely accessible through collaborative efforts.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
