优化Phi-2小型语言模型用于实时聊天机器人应用：基于QLoRA量化的参数高效微调（PEFT）

中文摘要 AI 辅助

本研究探讨了通过参数高效微调（PEFT）和量化低秩适配（QLoRA）优化Phi-2小型语言模型（SLMs）以用于实时聊天机器人应用。QLoRA特指将PEFT与LoRA结合，并伴随4比特量化过程，旨在提高计算效率。这些模型最初设计为以最小计算开销实现高性能，现进一步优化以应对移动和边缘计算环境的限制。通过将PEFT与QLoRA集成，本研究旨在显著减少内存使用，同时保持或可能提高模型在实时交互中响应的准确性。使用ROUGE指标系统评估了这些技术的有效性，结果显示模型在摘要任务中取得了显著改进。该方法不仅证实了在资源受限环境中使用SLMs的可行性，还为在实时场景中部署先进AI驱动应用开辟了新途径。本研究的发现对开发高效、可扩展且易获取的AI技术具有重要意义，为各行业更广泛的采用铺平了道路。

英文摘要

This study explores the optimization of the Phi-2 Small Language Models (SLMs) for real-time chatbot applications through Parameter-Efficient Fine-Tuning (PEFT) and Quantized Low-Rank Adaptation (QLoRA). QLoRA specifically refers to the integration of PEFT with LoRA alongside a 4-bit quantization process, aimed at enhancing computational efficiency. These models, initially designed for high performance with minimal computational overhead, are further refined to address the constraints of mobile and edge computing environments. By integrating PEFT with QLoRA, the research aims to reduce memory usage significantly while maintaining, or potentially improving, the accuracy of model responses in real-time interactions. The effectiveness of these techniques was evaluated using the ROUGE metric system, which showed notable improvements in the summarization tasks performed by the models. This approach not only confirms the feasibility of using SLMs in resource-restricted environments but also opens up new avenues for deploying advanced AI-driven applications in real-time settings. The study's findings have significant implications for the development of efficient, scalable, and accessible AI technologies, paving the way for broader adoption in various industries.

发表机构

• Beacom College of Computer and Cyber Sciences, Dakota State University（达科他州立大学比科姆计算机与网络科学学院）
• Mathematics and Computer Science, Augustana College（奥古斯塔纳学院数学与计算机科学系）

机构由 AI 辅助整理，请以论文原文为准。