LLaMa-sciQ：一个用于解答科学多项选择题的教育聊天机器人
LLaMa-SciQ: An Educational Chatbot for Answering Science MCQ
AI总结：
本文提出教育聊天机器人LLaMa-SciQ，通过微调LLaMa-8B并结合量化压缩技术，帮助大学生解答STEM领域的多项选择题，在GSM8k数据集上准确率达74.5%，且量化模型仅损失5%性能。
摘要：
大型语言模型（LLMs）通常在需要数学推理的任务上表现不佳，尤其是多项选择题（MCQs）。为了解决这一问题，我们开发了LLaMa-SciQ，这是一个旨在帮助大学生解答和理解STEM领域MCQs的教育聊天机器人。我们首先对模型进行微调，使其与人类偏好对齐。在比较了Mistral-7B和LLaMa-8B的性能后，由于后者评估准确率更高，我们选择其作为基础模型。为进一步提高准确率，我们实现了检索增强生成（RAG），并应用量化技术压缩模型，从而减少推理时间并提高学生的可及性。在数学推理方面，LLaMa-SciQ在GSM8k数据集上达到了74.5%的准确率，在MATH数据集上达到了30%。然而，RAG并未改善性能，甚至降低了性能，这可能是由于检索器问题或模型对上下文不熟悉。尽管如此，量化后的模型仅表现出5%的性能损失，证明了显著的效率提升。
英文摘要：
Large Language Models (LLMs) often struggle with tasks requiring mathematical reasoning, particularly multiple-choice questions (MCQs). To address this issue, we developed LLaMa-SciQ, an educational chatbot designed to assist college students in solving and understanding MCQs in STEM fields. We begin by fine-tuning and aligning the models to human preferences. After comparing the performance of Mistral-7B and LLaMa-8B, we selected the latter as the base model due to its higher evaluation accuracy. To further enhance accuracy, we implement Retrieval-Augmented Generation (RAG) and apply quantization to compress the model, reducing inference time and increasing accessibility for students. For mathematical reasoning, LLaMa-SciQ achieved 74.5% accuracy on the GSM8k dataset and 30% on the MATH dataset. However, RAG does not improve performance and even reduces it, likely due to retriever issues or the model's unfamiliarity with context. Despite this, the quantized model shows only a 5% loss in performance, demonstrating significant efficiency improvements.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
