通过多轮提示验证提升量化模型在定性分析中的性能
Improving Quantized Model Performance in Qualitative Analysis with Multi-Pass Prompt Verification
AI总结：
本文研究了不同位数量化级别和类型对LLaMA-3.1(8B)在定性分析中的性能影响，提出了一种量化感知的多轮提示验证方法以提高模型的稳定性和准确性，结果显示8位模型最接近黄金标准，4位模型在应用方法后变得稳定，3位和2位模型在提示设计和验证后性能有所提升。
摘要：
量化大型语言模型(LLMs)因其运行速度快且计算资源需求低而更常用于定性分析。本研究探讨了不同低位量化级别(8位、4位、3位和2位)和量化类型对LLaMA-3.1(8B)在定性分析中的性能影响。研究使用了82份访谈记录中的专家和非专家回应。低比特模型常产生较高的幻觉和不稳定结果，尤其是在处理非专家语言中的不明确术语时。为提高性能，我们提出了一种量化感知的多轮提示验证方法。该方法通过受控步骤引导模型减少幻觉，移除不可靠内容，并在验证后将结果传递给下一访谈文本，从而提高准确性。为了验证性能，人类编码器使用NVivo和BF16 LLaMA分析了访谈记录。BF16 LLaMA-3.1产生了高精度输出，但存在语义漂移和幻觉。这些错误被手动纠正。纠正后的BF16输出和NVivo人工编码被结合，以创建主题提取和频率分析的黄金标准地面真实值(GSGT)。结果表明，8位模型最接近GSGT。4位模型在应用所提方法后变得稳定。3位和2位模型因压缩严重而性能下降，但通过所提提示设计和验证有所提升。本研究还发现，相同位数的模型在不同量化类型下行为不同。总体而言，该方法帮助低资源LLM变得更加稳定、准确，并以更低的成本适用于定性研究。
英文摘要：
Quantized Large Language Models (LLMs) are used more often in qualitative analysis because they run fast and need fewer computing resources. This study examines how different lower bits quantization levels (8-bit, 4-bit, 3-bit, and 2-bit) and quantization types affect the performance of LLaMA-3.1 (8B) on qualitative analysis. The study uses expert and non-expert responses from 82 interview transcripts. Low-bit models often produce higher levels of hallucinations and unstable results, especially when reading non-expert language with unclear terms. To improve performance, we propose a quantization-aware multi-pass prompt verification method. This method guides the model through controlled steps that reduce hallucinations. It removes unreliable content and passes the results to the next transcript after verification, improving accuracy. To validate performance, human coders analyzed transcripts using NVivo and BF16 LLaMA. BF16 LLaMA-3.1 produced high-precision output but had semantic drift and hallucination. These errors were corrected manually. The corrected BF16 output and NVivo human coding were combined to create a gold-standard ground truth (GSGT) for thematic extraction and frequency analysis. The results show that 8-bit models stay closest to the GSGT. The 4-bit models lose accuracy but become stable when the proposed method is applied. The 3-bit and 2-bit models drop in performance because of heavy compression, but they improve with the proposed prompt design and verification. The study also finds that models at the same bit level behave differently depending on quantization type. Overall, the method helps low-resource LLMs become more stable, accurate, and suitable for qualitative research at lower cost.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
