Logistic Regression 让小型 LLM 成为强大且可解释的“数十样本”分类器
Logistic Regression makes small LLMs strong and explainable "tens-of-shot" classifiers
AI总结：
本研究针对简单分类任务，提出基于小型LLM嵌入向量的带惩罚Logistic Regression方法，在“数十样本”场景下性能媲美甚至优于大型LLM，兼具隐私、成本等优势且可提供稳定解释。
摘要：
针对简单分类任务，我们证明用户可受益于使用小型本地生成式语言模型而非大型商用模型的优势，且无需在性能上做出妥协，也不会产生额外的标注成本。这些优势涵盖隐私、可用性、成本和可解释性等方面，在商业应用及更广泛的AI民主化进程中均十分重要。通过在17项句子分类任务（2-4个类别）上开展实验，我们发现，在“数十样本”场景下，基于小型LLM嵌入向量的带惩罚Logistic Regression性能与大型LLM相当（且通常更优），其所需标注样本数不超过验证大型LLM性能所需的数量。最后，我们还提取出了分类决策的稳定且合理的解释。
英文摘要：
For simple classification tasks, we show that users can benefit from the advantages of using small, local, generative language models instead of large commercial models without a trade-off in performance or introducing extra labelling costs. These advantages, including those around privacy, availability, cost, and explainability, are important both in commercial applications and in the broader democratisation of AI. Through experiments on 17 sentence classification tasks (2-4 classes), we show that penalised logistic regression on the embeddings from a small LLM equals (and usually betters) the performance of a large LLM in the "tens-of-shot" regime. This requires no more labelled instances than are needed to validate the performance of the large LLM. Finally, we extract stable and sensible explanations for classification decisions.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
