指令微调与上下文学习：重新审视少样本计算社会科学中的大型语言模型
Instruction Tuning Vs. In-Context Learning: Revisiting Large Language Models in Few-Shot Computational Social Science
AI总结：
本文对比了少样本计算社会科学任务中指令微调与上下文学习对大型语言模型分类性能的影响，发现ICL在多数任务中优于IT，且单纯增加样本数量不一定提升性能，强调了优化样本质量与提示策略的重要性。
摘要：
大型语言模型（LLMs）在计算社会科学（CSS）任务中的实际应用主要取决于指令微调（IT）或上下文学习（ICL）的有效性。尽管IT在为各种任务微调LLMs方面表现出高度有效性，但ICL通过从示例中学习而无需显式的梯度更新，提供了一种快速的任务适应替代方案。在本文中，我们评估了在少样本CSS任务中使用IT与ICL的LLMs分类性能。实验结果表明，ICL在大多数CSS任务中始终优于IT。此外，我们研究了训练样本数量增加与LLM性能之间的关系。我们的发现表明，在不考虑质量的情况下单纯增加样本数量，并不能持续提升ICL或IT下LLMs的性能，有时甚至会导致性能下降。最后，我们比较了三种提示策略，证明ICL比零样本和思维链更有效。我们的研究突出了ICL在少样本设置下处理CSS任务的显著优势，并强调了优化样本质量和提示策略以提高LLM分类性能的重要性。代码将公开提供。
英文摘要：
Real-world applications of large language models (LLMs) in computational social science (CSS) tasks primarily depend on the effectiveness of instruction tuning (IT) or in-context learning (ICL). While IT has shown highly effective at fine-tuning LLMs for various tasks, ICL offers a rapid alternative for task adaptation by learning from examples without explicit gradient updates. In this paper, we evaluate the classification performance of LLMs using IT versus ICL in few-shot CSS tasks. The experimental results indicate that ICL consistently outperforms IT in most CSS tasks. Additionally, we investigate the relationship between the increasing number of training samples and LLM performance. Our findings show that simply increasing the number of samples without considering their quality does not consistently enhance the performance of LLMs with either ICL or IT and can sometimes even result in a performance decline. Finally, we compare three prompting strategies, demonstrating that ICL is more effective than zero-shot and Chain-of-Thought (CoT). Our research highlights the significant advantages of ICL in handling CSS tasks in few-shot settings and emphasizes the importance of optimizing sample quality and prompting strategies to improve LLM classification performance. The code will be made available.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
