大型语言模型也是优秀的原型常识推理器
Large Language Models Are Also Good Prototypical Commonsense Reasoners
AI总结：
本文通过半自动设计任务相关、支持性证据生成和多样化路径解码等提示策略，在无需微调的情况下使大型语言模型在ProtoQA等常识推理基准上取得新的最优结果，并提升了模型可解释性。
摘要：
常识推理是大型语言模型的一项关键技能，但在需要该能力的特定任务中仍面临持续挑战。传统的微调方法可能资源密集，并可能损害模型的泛化能力。此外，GPT-3.5和Claude等最先进的语言模型主要通过API调用访问，这使得微调模型变得困难。为应对这些挑战，我们从大型模型在定制任务上的输出中获得启发，并从多个视角半自动地开发了一组新颖提示，包括任务相关性、支持性证据生成（例如思维链和知识）、多样化路径解码以辅助模型。在ProtoQA数据集上的实验结果表明，通过设计更好的提示，我们可以在ProtoQA排行榜上取得新的最优结果（SOTA），与之前的SOTA模型相比，Max Answer@1分数提高了8%，Max Incorrect@1分数提高了4%（首次突破50%），并在StrategyQA和CommonsenseQA2.0上取得了提升（分别为3%和1%）。此外，借助生成的思维链和知识，我们提高了模型的可解释性，同时也超越了之前的SOTA模型。我们希望我们的工作能为NLP社区提供见解，以开发更好的提示并探索大型语言模型在更复杂推理任务中的潜力。
英文摘要：
Commonsense reasoning is a pivotal skill for large language models, yet it presents persistent challenges in specific tasks requiring this competence. Traditional fine-tuning approaches can be resource-intensive and potentially compromise a model's generalization capacity. Furthermore, state-of-the-art language models like GPT-3.5 and Claude are primarily accessible through API calls, which makes fine-tuning models challenging. To address these challenges, we draw inspiration from the outputs of large models for tailored tasks and semi-automatically developed a set of novel prompts from several perspectives, including task-relevance, supportive evidence generation (e.g. chain-of-thought and knowledge), diverse path decoding to aid the model. Experimental results on ProtoQA dataset demonstrate that with better designed prompts we can achieve the new state-of-art(SOTA) on the ProtoQA leaderboard, improving the Max Answer@1 score by 8%, Max Incorrect@1 score by 4% (breakthrough 50% for the first time) compared to the previous SOTA model and achieved an improvement on StrategyQA and CommonsenseQA2.0 (3% and 1%, respectively). Furthermore, with the generated Chain-of-Thought and knowledge, we can improve the interpretability of the model while also surpassing the previous SOTA models. We hope that our work can provide insight for the NLP community to develop better prompts and explore the potential of large language models for more complex reasoning tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
