零样本思维链提示在日语提示中的有效性
Effectiveness of Zero-shot-CoT in Japanese Prompts
AI总结：
本文比较了零样本思维链提示在日语和英语中的效果，发现其在GPT-3.5部分类别中有效，但在GPT-4o-mini中总体导致性能下降，仅大学数学等少数日语类别仍有提升。
摘要：
我们使用ChatGPT-3.5和4o-mini比较了零样本思维链（CoT）提示在日语和英语中的有效性。零样本CoT技术通过在提示后附加诸如“让我们一步一步思考”之类的短语来鼓励在回答前进行推理，已被证明能够在数学和推理任务中提升大语言模型（LLM）的性能，尤其是在英语中。我们利用日语多任务语言理解基准（JMMLU）和多任务语言理解基准（MMLU）研究了这些效果如何迁移到日语。我们的结果表明，虽然零样本CoT提示在GPT-3.5中可以为某些提示类别带来显著的性能提升，但它在GPT-4o-mini中的影响与显著的性能下降相关。然而，对于日语提示，仍然存在某些类别，如大学数学和抽象代数，尽管在更先进的模型中有效性呈下降的总体趋势，这些类别仍表现出改进。
英文摘要：
We compare the effectiveness of zero-shot Chain-of-Thought (CoT) prompting in Japanese and English using ChatGPT-3.5 and 4o-mini. The technique of zero-shot CoT, which involves appending a phrase such as "Let's think step by step" to a prompt to encourage reasoning before answering, has been shown to offer LLM performance improvements in mathematical and reasoning tasks, particularly in English. We investigate how these effects transfer to Japanese using the Japanese Multi-task Language Understanding Benchmark (JMMLU) and the Multi-task Language Understanding Benchmark (MMLU). Our results show that while zero-shot CoT prompting can lead to notable performance gains for some prompt categories in GPT-3.5, its impact in GPT-4o-mini is associated with significant performance declines. However, for Japanese prompts there remain certain categories, such as college mathematics and abstract algebra, that still exhibit improvements, despite the broader trend of diminishing effectiveness in more advanced models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
