重新审视大型推理模型的提示优化——以事件抽取为案例研究
Revisiting Prompt Optimization with Large Reasoning Models-A Case Study on Event Extraction
AI总结：
本研究以事件抽取为案例，系统验证了大型推理模型（如DeepSeek-R1和o1）仍需提示优化，且作为优化器效果更佳，并推广至其他任务，同时分析了其常见错误与稳定性。
摘要：
大型推理模型（LRMs），如DeepSeek-R1和OpenAI o1，在各种推理任务中展现出了卓越的能力。它们生成中间思考并对其进行推理的强大能力也引发了这样的论点：它们可能不再需要大量的提示工程或优化来理解人类指令并产生准确的输出。在这项工作中，我们旨在系统性地研究这个开放性问题，以结构化的事件抽取任务作为案例研究。我们实验了两种LRMs（DeepSeek-R1和o1）和两种通用大型语言模型（LLMs）（GPT-4o和GPT-4.5），分别将它们用作任务模型或提示优化器。我们的结果表明，在像事件抽取这样复杂的任务上，LRMs作为任务模型仍然受益于提示优化，并且使用LRMs作为提示优化器能产生更有效的提示。我们的发现也推广到了事件抽取之外的任务。最后，我们提供了对LRMs常见错误的错误分析，并强调了LRMs在细化任务指令和事件指南方面的稳定性和一致性。
英文摘要：
Large Reasoning Models (LRMs) such as DeepSeek-R1 and OpenAI o1 have demonstrated remarkable capabilities in various reasoning tasks. Their strong capability to generate and reason over intermediate thoughts has also led to arguments that they may no longer require extensive prompt engineering or optimization to interpret human instructions and produce accurate outputs. In this work, we aim to systematically study this open question, using the structured task of event extraction for a case study. We experimented with two LRMs (DeepSeek-R1 and o1) and two general-purpose Large Language Models (LLMs) (GPT-4o and GPT-4.5), when they were used as task models or prompt optimizers. Our results show that on tasks as complicated as event extraction, LRMs as task models still benefit from prompt optimization, and that using LRMs as prompt optimizers yields more effective prompts. Our finding also generalizes to tasks beyond event extraction. Finally, we provide an error analysis of common errors made by LRMs and highlight the stability and consistency of LRMs in refining task instructions and event guidelines.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
