赋予 GPT-4 许可与机会:提升少样本事件检测的准确性与置信度估计
Granting GPT-4 License and Opportunity: Enhancing Accuracy and Confidence Estimation for Few-Shot Event Detection
AI总结:
本文提出 L&O 提示扩展方法,赋予 GPT-4 推测许可与不确定性解释机会,从而在 BETTER 少样本事件检测中同时提升准确率和置信度估计,达到 0.759 AUC。
摘要:
诸如 GPT-4 之类的大语言模型(LLM)在少样本学习情境中已展现出足够前景,表明其可用于生成“银色”数据,并通过迭代应用与审查来完善新本体。此类工作流若具备可靠的置信度估计,将会更加有效。遗憾的是,置信度估计是 GPT-4 等模型有据可查的弱点,而用于弥补这一缺陷的成熟方法需要大量额外复杂性与计算。本研究以 BETTER 本体中的事件检测为载体,探索在少样本学习下利用 GPT-4 进行有效置信度估计的方法。关键创新在于扩展呈现给 GPT-4 的提示与任务,使其在不确定时获得进行推测的“许可”(License),以及量化并解释其不确定性的“机会”(Opportunity),即 L&O。该方法提高了准确性,并在无需额外机制的情况下提供了可用的置信度度量(0.759 AUC)。
英文摘要:
Large Language Models (LLMs) such as GPT-4 have shown enough promise in the few-shot learning context to suggest use in the generation of "silver" data and refinement of new ontologies through iterative application and review. Such workflows become more effective with reliable confidence estimation. Unfortunately, confidence estimation is a documented weakness of models such as GPT-4, and established methods to compensate require significant additional complexity and computation. The present effort explores methods for effective confidence estimation with GPT-4 with few-shot learning for event detection in the BETTER ontology as a vehicle. The key innovation is expanding the prompt and task presented to GPT-4 to provide License to speculate when unsure and Opportunity to quantify and explain its uncertainty (L&O). This approach improves accuracy and provides usable confidence measures (0.759 AUC) with no additional machinery.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。