arXiv 2308.167632023-09-07cs.CLcs.AI

Ladder-of-Thought：以知识为阶梯提升立场检测

Ladder-of-Thought: Using Knowledge as Steps to Elevate Stance Detection

Centre for Frontier AI Research (CFAR), A*STAR（新加坡科技研究局前沿人工智能研究中心（CFAR））；Institute of High Performance Computing (IHPC), A*STAR（新加坡科技研究局高性能计算研究所（IHPC））；DSO National Laboratories（DSO国家实验室）

Kairui Hu, Ming Yan, Joey Tianyi Zhou, Ivor W. Tsang, Wen Haw Chong, Yong Keong Yap

备注：: 5 pages, 2 figures, 2 tables

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出Ladder-of-Thought（LoT）框架，通过双阶段渐进优化引导小型语言模型吸收外部知识以优化中间理由，在立场检测任务中较GPT-3.5提升16%、较带CoT的GPT-3.5提升10%，兼顾效率与性能。

摘要：

立场检测旨在识别文档中针对给定目标所表达的态度。诸如思维链（Chain-of-Thought, CoT）提示等技术推进了这一任务，通过推导中间理由来增强模型的推理能力。然而，CoT在推理过程中主要依赖模型预训练的内部知识，从而忽略了模型此前未知的宝贵外部信息。这种遗漏，尤其是在无监督推理过程中，可能会影响模型的整体性能。此外，尽管CoT增强了大型语言模型（LLMs），较小的语言模型（LMs）虽然在操作上高效，但在提供细致推理方面面临挑战。针对这些已识别的差距，我们引入了用于立场检测任务的Ladder-of-Thought（LoT）。LoT通过双阶段渐进式优化框架构建，引导小型LMs吸收高质量外部知识，并优化所生成的中间理由。这些增强的理由随后作为更精确预测的基础——类似于梯子帮助达到更高目标。LoT在效率和性能之间实现了平衡。我们的实证评估突显了LoT的有效性，在立场检测任务上较GPT-3.5提升了16%，较带CoT的GPT-3.5提升了10%。

英文摘要：

Stance detection aims to identify the attitude expressed in a document towards a given target. Techniques such as Chain-of-Thought (CoT) prompting have advanced this task, enhancing a model's reasoning capabilities through the derivation of intermediate rationales. However, CoT relies primarily on a model's pre-trained internal knowledge during reasoning, thereby neglecting the valuable external information that is previously unknown to the model. This omission, especially within the unsupervised reasoning process, can affect the model's overall performance. Moreover, while CoT enhances Large Language Models (LLMs), smaller LMs, though efficient operationally, face challenges in delivering nuanced reasoning. In response to these identified gaps, we introduce the Ladder-of-Thought (LoT) for the stance detection task. Constructed through a dual-phase Progressive Optimization Framework, LoT directs the small LMs to assimilate high-quality external knowledge, refining the intermediate rationales produced. These bolstered rationales subsequently serve as the foundation for more precise predictions - akin to how a ladder facilitates reaching elevated goals. LoT achieves a balance between efficiency and performance. Our empirical evaluations underscore LoT's efficacy, marking a 16% improvement over GPT-3.5 and a 10% enhancement compared to GPT-3.5 with CoT on stance detection task.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。