arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2604.26319cs.CL
收藏

基于大语言模型的立场检测中提示与多智能体方法的系统比较

A Systematic Comparison of Prompting and Multi-Agent Methods for LLM-based Stance Detection

School of Artificial Intelligence, Shenzhen Technology University(人工智能学院,深圳技术大学)

Genan Dai, Zini Chen, Yi Yang, Bowen Zhang

AI总结:

本文系统比较了提示和多智能体方法在大语言模型立场检测中的表现,评估了五种方法在四个数据集14个子任务上的性能,发现提示方法在多数模型上表现更优,且模型规模对性能影响大于方法选择。

摘要:

本文系统比较了提示和多智能体方法在大语言模型立场检测中的表现,评估了五种方法在四个数据集14个子任务上的性能,发现提示方法在多数模型上表现更优,且模型规模对性能影响大于方法选择。

英文摘要:

Stance detection identifies the attitude of a text author toward a given target. Recent studies have explored various LLM-based strategies for this task, from zero-shot prompting to multi-agent debate. However, existing works differ in data splits, base models, and evaluation protocols, making fair comparison difficult. We conduct a systematic comparison that evaluates five methods across two categories -- prompt-based inference (Direct Prompting, Auto-CoT, StSQA) and agent-based debate (COLA, MPRF) -- on four datasets with 14 subtasks, using 15 LLMs from six model families with parameter sizes from 7B to 72B+. Our experiments yield several findings. First, on all models with complete results, the best prompt-based method outperforms the best agent-based method, while agent methods require 7 to 12 times more API calls per sample. Second, model scale has a larger impact on performance than method choice, with gains plateauing around 32B. Third, reasoning-enhanced models (DeepSeek-R1) do not consistently outperform general models of the same size on this task.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑