大语言模型作为细粒度意见分析的自动标注者和标注裁决者
Large Language Models as Automatic Annotators and Annotation Adjudicators for Fine-Grained Opinion Analysis
AI总结：
本文探索使用大语言模型作为自动标注者进行细粒度意见分析，提出声明式标注流水线和LLM裁决方法，实验表明LLM在跨度级别可靠但难以再现关系结构，更适合作为标注助手而非完全替代人类。
摘要：
文本的细粒度意见分析提供了对表达情感的详细理解，包括所涉及的实体。尽管这种详细程度很有价值，但在数据集中标注意见以训练模型需要大量人力投入和成本，尤其是在不同领域和实际应用中。为了解决领域特定标注数据集的短缺，我们探索了LLM作为自动标注者进行细粒度意见分析的可行性。我们使用声明式标注流水线，这种方法减少了在使用LLM识别文本中细粒度意见跨度时手动提示工程的可变性。我们还提出了一种专门的方法，让LLM裁决多个标签并产生最终标注。我们使用不同大小的模型在方面情感三元组提取（ASTE）和方面-类别-意见-情感（ACOS）分析任务上试用了该流水线。在这项工作中，我们试图开发完全自主的基于LLM的标注者，但我们的结果揭示了一个不均衡的画面，其特点是关键的性能分叉：LLM在跨度级别可靠，但难以忠实地再现连接这些跨度的关系结构。这表明LLM更适合作为高保真标注助手和数据增强工具，以扩展细粒度意见标注数据集，而不是完全取代人类标注者。
英文摘要：
Fine-grained opinion analysis of text provides a detailed understanding of expressed sentiments and their targets. Although this level of detail is valuable, annotating opinions in datasets for model training requires considerable human effort and substantial cost, especially across diverse domains and real-world applications. To address this shortage of domain-specific labelled datasets, we explore the feasibility of LLMs as automatic annotators for fine-grained opinion analysis. We use a declarative annotation pipeline, an approach that reduces the variability of manual prompt engineering when using LLMs to identify fine-grained opinion spans in text. We also present a dedicated methodology for an LLM to adjudicate multiple labels and produce final annotations, benchmarked against exact, flexible, and element-wise variants of a rule-based voting aggregator. We trial the pipeline with models of different sizes for the Aspect Sentiment Triplet Extraction (ASTE) and Aspect-Category-Opinion-Sentiment (ACOS) analysis tasks. Our results reveal a critical performance bifurcation: LLMs are reliable at the span level yet struggle to reproduce the relational structures that connect those spans faithfully. This suggests that LLMs are better positioned as high-fidelity annotation assistants and data augmentation tools to expand fine-grained opinion-annotated datasets, rather than replacing human annotators entirely.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
