arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2411.08752cs.CL
收藏

多视角立场检测

Multi-Perspective Stance Detection

University of Pisa, Italy(比萨大学)

Benedetta Muscato, Praveen Bushipaka, Gizem Gezici, Lucia Passaro, Fosca Giannotti

AI总结:

本研究提出多视角立场检测方法,利用多个标注者的分歧信息训练视角感知分类模型,实验表明该方法优于单一标签基线,并强调包容性AI设计在伦理与性能上的双重优势。

摘要:

主观性自然语言处理任务通常依赖于由多个标注者提供的人工标注,由于标注者背景和生活经历的不同,其判断可能有所差异。传统方法往往将多个标注聚合为单一的真实标签,忽视了因标注者分歧而产生的视角多样性。在这项初步研究中,我们考察了在分类任务中包含多个标注对模型准确率的影响。我们的方法研究了视角感知分类模型在立场检测任务中的性能,并进一步检验标注者分歧是否影响模型置信度。结果表明,多视角方法取得了更好的分类性能,优于使用单一标签的基线方法。这意味着设计更具包容性的视角感知AI模型不仅是实现负责任和合乎伦理的AI的关键第一步,而且也能比传统方法取得更优的结果。

英文摘要:

Subjective NLP tasks usually rely on human annotations provided by multiple annotators, whose judgments may vary due to their diverse backgrounds and life experiences. Traditional methods often aggregate multiple annotations into a single ground truth, disregarding the diversity in perspectives that arises from annotator disagreement. In this preliminary study, we examine the effect of including multiple annotations on model accuracy in classification. Our methodology investigates the performance of perspective-aware classification models in stance detection task and further inspects if annotator disagreement affects the model confidence. The results show that multi-perspective approach yields better classification performance outperforming the baseline which uses the single label. This entails that designing more inclusive perspective-aware AI models is not only an essential first step in implementing responsible and ethical AI, but it can also achieve superior results than using the traditional approaches.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑