AI-UPV 在 EXIST 2023 中的参赛系统——基于大语言模型在“带分歧学习”机制下的性别歧视表征
AI-UPV at EXIST 2023 -- Sexism Characterization Using Large Language Models Under The Learning with Disagreements Regime
AI总结：
本文提出 AI-UPV 系统，利用 mBERT 和 XLM-RoBERTa 大语言模型及集成策略，在“带分歧学习”机制下直接利用带分歧数据完成 EXIST 2023 性别歧视识别与表征任务，并在任务 3 中取得第一名。
摘要：
随着社交媒体平台的影响力日益增强，开发能够检测性别歧视及其他不尊重和仇恨行为的自动化系统，对于营造更具包容性和尊重性的网络环境已变得至关重要。然而，考虑到不同的仇恨类别和作者意图，这些任务具有相当大的挑战性，尤其是在“带分歧学习”（learning with disagreements）机制下。本文描述了 AI-UPV 团队参加 CLEF 2023 中 EXIST（社交网络性别歧视识别）实验室的情况。所提出的方法旨在通过直接从带分歧的数据中进行训练，而不使用任何聚合标签，来解决“带分歧学习”范式下的性别歧视识别与表征任务。同时，本文报告了在软评估和硬评估两种方式下的性能表现。所提出的系统使用大语言模型（即 mBERT 和 XLM-RoBERTa）以及集成策略，对英语和西班牙语中的性别歧视进行识别和分类。具体而言，我们的系统由三条不同的流水线组成。集成方法优于单个大语言模型，在采用软标签评估和硬标签评估时均取得了最佳性能。本文描述了在全部三项 EXIST 任务中的参与情况，在软评估下，系统在 EXIST 任务 2 中获得第四名，在任务 3 中获得第一名，取得了最高 ICM-Soft 为 -2.32、归一化 ICM-Soft 为 0.79 的成绩。我们方法的源代码已在 https://github.com/AngelFelipeMP/Sexism-LLM-Learning-With-Disagreement 上公开提供。
英文摘要：
With the increasing influence of social media platforms, it has become crucial to develop automated systems capable of detecting instances of sexism and other disrespectful and hateful behaviors to promote a more inclusive and respectful online environment. Nevertheless, these tasks are considerably challenging considering different hate categories and the author's intentions, especially under the learning with disagreements regime. This paper describes AI-UPV team's participation in the EXIST (sEXism Identification in Social neTworks) Lab at CLEF 2023. The proposed approach aims at addressing the task of sexism identification and characterization under the learning with disagreements paradigm by training directly from the data with disagreements, without using any aggregated label. Yet, performances considering both soft and hard evaluations are reported. The proposed system uses large language models (i.e., mBERT and XLM-RoBERTa) and ensemble strategies for sexism identification and classification in English and Spanish. In particular, our system is articulated in three different pipelines. The ensemble approach outperformed the individual large language models obtaining the best performances both adopting a soft and a hard label evaluation. This work describes the participation in all the three EXIST tasks, considering a soft evaluation, it obtained fourth place in Task 2 at EXIST and first place in Task 3, with the highest ICM-Soft of -2.32 and a normalized ICM-Soft of 0.79. The source code of our approaches is publicly available at https://github.com/AngelFelipeMP/Sexism-LLM-Learning-With-Disagreement.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
