arXiv 2403.189332024-04-17cs.CL

SemEval-2024 Task 1：面向非洲与亚洲语言的语义文本相关性

SemEval-2024 Task 1: Semantic Textual Relatedness for African and Asian Languages

Cardiff University（卡迪夫大学）；University of Pretoria（比勒陀利亚大学）；Northeastern University（东北大学）；BITS Pilani（pilani bits 学院）；MBZUAI（穆罕默德·本·扎耶德人工智能大学）；KU Leuven（鲁汶大学）；Uppsala University（乌普萨拉大学）；IIIT Hyderabad（海德拉巴印度信息技术学院）；University of Toronto（多伦多大学）；Universität Hamburg（汉堡大学）；National Research Council Canada（加拿大国家研究委员会）；Imperial College London（伦敦帝国学院）

Nedjma Ousidhoum, Shamsuddeen Hassan Muhammad, Mohamed Abdalla, Idris Abdulmumin, Ibrahim Said Ahmad, Sanchit Ahuja, Alham Fikri Aji, Vladimir Araujo, Meriem Be… 展开作者

Nedjma Ousidhoum, Shamsuddeen Hassan Muhammad, Mohamed Abdalla, Idris Abdulmumin, Ibrahim Said Ahmad, Sanchit Ahuja, Alham Fikri Aji, Vladimir Araujo, Meriem Beloucif, Christine De Kock, Oumaima Hourrane, Manish Shrivastava, Thamar Solorio, Nirmal Surange, Krishnapriya Vishnubhotla, Seid Muhie Yimam, Saif M. Mohammad

备注：: SemEval 2024 Task Description Paper. arXiv admin note: text overlap with arXiv:2402.08638

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文介绍SemEval-2024首个语义文本相关性共享任务，针对亚非14种低资源语言设置三类赛道，吸引众多团队参与，并总结了各赛道的最优系统与有效方法。

摘要：

我们提出了首个语义文本相关性（Semantic Textual Relatedness，STR）共享任务。早期共享任务主要聚焦于语义相似度，而我们则针对14种语言研究更宽泛的语义相关性现象，这14种语言包括：南非语、阿尔及利亚阿拉伯语、阿姆哈拉语、英语、豪萨语、印地语、印度尼西亚语、卢旺达语、马拉地语、摩洛哥阿拉伯语、现代标准阿拉伯语、旁遮普语、西班牙语和泰卢固语。这些语言分属五个不同语系，主要通行于非洲和亚洲——这两个地区的自然语言处理资源相对匮乏。数据集中的每个实例为一个句子对，附带一个表示两个句子间语义文本相关程度的分数。参赛系统需在三个主要赛道中，按语义相近程度（即语义相关度）对14种语言的句子对进行排序，三个赛道分别为：(a) 监督赛道、(b) 无监督赛道以及(c) 跨语言赛道。该任务共吸引163名参与者，我们收到了来自51支不同团队的总计70份提交（覆盖所有任务），以及38篇系统描述论文。本文报告了三个不同赛道中表现最优的系统，以及最常见、最有效的方法。

英文摘要：

We present the first shared task on Semantic Textual Relatedness (STR). While earlier shared tasks primarily focused on semantic similarity, we instead investigate the broader phenomenon of semantic relatedness across 14 languages: Afrikaans, Algerian Arabic, Amharic, English, Hausa, Hindi, Indonesian, Kinyarwanda, Marathi, Moroccan Arabic, Modern Standard Arabic, Punjabi, Spanish, and Telugu. These languages originate from five distinct language families and are predominantly spoken in Africa and Asia -- regions characterised by the relatively limited availability of NLP resources. Each instance in the datasets is a sentence pair associated with a score that represents the degree of semantic textual relatedness between the two sentences. Participating systems were asked to rank sentence pairs by their closeness in meaning (i.e., their degree of semantic relatedness) in the 14 languages in three main tracks: (a) supervised, (b) unsupervised, and (c) crosslingual. The task attracted 163 participants. We received 70 submissions in total (across all tasks) from 51 different teams, and 38 system description papers. We report on the best-performing systems as well as the most common and the most effective approaches for the three different tracks.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。