我们介绍了Unbabel团队对WMT 2020共享任务的贡献。我们打算参与所有语言对的段级、文档级和系统级轨道,以及“QE as a Metric”轨道。因此,我们参考去年的测试集来说明我们的模型在这些轨道上的结果。我们的意见书建立在最近提出的COMET框架:我们训练几个估计模型回归不同的人类产生的质量分数和一个新的排名模型训练的相对排名从直接评估。我们还提出了一种简单的技术,用于将段级预测转换为文档级分数。总的来说,我们的系统在以前的测试集上对所有语言对都取得了很好的结果,并且在许多情况下创造了一个新的最先进的水平。
英文摘要:
We present the contribution of the Unbabel team to the WMT 2020 Shared Task on Metrics. We intend to participate on the segment-level, document-level and system-level tracks on all language pairs, as well as the 'QE as a Metric' track. Accordingly, we illustrate results of our models in these tracks with reference to test sets from the previous year. Our submissions build upon the recently proposed COMET framework: We train several estimator models to regress on different human-generated quality scores and a novel ranking model trained on relative ranks obtained from Direct Assessments. We also propose a simple technique for converting segment-level predictions into a document-level score. Overall, our systems achieve strong results for all language pairs on previous test sets and in many cases set a new state-of-the-art.
Towards Linguistically Generalizable NLP Systems: A Workshop and Shared Task
University of Maryland(马里兰大学);University of Washington(华盛顿大学);Microsoft Research New York(微软纽约研究院)
本文介绍了第一次研讨会上建立语言概括的自然语言处理系统,以及相关的Build It Break It,语言版共享任务的总结。这次研讨会的目的是将NLP和语言学的研究人员聚集在一起,共同完成一项任务,旨在测试NLP系统在其训练数据分布之外的通用性。我们描述了共享任务的动机、设置和参与,讨论了一些突出的结果,并讨论了经验教训。
LeWiDi-2025 at NLPerspectives: Third Edition of the Learning with Disagreements Shared Task
Fondazione Bruno Kessler(布鲁诺·科塞拉基金会);LMU Munich & MCML(慕尼黑大学及MCML);Università Milano Bicocca(米兰Bicocca大学);Università di Torino(都灵大学);University of Gothenburg(哥德堡大学);Queen Mary University of London(伦敦大学玛丽女王学院);Utrecht University(乌得勒支大学)
AraHealthQA 2025: The First Shared Task on Arabic Health Question Answering
Umm Al-Qura University(乌姆·阿勒·库拉大学);New York University Abu Dhabi(纽约大学阿布扎比分校);The University of British Columbia(不列颠哥伦比亚大学);King Khalid University(国王 Khalid 大学)
SJTU-NICT's Supervised and Unsupervised Neural Machine Translation Systems for the WMT20 News Translation Task
Shanghai Jiao Tong University (SJTU)(上海交通大学);National Institute of Information and Communications Technology (NICT)(日本信息通信研究机构(NICT));Department of Computer Science and Engineering, Shanghai Jiao Tong University (SJTU)(上海交通大学计算机科学与工程系)
Progress Note Understanding -- Assessment and Plan Reasoning: Overview of the 2022 N2C2 Track 3 Shared Task
University of Wisconsin Madison(威斯康星大学麦迪逊分校);Loyola University Chicago(芝加哥洛约拉大学);Boston Children’s Hospital(波士顿儿童医院);Harvard University(哈佛大学);George Mason University(乔治梅森大学)
Overview of the MedHopQA track at BioCreative IX: track description, participation and evaluation of systems for multi-hop medical question answering
National Library of Medicine (NLM), National Institutes of Health (NIH)(美国国家医学图书馆(NLM)、国家卫生研究院(NIH));University of Illinois at Urbana Champaign(伊利诺伊大学厄巴纳-香槟分校);Korea University(韩国大学);VNU University of Engineering and Technology, Hanoi, Vietnam(越南河内工程大学);Concordia University, Montreal, QC, CA(蒙特利尔大学);Institute of Biomedical Chemistry (IBMC), 10 bld. 8, Pogodinskaya str., 119121 Moscow, Russia(俄罗斯生物医学化学研究所(IBMC));LASIGE, Departamento de Informática, Faculdade de Ciências, Universidade de Lisboa, 1749-016 Lisbon, Portugal(葡萄牙里斯本大学 LASIGE 实验室);Faculty of Engineering, Computer Engineering Department Cairo University(埃及开罗大学工程学院);Menzies School of Health Research, Charles Darwin University, NT, Australia(澳大利亚查尔斯达尔文大学梅恩兹健康研究中心);CaresAI, Australia(澳大利亚 CaresAI)
本文介绍了BioCreative IX MedHopQA共享任务,旨在评估大型语言模型在多跳推理中的表现,通过构建1000个挑战性问题,展示了检索增强生成策略的重要性,并提供了公开数据集和评估结果。
Transformers and Ensemble methods: A solution for Hate Speech Detection in Arabic languages
Universitat Politècnica de València(瓦伦西亚理工大学);Constantine 2 University(君士坦丁2大学);ESCF de Constantine(君士坦丁ESCF);Hamad Bin Khalifa University(哈马德·本·哈利法大学)