In this paper we determine how multi-layer ensembling improves performance on multilingual intent classification. We develop a novel multi-layer ensembling approach that ensembles both different model initializations and different model architectures. We also introduce a new banking domain dataset and compare results against the standard ATIS dataset and the Chinese SMP2017 dataset to determine ensembling performance in multilingual and multi-domain contexts. We run ensemble experiments across all three datasets, and conclude that ensembling provides significant performance increases, and that multi-layer ensembling is a no-risk way to improve performance on intent classification. We also find that a diverse ensemble of simple models can reach perform comparable to much more sophisticated state-of-the-art models. Our best F 1 scores on ATIS, Banking, and SMP are 97.54%, 91.79%, and 93.55% respectively, which compare well with the state-of-the-art on ATIS and best submission to the SMP2017 competition. The total ensembling performance increases we achieve are 0.23%, 1.96%, and 4.04% F 1 respectively.
Ensembling Multilingual Transformers for Robust Sentiment Analysis of Tweets
Department of Computer Science, University of South Carolina, USA(南卡罗来纳大学计算机科学系);Artificial Intelligence Institute, University of South Carolina, USA(南卡罗来纳大学人工智能研究所);Department of Civil, Environmental, and Ocean Engineering, Stevens Institute of Technology, New Jersey, USA(史蒂文斯理工学院土木、环境与海洋工程系);School of Computing, University of Nebraska–Lincoln, Lincoln, Nebraska, USA(内布拉斯加大学林肯分校计算机学院);Department of Computer Science, Bowling Green State University, Bowling Green, Ohio, USA(伯灵顿州立大学计算机科学系)
STIL -- Simultaneous Slot Filling, Translation, Intent Classification, and Language Identification: Initial Results using mBART on MultiATIS++
Amazon Alexa AI(亚马逊Alexa人工智能部门)
时隙填充、翻译、意图分类和语言识别(STIL)是多语言自然语言理解(NLU)的一个新任务。通过同时执行槽填充和翻译成单一输出语言(在这种情况下为英语),下游系统组件的某些部分可以是单语的,从而降低开发和维护成本。使用多语言BART模型给出了结果(Liu et al.,2020)使用MultiATIS ++数据集对7种语言进行了微调。当不执行翻译时,mBART的性能与测试语言的当前最先进系统(Xu等人的跨语言BERT(2020))相当,平均意图分类准确率更高(96.07%对95.50%),但平均槽F1更差(89.87%对90.81%)。当执行同时翻译时,平均意图分类准确度相对仅降低1.7%,平均槽F1相对仅降低1.2%。
Intent Classification for Bank Chatbots through LLM Fine-Tuning
Slovak National Supercomputing Centre(斯洛伐克国家超级计算中心);nettle, s.r.o.(nettle有限责任公司);Institute of Information Engineering, Automation, and Mathematics, Slovak University of Technology in Bratislava(布拉迪斯拉发斯洛伐克理工大学信息工程、自动化与数学研究所)
Multilingual Datasets for Custom Input Extraction and Explanation Requests Parsing in Conversational XAI Systems
Technische Universität Berlin(技术大学柏林);German Research Center for Artificial Intelligence (DFKI)(德国人工智能研究中心);Saarland Informatics Campus(萨尔州信息学校区);Centre for European Research in Trusted AI (CERTAIN)(可信人工智能欧洲研究中心);BIFOLD – Berlin Institute for the Foundations of Learning and Data(柏林学习与数据基础研究院)
HIT-SCIR at MMNLU-22: Consistency Regularization for Multilingual Spoken Language Understanding
Harbin Institute of Technology(哈尔滨工业大学)
多语言口语语言理解(SLU)包括两个子任务,即意图检测和槽填充。为了提高这两个子任务的性能,我们建议使用基于混合数据增强策略的一致性正则化。一致性正则化强制示例的预测分布及其语义上等价的增广是一致的。我们在MASSIVE数据集上进行了全数据集和zero-shot设置下的实验。实验结果表明,我们提出的方法提高了性能的意图检测和插槽填充任务。我们的系统\footnote{The code will be available at \url{https://github.com/bozheng-hit/MMNLU-22-HIT-SCIR}.}在MMNLU-22比赛中在全数据集设置下排名第一。
M4FC: a Multimodal, Multilingual, Multicultural, Multitask Real-World Fact-Checking Dataset
Mohamed bin Zayed University of Artificial Intelligence(Mohamed bin Zayed人工智能大学);Ubiquitous Knowledge Processing Lab(ubiquitous知识处理实验室);Department of Computer Science, TU Darmstadt(TU Darmstadt计算机科学系);National Research Center for Applied Cybersecurity ATHENE(应用网络安全国家研究中心ATHENE);Department of Electrical Engineering, KU Leuven(KU Leuven电气工程系);Department of Computer Science, KU Leuven(KU Leuven计算机科学系)
Extended Multilingual Protest News Detection -- Shared Task 1, CASE 2021 and 2022
KNAW Humanities Cluster(荷兰皇家艺术与科学学院人文集群);Koc University(科奇大学);Mersin University(梅尔辛大学);Huawei(华为);UNC Charlotte(北卡罗来纳大学夏洛特分校);Birmingham City University(伯明翰城市大学);ETH Zurich(苏黎世联邦理工学院);National Institute of Japanese Literature(日本国立文学研究所);Bulgarian Academy of Sciences(保加利亚科学院);University of São Paulo(圣保罗大学);Sabanci University(萨班哲大学)
INJONGO: A Multicultural Intent Detection and Slot-filling Dataset for 16 African Languages
McGill University(麦吉尔大学);Mila, Quebec AI Institute(米拉魁北克人工智能研究所);Saarland University(萨尔大学);SADiLaR(南非数字语言资源中心);University of Toronto(多伦多大学);OntarioTech University(安大略理工大学);Princeton University(普林斯顿大学);Makerere University(马凯雷雷大学);L3S Research Center(L3S研究中心);Imperial College London(伦敦帝国学院);Universidad de Deusto(德乌斯托大学);Dakar American University Of Science and Technology(达喀尔美国科技大学)
Exploring Adversarial Robustness and Safety Alignment in Multilingual Multi-Modal Large Language Models
Mohamed Bin Zayed University of AI, UAE(穆罕默德·本·扎耶德人工智能大学,阿联酋);Khalifa University, UAE(卡比拉大学,阿联酋);Australian National University, Australia(澳大利亚国立大学,澳大利亚)
Fleurs-SLU: A Massively Multilingual Benchmark for Spoken Language Understanding
Center For Artificial Intelligence and Data Science, University of Würzburg(人工智能与数据科学中心,乌尔姆大学);Language Technology Lab, University of Cambridge(语言技术实验室,剑桥大学);Mila, McGill University and Canada CIFAR AI Chair(Mila,麦吉尔大学及加拿大CIFAR人工智能主席)
Qatar Computing Research Institute(卡塔尔计算研究所);Sofia University "St. Kliment Ohridski"(索菲亚大学"圣克莱孟·奥赫里迪斯");University of Padova(帕多瓦大学);Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学)
IKS-Instruct: A 24,000-Example Multilingual Dataset for Teaching Language Models Indian Knowledge Systems
Intrinsic Lab, Centre for Sensors, Instrumentation and Cyber-Physical System Engineering (Centre for SeNSE), Indian Institute of Technology Delhi(印度理工学院德里分校传感器、仪器仪表和网络物理系统工程中心(传感中心)本征实验室);RSL Quantum, FITT, IIT Delhi(印度理工学院德里分校 FITT 学院 RSL 量子实验室)