Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing: System Demonstrations (2020) 40-46
Statistical significance testing centered on p-values is commonly used to compare NLP system performance, but p-values alone are insufficient because statistical significance differs from practical significance. The latter can be measured by estimating effect size. In this paper, we propose a three-stage procedure for comparing NLP system performance and provide a toolkit, NLPStatTest, that automates the process. Users can upload NLP system evaluation scores and the toolkit will analyze these scores, run appropriate significance tests, estimate effect size, and conduct power analysis to estimate Type II error. The toolkit provides a convenient and systematic way to compare NLP system performance that goes beyond statistical significance testing
Massive Choice, Ample Tasks (MaChAmp): A Toolkit for Multi-task Learning in NLP
IT University of Copenhagen(哥本哈根信息技术大学);University of Groningen(格罗宁根大学);University of Trento(特伦托大学);Factmata(Factmata公司);Microsoft Research - University of Trento(微软研究院-特伦托大学)
ParsiPy: NLP Toolkit for Historical Persian Texts in Python
Amirkabir University of Technology(阿米尔卡比尔理工大学);University of Tehran(德黑兰大学);Open Science Laboratory(开放科学实验室);University of Southern California(南加州大学);Institute for Humanities and Cultural Studies(人文与文化研究院);Qatar Computing Research Institute(卡塔尔计算研究所);Sharif University of Technology(谢里夫理工大学)
pysentimiento: A Python Toolkit for Opinion Mining and Social NLP tasks
Instituto de Ciencias de la Computación, CONICET, Universidad de Buenos Aires(布宜诺斯艾利斯大学 CONICET 计算机科学研究所);Facultad de Astronomía, Matemática y Física, Universidad Nacional de Córdoba(科尔多瓦国立大学天文、数学与物理学院);IIIA-CSIC(西班牙国家研究委员会人工智能研究所)
Ascle: A Python Natural Language Processing Toolkit for Medical Text Generation
Centre for Quantitative Medicine, Duke-NUS Medical School(杜克-新加坡国立大学医学院定量医学中心);Department of Biomedical Informatics, Yong Loo Lin School of Medicine, National University of Singapore(新加坡国立大学杨潞龄医学院生物医学信息学系);Department of Linguistics, Northwestern University(西北大学语言学系);Department of Computer Science, Yale University(耶鲁大学计算机科学系);Yale School of Public Health, Yale University(耶鲁大学公共卫生学院);Yale New Haven Hospital, Yale School of Medicine, Yale University(耶鲁大学医学院耶鲁纽黑文医院);Division of Epidemiology and Clinical Applications, National Eye Institute, National Institutes of Health(美国国立卫生研究院国家眼科研究所流行病学与临床应用部);National Center for Biotechnology Information, National Library of Medicine, National Institutes of Health(美国国立卫生研究院国家医学图书馆国家生物技术信息中心);Section of Biomedical Informatics and Data Science, Yale School of Medicine, Yale University(耶鲁大学医学院生物医学信息学与数据科学科);Information Technology Center, University of Tokyo(东京大学信息技术中心)
Beijing Academy of Artificial Intelligence(北京人工智能研究院);Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);University of Electronic Science and Technology of China(电子科技大学);Harbin Institute of Technology, Shenzhen(哈尔滨工业大学(深圳));College of Computing and Data Science, Nanyang Technological University(南洋理工大学计算与数据科学学院);AstralForge AI Lab(AstralForge AI实验室)