Preregistration refers to the practice of specifying what you are going to do, and what you expect to find in your study, before carrying out the study. This practice is increasingly common in medicine and psychology, but is rarely discussed in NLP. This paper discusses preregistration in more detail, explores how NLP researchers could preregister their work, and presents several preregistration questions for different kinds of studies. Finally, we argue in favour of registered reports, which could provide firmer grounds for slow science in NLP research. The goal of this paper is to elicit a discussion in the NLP community, which we hope to synthesise into a general NLP preregistration form in future research.
Sejarah dan Perkembangan Teknik Natural Language Processing (NLP) Bahasa Indonesia: Tinjauan tentang sejarah, perkembangan teknologi, dan aplikasi NLP dalam bahasa Indonesia
The Nature of NLP: Analyzing Contributions in NLP Papers
Ubiquitous Knowledge Processing Lab (UKP Lab) Department of Computer Science and Hessian Center for AI (hessian.AI) Technische Universität Darmstadt(德累斯顿技术大学计算机科学系及海瑟森人工智能中心);IT:U Interdisciplinary Transformation University Austria(奥地利 interdisciplinary Transformation 大学);IBM Research Europe - Ireland(IBM欧洲爱尔兰研究中心);National Research Council Canada(加拿大国家研究委员会)
A Two-Sided Discussion of Preregistration of NLP Research
University of Copenhagen(哥本哈根大学);KU Leuven(鲁汶大学)
Van Miltenburg et al.(2021)建议NLP研究应该采用预注册,以防止钓鱼式搜索,并促进负面结果的发表。从表面上看,这是一个非常合理的建议,似乎解决了NLP研究的许多方法论问题。我们讨论了利弊-一些旧的,一些新的:a)预注册受到结果已知后检索假设的实践的挑战; b)预注册可能会使NLP偏向验证性研究; c)预注册必须允许将研究重新分类为探索性研究; d)预注册可能会增加出版偏见; e)预注册可能会增加插旗; f)预注册可能会增加p-hacking;最后,g)预先登记可能会降低我们的风险容忍度。我们把我们的讨论作为一个对话,提出辩论的双方。
Beyond Good Intentions: Reporting the Research Landscape of NLP for Social Good
ETH Zürich(苏黎世联邦理工学院);MPI for Intelligent Systems(马克斯·普朗克智能系统研究所);University of Michigan(密歇根大学);The Hebrew University of Jerusalem(耶路撒冷希伯来大学);Allen Institute for AI (AI2)(艾伦人工智能研究所(AI2))
From Insights to Actions: The Impact of Interpretability and Analysis Research on NLP
Pontificia Universidad Católica de Chile(智利天主教大学);CENIA(智利国家环境与可持续发展中心);Mila Quebec AI Institute(米拉魁北克人工智能研究所);McGill University(麦吉尔大学);Saarland University(萨尔大学);Tel Aviv University(特拉维夫大学)
Modelling Analogies and Analogical Reasoning: Connecting Cognitive Science Theory and NLP Research
Computation, Cognition & Language Group, Idiap Research Institute(计算、认知与语言组,Idiap研究所);NLP Lab, EPFL(NLP实验室,EPFL);Psychological Methods, University of Amsterdam(心理方法,阿姆斯特丹大学);Faculty of Communication, Culture and Society and Faculty of Informatics Università della Svizzera italiana(传播、文化与社会学院和信息学院,瑞士大学)
Social Good or Scientific Curiosity? Uncovering the Research Framing Behind NLP Artefacts
Department of Computer Science and Technology, University of Cambridge(计算机科学与技术系,剑桥大学);Cardiff University(卡迪夫大学);Queen Mary University of London(伦敦女王学院)
Culturally Aware and Adapted NLP: A Taxonomy and a Survey of the State of the Art
Technical University of Darmstadt(达姆施塔特工业大学);Konrad Zuse School of Excellence in Learning and Intelligent Systems (ELIZA)(康拉德·楚泽学习与智能系统卓越学院(ELIZA));University of Cambridge(剑桥大学)
The Elephant in the Room: Analyzing the Presence of Big Tech in Natural Language Processing Research
Institute for Better Health(健康促进研究所);University of Göttingen(哥廷根大学);Université Paris-Saclay(巴黎萨克雷大学);Sorbonne Université(索邦大学);LORIA(洛里亚实验室);National Research Council(国家研究委员会)
North Africa's Missing Framework: NLP-Driven Mental Healthcare in Algeria and Implications for Low-resource Settings
University Mohamed El Bachir El Ibrahimi of Bordj Bou Arreridj(布吉布阿雷里吉·穆罕默德·巴希尔·易卜拉欣大学);Laboratory of Informatics and its Applications of M’sila(姆西拉信息学及其应用实验室);Laboratory of Materials and Electronic Systems(材料与电子系统实验室)
Handling and Presenting Harmful Text in NLP Research
University of Oxford(牛津大学);The Alan Turing Institute(艾伦·图灵研究所);Mozilla Foundation(Mozilla基金会);University College Dublin(都柏林大学学院);IT University of Copenhagen(哥本哈根信息技术大学)
One Country, 700+ Languages: NLP Challenges for Underrepresented Languages and Dialects in Indonesia
Amazon(亚马逊);Bloomberg(彭博);The University of Melbourne(墨尔本大学);HKUST(香港科技大学);Telkom University(电信大学);INACL(印尼计算语言学协会);Universitas Indonesia(印度尼西亚大学);Kanda University of International Studies(神田外国语大学);MBZUAI(穆罕默德·本·扎耶德人工智能大学);Google Research(谷歌研究院)
Natural Language Processing in Ethiopian Languages: Current State, Challenges, and Opportunities
Bahir Dar University(巴赫达尔大学);Universität Hamburg(汉堡大学);Wuhan University of Technology(武汉理工大学);Instituto Politécnico Nacional(国立理工学院);Wollo University(沃洛大学);Saarland University(萨尔大学)
The Rise of AfricaNLP: A Survey of Contributions, Contributors, Community Impact, and Bibliometric Analysis
Instituto Politécnico Nacional(国立理工学院);University of Gondar(贡德尔大学);Northwest University Kano(卡诺西北大学);Bayero University Kano(巴耶罗大学卡诺分校);University of Huddersfield(哈德斯菲尔德大学);iCog Labs(iCog实验室);Pennsylvania State University(宾夕法尼亚州立大学);University of Pretoria(比勒陀利亚大学);Masakhane;University of Hamburg(汉堡大学);Imperial College London(伦敦帝国理工学院)
Speciesism in Natural Language Processing Research
Graduate School of Information Science and Technology, Hokkaido University(北海道大学信息科学与技术研究生院);Faculty of Information Science and Technology, Hokkaido University(北海道大学信息科学与技术学院)
HatePRISM: Policies, Platforms, and Research Integration. Advancing NLP for Hate Speech Proactive Mitigation
IIT Kharagpur(印度理工学院Kharagpur分校);University of Hamburg(汉堡大学);Technical University of Munich(慕尼黑技术大学);Bucerius Law School(布勒斯海姆法学院);Skolkovo Institute of Science and Technology(斯克尔科维科学与技术研究所);Artificial Intelligence Research Institute(人工智能研究 institute)
Charting the Landscape of African NLP: Mapping Progress and Shaping the Road Ahead
Saarland University, Saarland Informatics Campus(萨尔兰大学,萨尔兰信息学校区);LMU Munich and Munich Center for Machine Learning(慕尼黑大学及慕尼黑机器学习中心);Mila - Quebec AI Institute, McGill University & Canada CIFAR AI Chair(魁北克人工智能研究所,麦吉尔大学及加拿大 CIFAR 人工智能 chair)