When and What to Ask Through World States and Text Instructions: IGLU NLP Challenge Solution
University College London(伦敦大学学院)
Zhengxiang Shi, Jerome Ramos, To Eun Kim, Xi Wang, Hossein A. Rahmani, Aldo Lipani
备注:
The work won NIPS 2022 IGLU Competition Research Prize. The first two authors contribute equally. arXiv admin note: substantial text overlap with arXiv:2204.08373
In collaborative tasks, effective communication is crucial for achieving joint goals. One such task is collaborative building where builders must communicate with each other to construct desired structures in a simulated environment such as Minecraft. We aim to develop an intelligent builder agent to build structures based on user input through dialogue. However, in collaborative building, builders may encounter situations that are difficult to interpret based on the available information and instructions, leading to ambiguity. In the NeurIPS 2022 Competition NLP Task, we address two key research questions, with the goal of filling this gap: when should the agent ask for clarification, and what clarification questions should it ask? We move towards this target with two sub-tasks, a classification task and a ranking task. For the classification task, the goal is to determine whether the agent should ask for clarification based on the current world state and dialogue history. For the ranking task, the goal is to rank the relevant clarification questions from a pool of candidates. In this report, we briefly introduce our methods for the classification and ranking task. For the classification task, our model achieves an F1 score of 0.757, which placed the 3rd on the leaderboard. For the ranking task, our model achieves about 0.38 for Mean Reciprocal Rank by extending the traditional ranking model. Lastly, we discuss various neural approaches for the ranking task and future direction.
What Do NLP Researchers Believe? Results of the NLP Community Metasurvey
Paul G. Allen School for Computer Science & Engineering, University of Washington(华盛顿大学保罗·G·艾伦计算机科学与工程学院);Center for Data Science, New York University(纽约大学数据科学中心);Courant Institute of Mathematical Sciences, New York University(纽约大学柯朗数学科学研究所);Department of Linguistics, New York University(纽约大学语言学系);Department of Computer Science, Johns Hopkins University(约翰斯·霍普金斯大学计算机科学系)
The Web Is Your Oyster - Knowledge-Intensive NLP against a Very Large Web Corpus
Facebook AI Research(脸书人工智能研究院);University College London(伦敦大学学院);University of Mannheim(曼海姆大学);ENS, PSL University(巴黎高等师范学院,巴黎文理研究大学);Inria(法国国家信息与自动化研究所);University of Washington(华盛顿大学)
NLP Methods May Actually Be Better Than Professors at Estimating Question Difficulty
Center for Language and Cognition, University of Groningen(语言与认知中心,格罗宁根大学);Bernoulli Institute, University of Groningen(伯努利研究所,格罗宁根大学);Department of Experimental Psychology, University of Groningen(实验心理学系,格罗宁根大学)
Prompting Multilingual Large Language Models to Generate Code-Mixed Texts: The Case of South East Asian Languages
Brown University(布朗大学);UC Berkeley(加州大学伯克利分校);University of California, Los Angeles(加州大学洛杉矶分校);HKUST(香港科技大学);Bloomberg(彭博);EleutherAI;Samsung R&D Institute Philippines(三星菲律宾研发中心);Stanford University(斯坦福大学);National University of Singapore(新加坡国立大学);VietAI Research(VietAI研究院);University of Potsdam(波茨坦大学);MBZUAI(穆罕默德·本·扎耶德人工智能大学)
Let Me Know What to Ask: Interrogative-Word-Aware Question Generation
KAIST(韩国科学技术院)
问题生成(Question Generation,QG)是一项自然语言处理(Natural Language Processing,NLP)任务,有助于问答(Question Questioning,QA)和会话助理的发展。现有的模型专注于基于文本生成问题,并且可能是对所生成的问题的答案。他们需要确定要生成的疑问词的类型,同时必须注意问题的语法和词汇。在这项工作中,我们提出了疑问词感知问题生成(IWAQG),一个流水线系统由两个模块组成:一个疑问词分类器和QG模型。第一模块预测提供给第二模块以创建问题的疑问词。由于增加了决定用于生成的问题的疑问词的召回,所提出的模型在SQuAD中的QG任务上实现了新的最先进的结果,在BLEU-1中从46.58提高到47.69,在BLEU-4中从17.55提高到18.53,在METEOR中从21.24提高到22.33,在ROUGE-L中从44.53提高到46.94。