arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2305.05754cs.CLcs.AIcs.LG
收藏

何时提问与问什么:基于世界状态和文本指令的 IGLU NLP 挑战赛解决方案

When and What to Ask Through World States and Text Instructions: IGLU NLP Challenge Solution

University College London(伦敦大学学院)

Zhengxiang Shi, Jerome Ramos, To Eun Kim, Xi Wang, Hossein A. Rahmani, Aldo Lipani

AI总结:

针对协作建造中的歧义问题,提出分类与排序方法,分别决定何时提问及选择澄清问题,在NeurIPS 2022 NLP任务中取得F1 0.757和MRR约0.38的成绩。

摘要:

在协作任务中,有效沟通对于实现共同目标至关重要。其中一项协作任务是协作建造,建造者必须相互沟通,在类似Minecraft的模拟环境中构建所需结构。我们旨在开发一个智能建造代理,基于通过对话进行的用户输入来构建结构。然而,在协作建造中,建造者可能会遇到基于现有信息和指令难以解释的情况,导致歧义。在NeurIPS 2022竞赛NLP任务中,我们解决两个关键研究问题,以填补这一空白:代理何时应请求澄清,以及应提出哪些澄清问题?我们通过两个子任务来实现这一目标:分类任务和排序任务。对于分类任务,目标是基于当前世界状态和对话历史确定代理是否应请求澄清。对于排序任务,目标是从候选池中对相关澄清问题进行排序。在本报告中,我们简要介绍我们用于分类和排序任务的方法。对于分类任务,我们的模型取得了0.757的F1分数,在排行榜上位列第3。对于排序任务,通过扩展传统排序模型,我们的模型在平均倒数排名上取得了约0.38的成绩。最后,我们讨论了用于排序任务的各种神经方法以及未来方向。

英文摘要:

In collaborative tasks, effective communication is crucial for achieving joint goals. One such task is collaborative building where builders must communicate with each other to construct desired structures in a simulated environment such as Minecraft. We aim to develop an intelligent builder agent to build structures based on user input through dialogue. However, in collaborative building, builders may encounter situations that are difficult to interpret based on the available information and instructions, leading to ambiguity. In the NeurIPS 2022 Competition NLP Task, we address two key research questions, with the goal of filling this gap: when should the agent ask for clarification, and what clarification questions should it ask? We move towards this target with two sub-tasks, a classification task and a ranking task. For the classification task, the goal is to determine whether the agent should ask for clarification based on the current world state and dialogue history. For the ranking task, the goal is to rank the relevant clarification questions from a pool of candidates. In this report, we briefly introduce our methods for the classification and ranking task. For the classification task, our model achieves an F1 score of 0.757, which placed the 3rd on the leaderboard. For the ranking task, our model achieves about 0.38 for Mean Reciprocal Rank by extending the traditional ranking model. Lastly, we discuss various neural approaches for the ranking task and future direction.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑