arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2205.13754cs.CLcs.HC
收藏

NLU用于基于游戏的真实学习:初步评估

NLU for Game-based Learning in Real: Initial Evaluations

Intel Labs(英特尔研究院)

Eda Okur, Saurav Sahay, Lama Nachman

摘要:

为基于游戏的交互设计的智能系统应该对用户及其周围环境有上下文意识。口语对话系统(SDS)是这些交互式代理进行有效的目标为导向的沟通与用户实时的关键。对于现实世界(即,在野外)部署这种会话代理时,改进面向目标的SDS管道的自然语言理解(NLU)模块至关重要,特别是对于有限的特定于任务的数据集。这项研究探讨了最近提出的基于transformer的多任务NLU架构的潜在好处,主要是对小型特定领域的教育游戏数据集进行意图识别。评估数据集是从儿童通过基于游戏的学习环境中基于游戏的互动练习基本数学概念中收集的。我们研究了NLU在初始概念验证游戏数据集与真实世界部署数据集上的性能,并观察了预期的性能下降。我们已经表明,与更简单的基线方法相比,双重意图和实体Transformer(DIET)架构足够强大,可以在很大程度上处理真实世界的数据,用于这些特定于域的野外游戏数据集上的意图识别任务。

英文摘要:

Intelligent systems designed for play-based interactions should be contextually aware of the users and their surroundings. Spoken Dialogue Systems (SDS) are critical for these interactive agents to carry out effective goal-oriented communication with users in real-time. For the real-world (i.e., in-the-wild) deployment of such conversational agents, improving the Natural Language Understanding (NLU) module of the goal-oriented SDS pipeline is crucial, especially with limited task-specific datasets. This study explores the potential benefits of a recently proposed transformer-based multi-task NLU architecture, mainly to perform Intent Recognition on small-size domain-specific educational game datasets. The evaluation datasets were collected from children practicing basic math concepts via play-based interactions in game-based learning settings. We investigate the NLU performances on the initial proof-of-concept game datasets versus the real-world deployment datasets and observe anticipated performance drops in-the-wild. We have shown that compared to the more straightforward baseline approaches, Dual Intent and Entity Transformer (DIET) architecture is robust enough to handle real-world data to a large extent for the Intent Recognition task on these domain-specific in-the-wild game datasets.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑