How to build a high-quality multi-domain dialogue system is a challenging work due to its complicated and entangled dialogue state space among each domain, which seriously limits the quality of dialogue policy, and further affects the generated response. In this paper, we propose a novel method to acquire a satisfying policy and subtly circumvent the knotty dialogue state representation problem in the multi-domain setting. Inspired by real school teaching scenarios, our method is composed of multiple domain-specific teachers and a universal student. Each individual teacher only focuses on one specific domain and learns its corresponding domain knowledge and dialogue policy based on a precisely extracted single domain dialogue state representation. Then, these domain-specific teachers impart their domain knowledge and policies to a universal student model and collectively make this student model a multi-domain dialogue expert. Experiment results show that our method reaches competitive results with SOTAs in both multi-domain and single domain setting.
Dialogue systems dealing with multi-domain tasks are highly required. How to record the state remains a key problem in a task-oriented dialogue system. Normally we use human-defined features as dialogue states and apply a state tracker to extract these features. However, the performance of such a system is limited by the error propagation of a state tracker. In this paper, we propose a dialogue generation model that needs no external state trackers and still benefits from human-labeled semantic data. By using a teacher-student framework, several teacher models are firstly trained in their indi…
How can we test whether state-of-the-art generative models, such as Blender and GPT-3, are good AI teachers, capable of replying to a student in an educational dialogue? Designing an AI teacher test is challenging: although evaluation methods are much-needed, there is no off-the-shelf solution to measuring pedagogical ability. This paper reports on a first attempt at an AI teacher test. We built a solution around the insight that you can run conversational agents in parallel to human teachers in real-world dialogues, simulate how different agents would respond to a student, and compare these c…
Tell Me Why: Designing an Explainable LLM-based Dialogue System for Student Problem Behavior Diagnosis
School of Educational Technology, Beijing Normal University, Beijing, China（北京师范大学教育技术学院）；Advanced Innovation Center for Future Education, Beijing Normal University, Beijing, China（北京师范大学未来教育创新中心）；Faculty of Education, The University of Hong Kong, Hong Kong SAR, China（香港大学教育学院）
National University of Singapore（新加坡国立大学）；Robert Bosch (SEA)（罗伯特·博世（东南亚）公司）；Kriston AI Lab（克里斯顿人工智能实验室）；The Chinese University of Hong Kong (Shenzhen)（香港中文大学（深圳））
Chatbots are designed to carry out human-like conversations across different domains, such as general chit-chat, knowledge exchange, and persona-grounded conversations. To measure the quality of such conversational agents, a dialogue evaluator is expected to conduct assessment across domains as well. However, most of the state-of-the-art automatic dialogue evaluation metrics (ADMs) are not designed for multi-domain evaluation. We are motivated to design a general and robust framework, MDD-Eval, to address the problem. Specifically, we first train a teacher evaluator with human-annotated data t…
NetEase Inc.（网易公司）；Tsinghua University（清华大学）；Samsung Research China - Beijing（三星电子中国研究院（北京））；Alibaba Group（阿里巴巴集团）
Recent advances in open-domain dialogue systems rely on the success of neural models that are trained on large-scale data. However, collecting large-scale dialogue data is usually time-consuming and labor-intensive. To address this data dilemma, we propose a novel data augmentation method for training open-domain dialogue models by utilizing unpaired data. Specifically, a data-level distillation process is first proposed to construct augmented dialogues where both post and response are retrieved from the unpaired data. A ranking module is employed to filter out low-quality dialogues. Further, …
IBM Research AI（IBM研究院人工智能部门）；University of Central Florida（中佛罗里达大学）
We propose novel AI-empowered chat bots for learning as conversation where a user does not read a passage but gains information and knowledge through conversation with a teacher bot. Our information-acquisition-oriented dialogue system employs a novel adaptation of reinforced self-play so that the system can be transferred to various domains without in-domain dialogue data, and can carry out conversations both informative and attentive to users. Our extensive subjective and objective evaluations on three large public data corpora demonstrate the effectiveness of our system to deliver knowledge…
National Taiwan University（国立台湾大学）；Google Research（谷歌研究院）；Microsoft Research（微软研究院）
Natural language understanding (NLU) is a core component of a spoken dialogue system. Recently recurrent neural networks (RNN) obtained strong results on NLU due to their superior ability of preserving sequential information over time. Traditionally, the NLU module tags semantic slots for utterances considering their flat structures, as the underlying RNN structure is a linear chain. However, natural language exhibits linguistic properties that provide rich, structured information for better understanding. This paper introduces a novel model, knowledge-guided structural attention networks (K-S…
We study learning of a matching model for response selection in retrieval-based dialogue systems. The problem is equally important with designing the architecture of a model, but is less explored in existing literature. To learn a robust matching model from noisy training data, we propose a general co-teaching framework with three specific teaching strategies that cover both teaching with loss functions and teaching with data curriculum. Under the framework, we simultaneously learn two matching models with independent training sets. In each iteration, one model transfers the knowledge learned …
Identifying breakdowns in ongoing dialogues helps to improve communication effectiveness. Most prior work on this topic relies on human annotated data and data augmentation to learn a classification model. While quality labeled dialogue data requires human annotation and is usually expensive to obtain, unlabeled data is easier to collect from various sources. In this paper, we propose a novel semi-supervised teacher-student learning framework to tackle this task. We introduce two teachers which are trained on labeled data and perturbed labeled data respectively. We leverage unlabeled data to i…
This paper presents a new adversarial learning method for generative conversational agents (GCA) besides a new model of GCA. Similar to previous works on adversarial learning for dialogue generation, our method assumes the GCA as a generator that aims at fooling a discriminator that labels dialogues as human-generated or machine-generated; however, in our approach, the discriminator performs token-level classification, i.e. it indicates whether the current token was generated by humans or machines. To do so, the discriminator also receives the context utterances (the dialogue history) and the …
DiaCDM: Cognitive Diagnosis in Teacher-Student Dialogues using the Initiation-Response-Evaluation Framework
Lab of Artificial Intelligence for Education, East China Normal University, China（人工智能教育实验室，华东师范大学，中国）；Shanghai Institute of Artificial Intelligence for Education, East China Normal University, China（人工智能教育上海研究院，华东师范大学，中国）；XR System Application Research Center, Nanchang Research Institute, Zhejiang University, China（XR系统应用研究中心，南昌研究所，浙江大学，中国）
Embodied dialogue instruction following requires an agent to complete a complex sequence of tasks from a natural language exchange. The recent introduction of benchmarks (Padmakumar et al., 2022) raises the question of how best to train and evaluate models for this multi-turn, multi-agent, long-horizon task. This paper contributes to that conversation, by arguing that imitation learning (IL) and related low-level metrics are actually misleading and do not align with the goals of embodied dialogue research and may hinder progress. We provide empirical comparisons of metrics, analysis of three m…
Tianjin University（天津大学）；China Mobile Communication Group Tianjin Co., Ltd.（中国移动通信集团天津有限公司）
Task-oriented dialog(TOD) aims to assist users in achieving specific goals through multi-turn conversation. Recently, good results have been obtained based on large pre-trained models. However, the labeled-data scarcity hinders the efficient development of TOD systems at scale. In this work, we constructed a weakly supervised dataset based on a teacher/student paradigm that leverages a large collection of unlabelled dialogues. Furthermore, we built a modular dialogue system and integrated coarse-to-fine grained classification for user intent detection. Experiments show that our method can reac…
School of software engineering, South China University of Technology（华南理工大学软件学院）
Dialogue policy learning based on reinforcement learning is difficult to be applied to real users to train dialogue agents from scratch because of the high cost. User simulators, which choose random user goals for the dialogue agent to train on, have been considered as an affordable substitute for real users. However, this random sampling method ignores the law of human learning, making the learned dialogue policy inefficient and unstable. We propose a novel framework, Automatic Curriculum Learning-based Deep Q-Network (ACL-DQN), which replaces the traditional random sampling method with a tea…
The BEA 2023 Shared Task on Generating AI Teacher Responses in Educational Dialogues
KU Leuven（鲁汶大学）；imec（比利时微电子研究中心）；MBZUAI（穆罕默德·本·扎耶德人工智能大学）；King’s College London（伦敦国王学院）；Universidad Central del Ecuador（厄瓜多尔中央大学）；Stanford University（斯坦福大学）
School of Computer Science, Beijing Institute of Technology（北京理工大学计算机学院）
With the development of deep learning, advanced dialogue generation methods usually require a greater amount of computational resources. One promising approach to obtaining a high-performance and lightweight model is knowledge distillation, which relies heavily on the pre-trained powerful teacher. Collaborative learning, also known as online knowledge distillation, is an effective way to conduct one-stage group distillation in the absence of a well-trained large teacher model. However, previous work has a severe branch homogeneity problem due to the same training objective and the independent …
NTT Media Intelligence Laboratories, NTT Corporation（NTT公司NTT媒体情报实验室）
This paper presents a novel knowledge distillation method for dialogue sequence labeling. Dialogue sequence labeling is a supervised learning task that estimates labels for each utterance in the target dialogue document, and is useful for many applications such as dialogue act estimation. Accurate labeling is often realized by a hierarchically-structured large model consisting of utterance-level and dialogue-level networks that capture the contexts within an utterance and between utterances, respectively. However, due to its large model size, such a model cannot be deployed on resource-constra…
PRIDE: Privileged Information-enhanced Distillation for Empathetic Dialogue Generation
Anhui Robot Technology Standard Innovation Base, School of Computer Science and Technology, University of Science and Technology of China（中国科学技术大学计算机科学与技术学院安徽机器人技术标准创新基地）
MathDial: A Dialogue Tutoring Dataset with Rich Pedagogical Properties Grounded in Math Reasoning Problems
ETH AI Center（苏黎世联邦理工学院人工智能中心）；ETH Zurich（苏黎世联邦理工学院）；TU Darmstadt（达姆施塔特工业大学）；Hessian Center for AI (hessian.AI)（黑森人工智能中心）；National Institute of Education, Nanyang Technological University（南洋理工大学国立教育学院）
Beijing Institute of Technology（北京理工大学）；Peking University（北京大学）
Neural dialogue models suffer from low-quality responses when interacted in practice, demonstrating difficulty in generalization beyond training data. Recently, knowledge distillation has been used to successfully regularize the student by transferring knowledge from the teacher. However, the teacher and the student are trained on the same dataset and tend to learn similar feature representations, whereas the most general knowledge should be found through differences. The finding of general knowledge is further hindered by the unidirectional distillation, as the student should obey the teacher…
How Real Is AI Tutoring? Comparing Simulated and Human Dialogues in One-on-One Instruction
Shanghai Institute of Artificial Intelligence for Education, East China Normal University（华东师范大学上海人工智能教育研究院）；Lab of Artificial Intelligence for Education, East China Normal University（华东师范大学人工智能教育实验室）
ACR: Adaptive Context Refactoring via Context Refactoring Operators for Multi-Turn Dialogue
Zhejiang Normal University（浙江师范大学）；Alibaba Group（阿里巴巴集团）；Hong Kong University of Science and Technology（香港科学与技术大学）；Southeast University（东南大学）；Soochow University（苏州大学）
Rensselaer Polytechnic Institute（伦斯勒理工学院）；IBM Thomas J Watson Research Center（IBM托马斯·J·沃森研究中心）
Recently, a more challenging state tracking task, Audio-Video Scene-Aware Dialogue (AVSD), is catching an increasing amount of attention among researchers. Different from purely text-based dialogue state tracking, the dialogue in AVSD contains a sequence of question-answer pairs about a video and the final answer to the given question requires additional understanding of the video. This paper interprets the AVSD task from an open-domain Question Answering (QA) point of view and proposes a multimodal open-domain QA system to deal with the problem. The proposed QA system uses common encoder-deco…
Transcripts of teaching episodes can be effective tools to understand discourse patterns in classroom instruction. According to most educational experts, sustained classroom discourse is a critical component of equitable, engaging, and rich learning environments for students. This paper describes the TalkMoves dataset, composed of 567 human-annotated K-12 mathematics lesson transcripts (including entire lessons or portions of lessons) derived from video recordings. The set of transcripts primarily includes in-person lessons with whole-class discussions and/or small group work, as well as some …
University of California, Davis（加利福尼亚大学戴维斯分校）；Google Inc.（谷歌公司）
Numerous new dialog domains are being created every day while collecting data for these domains is extremely costly since it involves human interactions. Therefore, it is essential to develop algorithms that can adapt to different domains efficiently when building data-driven dialog models. The most recent researches on domain adaption focus on giving the model a better initialization, rather than optimizing the adaptation process. We propose an efficient domain adaptive task-oriented dialog system model, which incorporates a meta-teacher model to emphasize the different impacts between genera…
Korbit Technologies Inc.（Korbit科技公司）；University of Cambridge（剑桥大学）；École de Technologie Supérieure（高等工程技术学院）；MILA (Quebec Artificial Intelligence Institute)（MILA（魁北克人工智能研究所））
We present Korbit, a large-scale, open-domain, mixed-interface, dialogue-based intelligent tutoring system (ITS). Korbit uses machine learning, natural language processing and reinforcement learning to provide interactive, personalized learning online. Korbit has been designed to easily scale to thousands of subjects, by automating, standardizing and simplifying the content creation process. Unlike other ITS, a teacher can develop new learning modules for Korbit in a matter of hours. To facilitate learning across a widerange of STEM subjects, Korbit uses a mixed-interface, which includes video…
Instructional Goal-Aligned Question Generation for Student Evaluation in Virtual Lab Settings: How Closely Do LLMs Actually Align?
Department of EdPsych, University of Wisconsin-Madison（威斯康星大学麦迪逊分校教育心理学系）；Department of CS, Wichita State University（威斯康星州立大学Wichita分校计算机科学系）；Department of CSSE, Auburn University（阿伯茨罕大学计算机科学与工程系）
Massachusetts Institute of Technology（麻省理工学院）；MIT Media Lab（麻省理工学院媒体实验室）
As artificial intelligence (AI) becomes a prominent part of modern life, AI literacy is becoming important for all citizens, not just those in technology careers. Previous research in AI education materials has largely focused on the introduction of terminology as well as AI use cases and ethics, but few allow students to learn by creating their own machine learning models. Therefore, there is a need for enriching AI educational tools with more adaptable and flexible platforms for interested educators with any level of technical experience to utilize within their teaching material. As such, we…
