首页/论文/2011.04512

arXiv 2011.045122020-11cs.CLcs.LG

Auxiliary Sequence Labeling Tasks for Disfluency Detection

Kakao Corp.（Kakao 公司）；Kakao Enterprise（Kakao 企业）；Wisenut Inc.（Wisenut 公司）；Hanshin University（韩神大学）

Dongyub Lee, Byeongil Ko, Myeong Cheol Shin, Taesun Whang, Daniel Lee, Eun Hwa Kim, EungGyun Kim, Jaechoon Jo

备注：: Submitted to INTERSPEECH 2021

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

英文摘要相关论文

英文摘要：

Detecting disfluencies in spontaneous speech is an important preprocessing step in natural language processing and speech recognition applications. Existing works for disfluency detection have focused on designing a single objective only for disfluency detection, while auxiliary objectives utilizing linguistic information of a word such as named entity or part-of-speech information can be effective. In this paper, we focus on detecting disfluencies on spoken transcripts and propose a method utilizing named entity recognition (NER) and part-of-speech (POS) as auxiliary sequence labeling (SL) tasks for disfluency detection. First, we investigate cases that utilizing linguistic information of a word can prevent mispredicting important words and can be helpful for the correct detection of disfluencies. Second, we show that training a disfluency detection model with auxiliary SL tasks can improve its F-score in disfluency detection. Then, we analyze which auxiliary SL tasks are influential depending on baseline models. Experimental results on the widely used English Switchboard dataset show that our method outperforms the previous state-of-the-art in disfluency detection.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

阅读原文

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

本文内容

英文摘要相关论文

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

继续探索

相关论文

Multi-Task Self-Supervised Learning for Disfluency Detection

Harbin Institute of Technology（哈尔滨工业大学）；University of Oxford（牛津大学）；Beijing University of Posts and Telecommunications（北京邮电大学）；University of California, Santa Barbara（加州大学圣塔芭芭拉分校）

Most existing approaches to disfluency detection heavily rely on human-annotated data, which is expensive to obtain in practice. To tackle the training data bottleneck, we investigate methods for combining multiple self-supervised tasks-i.e., supervised tasks where data can be collected without manual labeling. First, we construct large-scale pseudo training data by randomly adding or deleting words from unlabeled news data, and propose two self-supervised pre-training tasks: (i) tagging task to detect the added noisy words. (ii) sentence classification to distinguish original sentences from g…

2020-04-09cs.CL

Handling Noisy Labels for Robustly Learning from Self-Training Data for Low-Resource Sequence Labeling

Heidelberg University（海德堡大学）；Aalto University（阿尔托大学）；Saarland University（萨尔兰大学）

In this paper, we address the problem of effectively self-training neural networks in a low-resource setting. Self-training is frequently used to automatically increase the amount of training data. However, in a low-resource scenario, it is less effective due to unreliable annotations created using self-labeling of unlabeled data. We propose to combine self-training with noise handling on the self-labeled data. Directly estimating noise on the combined clean training set and self-labeled data can lead to corruption of the clean data and hence, performs worse. Thus, we propose the Clean and Noi…

2019-03-28cs.CLcs.LG

Improving Long-Tail Relation Extraction with Collaborating Relation-Augmented Attention

University of Technology Sydney（悉尼科技大学）；Australian AI Institute（澳大利亚人工智能研究所）；University of Washington（华盛顿大学）

Wrong labeling problem and long-tail relations are two main challenges caused by distant supervision in relation extraction. Recent works alleviate the wrong labeling by selective attention via multi-instance learning, but cannot well handle long-tail relations even if hierarchies of the relations are introduced to share knowledge. In this work, we propose a novel neural network, Collaborating Relation-augmented Attention (CoRA), to handle both the wrong labeling and long-tail relations. Particularly, we first propose relation-augmented attention network as base model. It operates on sentence …

2020-11-02cs.CL

Sequence Labeling Parsing by Learning Across Representations

Universidade da Coruña（拉科鲁尼亚大学）；CITIC

We use parsing as sequence labeling as a common framework to learn across constituency and dependency syntactic abstractions. To do so, we cast the problem as multitask learning (MTL). First, we show that adding a parsing paradigm as an auxiliary loss consistently improves the performance on the other paradigm. Secondly, we explore an MTL sequence labeling model that parses both representations, at almost no cost in terms of performance and speed. The results across the board show that on average MTL models with auxiliary losses for constituency parsing outperform single-task ones by 1.14 F1 p…

2020-01-07cs.CLcs.LG

Viable Dependency Parsing as Sequence Labeling

Universidade da Coruña（拉科鲁尼亚大学）；CITIC（CITIC研究所）；FASTPARSE Lab（FASTPARSE实验室）；LyS Research Group（LyS研究组）；Departamento de Computación（计算机系）

We recast dependency parsing as a sequence labeling problem, exploring several encodings of dependency trees as labels. While dependency parsing by means of sequence labeling had been attempted in existing work, results suggested that the technique was impractical. We show instead that with a conventional BiLSTM-based model it is possible to obtain fast and accurate parsers. These parsers are conceptually simple, not needing traditional parsing algorithms or auxiliary structures. However, experiments on the PTB and a sample of UD treebanks show that they provide a good speed-accuracy tradeoff,…

2019-03-29cs.CLcs.LG

Auxiliary Objectives for Neural Error Detection Models

University of Cambridge（剑桥大学）

We investigate the utility of different auxiliary objectives and training strategies within a neural sequence labeling approach to error detection in learner writing. Auxiliary costs provide the model with additional linguistic information, allowing it to learn general-purpose compositional features that can then be exploited for other objectives. Our experiments show that a joint learning approach trained with parallel labels on in-domain data improves performance over the previous best error detection system. While the resulting model has the same number of parameters, the additional objecti…

2017-07-17cs.CLcs.LGcs.NE

更多相关论文收起更多论文

Deep Semi-Supervised Learning with Linguistically Motivated Sequence Labeling Task Hierarchies

University College London（伦敦大学学院）

In this paper we present a novel Neural Network algorithm for conducting semi-supervised learning for sequence labeling tasks arranged in a linguistically motivated hierarchy. This relationship is exploited to regularise the representations of supervised tasks by backpropagating the error of the unsupervised task through the supervised tasks. We introduce a neural network where lower layers are supervised by junior downstream tasks and the final layer task is an auxiliary unsupervised task. The architecture shows improvements of up to two percentage points F1 for Chunking compared to a plausib…

2016-12-29cs.CL

GIRNet: Interleaved Multi-Task Recurrent State Sequence Models

IIIT Delhi（德里英迪拉·甘地信息技术学院）；IIT Bombay（印度理工学院孟买分校）

In several natural language tasks, labeled sequences are available in separate domains (say, languages), but the goal is to label sequences with mixed domain (such as code-switched text). Or, we may have available models for labeling whole passages (say, with sentiments), which we would like to exploit toward better position-specific label inference (say, target-dependent sentiment annotation). A key characteristic shared across such tasks is that different positions in a primary instance can benefit from different `experts' trained from auxiliary data, but labeled primary instances are scarce…

2018-12-25cs.CLcs.LGstat.ML

An Effective System for Multi-format Information Extraction

Northeastern University（东北大学）

The multi-format information extraction task in the 2021 Language and Intelligence Challenge is designed to comprehensively evaluate information extraction from different dimensions. It consists of an multiple slots relation extraction subtask and two event extraction subtasks that extract events from both sentence-level and document-level. Here we describe our system for this multi-format information extraction competition task. Specifically, for the relation extraction subtask, we convert it to a traditional triple extraction task and design a voting based method that makes full use of exist…

2021-08-16cs.AIcs.CL

Exploiting Global Contextual Information for Document-level Named Entity Recognition

Huazhong University of Science and Technology（华中科技大学）；Beijing Institute of Technology（北京理工大学）；Inception Institute of Artificial Intelligence（起源人工智能研究院）；Naval University of Engineering（海军工程大学）

Most existing named entity recognition (NER) approaches are based on sequence labeling models, which focus on capturing the local context dependencies. However, the way of taking one sentence as input prevents the modeling of non-sequential global context, which is useful especially when local context information is limited or ambiguous. To this end, we propose a model called Global Context enhanced Document-level NER (GCDoc) to leverage global contextual information from two levels, i.e., both word and sentence. At word-level, a document graph is constructed to model a wider range of dependen…

2021-06-02cs.AIcs.CL

When is multitask learning effective? Semantic sequence prediction under varying data conditions

University of Groningen（格罗宁根大学）；Univ. Paris Diderot（巴黎狄德罗大学）；INRIA（法国国家信息与自动化研究所）

Multitask learning has been applied successfully to a range of tasks, mostly morphosyntactic. However, little is known on when MTL works and whether there are data characteristics that help to determine its success. In this paper we evaluate a range of semantic sequence labeling tasks in a MTL setup. We examine different auxiliary tasks, amongst which a novel setup, and correlate their impact to data-dependent conditions. Our results show that MTL is not always effective, significant improvements are obtained only for 1 out of 5 tasks. When successful, auxiliary tasks with compact and more uni…

2017-01-10cs.CL

Improving Distinction between ASR Errors and Speech Disfluencies with Feature Space Interpolation

ActionPower

Fine-tuning pretrained language models (LMs) is a popular approach to automatic speech recognition (ASR) error detection during post-processing. While error detection systems often take advantage of statistical language archetypes captured by LMs, at times the pretrained knowledge can hinder error detection performance. For instance, presence of speech disfluencies might confuse the post-processing system into tagging disfluent but accurate transcriptions as ASR errors. Such confusion occurs because both error detection and disfluency detection tasks attempt to identify tokens at statistically…

2021-08-04cs.CLcs.SDeess.AS

MLR: A Two-stage Conversational Query Rewriting Model with Multi-task Learning

Alibaba Group（阿里巴巴集团）

Conversational context understanding aims to recognize the real intention of user from the conversation history, which is critical for building the dialogue system. However, the multi-turn conversation understanding in open domain is still quite challenging, which requires the system extracting the important information and resolving the dependencies in contexts among a variety of open topics. In this paper, we propose the conversational query rewriting model - MLR, which is a Multi-task model on sequence Labeling and query Rewriting. MLR reformulates the multi-turn conversational queries into…

2020-04-13cs.AIcs.CLcs.LG