Modern speech understanding (SU) runs a sophisticated pipeline: ingesting streaming voice input, the pipeline executes encoder-decoder based deep neural networks repeatedly; by doing so, the pipeline generates tentative outputs (called hypotheses), and periodically scores the hypotheses. This paper sets to accelerate SU on resource-constrained edge devices. It takes a hybrid approach: to speed up on-device execution; to offload inputs that are beyond the device's capacity. While the approach is well-known, we address SU's unique challenges with novel techniques: (1) late contextualization, which executes a model's attentive encoder in parallel to the input ingestion; (2) pilot inference, which mitigates the SU pipeline's temporal load imbalance; (3) autoregression offramps, which evaluate offloading decisions based on pilot inferences and hypotheses. Our techniques are compatible with existing speech models, pipelines, and frameworks; they can be applied independently or in combination. Our prototype, called PASU, is tested on Arm platforms with 6 - 8 cores: it delivers SOTA accuracy; it reduces the end-to-end latency by 2x and reduces the offloading needs by 2x.
Speech and Natural Language Processing Technologies for Pseudo-Pilot Simulator
Idiap Research Institute(Idiap 研究所);Brno University of Technology(布尔诺理工大学);Ecole Polytechnique Federale de Lausanne(洛桑联邦理工学院);University of Zurich(苏黎世大学)
Grammar Based Speaker Role Identification for Air Traffic Control Speech Recognition
Idiap Research Institute(Idiap研究所);Brno University of Technology(布尔诺理工大学);Ecole Polytechnique Federale de Lausanne (EPFL)(洛桑联邦理工学院);University of Zurich(苏黎世大学);German Aerospace Center (DLR)(德国航空航天中心)
ATCSpeech: a multilingual pilot-controller speech corpus from real Air Traffic Control environment
National Key Laboratory of Air Traffic Control Automation System Technology, Sichuan University(四川大学);Southwest Air Traffic Management Bureau, Civil Aviation Administration of China (CAAC)(中国民用航空局西南空中交通管理局);Wisesoft Co., Ltd.(四川川大智胜软件股份有限公司)
Gipsa-lab, University Grenoble Alpes(格勒诺布尔阿尔卑斯大学Gipsa-lab);Shenzhen Research Institute of Big Data, The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳)深圳大数据研究院);Grenoble Institute of Technology(格勒诺布尔理工学院)
Assessing clinical utility of Machine Learning and Artificial Intelligence approaches to analyze speech recordings in Multiple Sclerosis: A Pilot Study
Charles University(查理大学);Czech Technical University in Prague(布拉格捷克理工大学);General University Hospital(综合大学医院);Brigham and Women’s Hospital(布莱根妇女医院);Harvard Medical School(哈佛医学院);Johns Hopkins University(约翰斯·霍普金斯大学)
Combining Textual and Spectral Features for Robust Classification of Pilot Communications
1 Department of Computer Science, University of Nebraska Omaha, Omaha, NE, USA 2 Aviation Institute, University of Nebraska Omaha, Omaha, NE, USA 3 Durham School of Architectural Engineering \& Construction, University of Nebraska Lincoln, Lincoln, NE, USA 4 School of Graduate Studies, Embry-Riddle Aeronautical University Daytona Beach, Daytona Beach, FL, USA
ATCO2 corpus: A Large-Scale Dataset for Research on Automatic Speech Recognition and Natural Language Understanding of Air Traffic Control Communications
Idiap Research Institute(Idiap 研究所);Ecole Polytechnique Federale de Lausanne (EPFL)(洛桑联邦理工学院);Brno University of Technology(布尔诺理工大学);ReplayWell(ReplayWell公司);Saarland University(萨尔大学);Evaluations and Language Resources Distribution Agency (ELDA)(评估与语言资源分发局);Romagna Tech(罗马涅科技公司);Honeywell(霍尼韦尔);OpenSky Network(开放天空网络)
Lessons Learned in ATCO2: 5000 hours of Air Traffic Control Communications for Robust Automatic Speech Recognition and Understanding
Idiap Research Institute(伊迪亚普研究所);University of Zurich(苏黎世大学);Brno University of Technology(布尔诺理工大学);OpenSky Network(开放天空网络);Evaluations and Language Resources Distribution Agency (ELDA)(评估与语言资源分发局(ELDA))