Human speech data comprises a rich set of domain factors such as accent, syntactic and semantic variety, or acoustic environment. Previous work explores the effect of domain mismatch in automatic speech recognition between pre-training and fine-tuning as a whole but does not dissect the contribution of individual factors. In this paper, we present a controlled study to better understand the effect of such factors on the performance of pre-trained representations on automatic speech recognition. To do so, we pre-train models either on modified natural speech or synthesized audio, with a single domain factor modified, and then measure performance after fine-tuning. Results show that phonetic domain factors play an important role during pre-training while grammatical and syntactic factors are far less important. To our knowledge, this is the first study to better understand the domain characteristics of pre-trained sets in self-supervised pre-training for speech.
Prompting is not enough: supervised baselines and leakage control for measuring shared decision-making with LLMs in pediatric encounters
University of Utah(犹他大学);University of Utah Spencer Fox Eccles School of Medicine(犹他大学斯宾塞·福克斯·埃克尔斯医学院);Kahlert School of Computing, University of Utah(犹他大学卡勒特计算机学院);Huntsman Cancer Institute, University of Utah(犹他大学亨茨曼癌症研究所)
Measuring a hate speech spectrum with faceted Rasch item response theory and perspective-aware, explainable-by-design deep learning
Center for Precision Psychiatry, Mass General Hospital Department of Psychiatry, Harvard Medical School(精准精神病学中心,麻省总医院精神病科,哈佛医学院);D-Lab University of California, Berkeley(加州大学伯克利分校D实验室)
Towards Early Prediction of Self-Supervised Speech Model Performance
Avignon UniversitéFrance(阿维尼翁大学);Thales SIXFrance(泰勒斯集团);Samsung AI Center CambridgeUnited Kingdom(三星人工智能中心);Université Grenoble AlpesFrance(格勒诺布尔阿尔卑斯大学);Laboratoire Informatique d’Avignon(阿维尼翁计算机实验室);Laboratoire d’Informatique de Grenoble(格勒诺布尔计算机实验室)
Self-Supervised Speech Representations are More Phonetic than Semantic
Carnegie Mellon University(卡内基梅隆大学);Toyota Technological Institute at Chicago(芝加哥丰田理工学院);National Institute of Advanced Industrial Science and Technology (AIST)(日本国立先进工业科学技术研究院(AIST))
Measuring Fine-Grained Domain Relevance of Terms: A Hierarchical Core-Fringe Approach
University of Illinois at Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);IBM Thomas J. Watson Research Center(IBM托马斯·J·沃森研究中心);IBM-Illinois Center for Cognitive Computing Systems Research(IBM-伊利诺伊认知计算系统研究中心)
Measuring and Improving Attentiveness to Partial Inputs with Counterfactuals
Allen Institute for AI(艾伦人工智能研究所);University of Washington(华盛顿大学);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);University of Utah(犹他大学);University of California, Irvine(加利福尼亚大学欧文分校)