Voice2Series: Reprogramming Acoustic Models for Time Series Classification
Georgia Institute of Technology（佐治亚理工学院）；Columbia University（哥伦比亚大学）；IBM Research（IBM研究院）
Chao-Han Huck Yang, Yun-Yun Tsai, Pin-Yu Chen
备注：
Updated version with a correction. The full draft was submitted in Jan 2021. The Voice2Series project initially was launched in Sep 2020. Accepted to ICML 2021, 16 Pages
Learning to classify time series with limited data is a practical yet challenging problem. Current methods are primarily based on hand-designed feature extraction rules or domain-specific data augmentation. Motivated by the advances in deep speech processing models and the fact that voice data are univariate temporal signals, in this paper, we propose Voice2Series (V2S), a novel end-to-end approach that reprograms acoustic models for time series classification, through input transformation learning and output label mapping. Leveraging the representation learning power of a large-scale pre-trained speech processing model, on 30 different time series tasks we show that V2S performs competitive results on 19 time series classification tasks. We further provide a theoretical justification of V2S by proving its population risk is upper bounded by the source risk and a Wasserstein distance accounting for feature alignment via reprogramming. Our results offer new and effective means to time series classification.
Reprogramming Foundational Large Language Models(LLMs) for Enterprise Adoption for Spatio-Temporal Forecasting Applications: Unveiling a New Era in Copilot-Guided Cross-Modal Time Series Representation Learning
Time-LLM: Time Series Forecasting by Reprogramming Large Language Models
Monash University（莫纳什大学）；Ant Group（蚂蚁集团）；IBM Research（IBM研究院）；Griffith University（格里菲斯大学）；Alibaba Group（阿里巴巴集团）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））
RePST: Language Model Empowered Spatio-Temporal Forecasting via Semantic-Oriented Reprogramming
The Hong Kong University of Science and Technology (Guangzhou)（香港科学与技术大学（广州））；The Hong Kong University of Science and Technology（香港科学与技术大学）；Shandong University（山东大学）；Oxford University（牛津大学）；Beihang University（北航大学）
Multilingual and crosslingual speech recognition using phonological-vector based phone embeddings
Tsinghua University（清华大学）；Beijing National Research Center for Information Science and Technology（北京信息科学与技术国家研究中心）
语音特征（PF）的使用潜在地允许特定语言的音素在训练中保持链接，这对于用于低资源语言的多语言和跨语言语音识别方法的信息共享是非常期望的。以前的方法在使用语音特征时的缺点是，以自底向上的方式提取声学到PF本身是困难的。在本文中，我们建议将音位驱动的音素嵌入（自上而下）和基于深度神经网络（DNN）的声学特征提取（自下而上）结合起来，以计算音素概率。这种新方法被称为JoinAP（Joining of Acoustics and Phonology）。值得注意的是，语音识别不需要从声学特征到音位特征的反演。对于IPA（International Phonetic Alphabet）表中的每个音素，我们将其音素特征编码为一个音素向量，然后对该音素向量进行线性或非线性变换，得到音素嵌入。在CommonVoice数据集（德语、法语、西班牙语和意大利语）和AISHLL-1数据集（普通话）上进行了一系列多语种和跨语种（包括zero-shot和Few-Shot）语音识别实验，证明了非线性音素嵌入的JoinAP方法优于线性音素嵌入的JoinAP方法和传统的平面音素嵌入方法。
