Diffusion-based text-to-audio (TTA) generation has made substantial progress, leveraging latent diffusion model (LDM) to produce high-quality, diverse and instruction-relevant audios. However, beyond generation, the task of audio editing remains equally important but has received comparatively little attention. Audio editing tasks face two primary challenges: executing precise edits and preserving the unedited sections. While workflows based on LDMs have effectively addressed these challenges in the field of image processing, similar approaches have been scarcely applied to audio editing. In this paper, we introduce AudioEditor, a training-free audio editing framework built on the pretrained diffusion-based TTA model. AudioEditor incorporates Null-text Inversion and EOT-suppression methods, enabling the model to preserve original audio features while executing accurate edits. Comprehensive objective and subjective experiments validate the effectiveness of AudioEditor in delivering high-quality audio edits. Code and demo can be found at https://github.com/NKU-HLT/AudioEditor.
Unified Audio Generation and Editing via Joint Condition Modeling and Progressive Training
Department of Electronic Engineering and Information Science, University of Science and Technology of China(电子工程与信息科学系,中国科学技术大学);Tianjin Key Laboratory of Cognitive Computing and Application, School of Artificial Intelligence, Tianjin University(认知计算与应用重点实验室,天津大学人工智能学院);Institute of Artificial Intelligence of China Telecom (TeleAI)(中国电信人工智能研究院(TeleAI))
Fondazione Bruno Kessler(布鲁诺·凯塞勒基金会);Mila-Québec AI Institute(米拉-魁北克人工智能研究所);Concordia University(康考迪亚大学);University of Illinois at Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Laval University(拉瓦尔大学)
RFM-Editing: Rectified Flow Matching for Text-guided Audio Editing
Centre for Vision, Speech and Signal Processing (CVSSP), University of Surrey, United Kingdom(视觉、语音和信号处理中心(CVSSP),英国萨里大学);College of Information and Electrical Engineering, China Agricultural University, China(信息与电气工程学院,中国农业大学);Fisheries College, Ocean University of China, China(海洋大学渔业学院,中国)
UNISON: A Unified Sound Generation and Editing Framework via Deep LLM Fusion
The Chinese University of Hong Kong(香港中文大学);The Hong Kong Polytechnic University(香港理工大学);City University of Hong Kong(香港城市大学);The Hong Kong University of Science and Technology(香港科学与技术大学);Tsinghua University(清华大学);Huawei Research Hong Kong(华为香港研究)
Arrange, Inpaint, and Refine: Steerable Long-term Music Audio Generation and Editing via Content-based Controls
New York University Shanghai(纽约大学上海分校);Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学);Queen Mary University of London(伦敦大学玛丽皇后学院)
Joint and Cross-Modal Video-Audio Generation and Editing: A Unified Formulation and Design Taxonomy
University of Massachusetts Amherst(马萨诸塞大学阿默斯特分校);University of Texas at Dallas(德克萨斯大学达拉斯分校);Virginia Tech(弗吉尼亚理工大学);Texas A&M University(德克萨斯A&M大学);University of Southern California(南加州大学);Vanderbilt University(范德堡大学);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Adobe Research(Adobe研究院);Arizona State University(亚利桑那州立大学);University of Oregon(俄勒冈大学);Stanford University(斯坦福大学);Dolby Laboratories(杜比实验室);Cisco(思科)
UniCATS: A Unified Context-Aware Text-to-Speech Framework with Contextual VQ-Diffusion and Vocoding
MoE Key Lab of Artificial Intelligence, AI Institute X-LANCE Lab, Department of Computer Science and Engineering, Shanghai Jiao Tong University(上海交通大学电子信息与电气工程学院计算机科学与工程系人工智能教育部重点实验室X-LANCE实验室);Shenzhen Research Institute of Big Data(深圳市大数据研究院);AISpeech Ltd(思必驰科技股份有限公司)
SALM: Spatial Audio Language Model with Structured Embeddings for Understanding and Editing
Institute of Acoustics, Chinese Academy of Sciences, China(中国科学院声学研究所);MiLM Plus, Xiaomi Inc., China(小米公司);Xi’an Jiaotong Liverpool University, China(西安交通大学利物浦大学);University of Chinese Academy of Sciences, China(中国科学院大学)
Cross-Utterance Conditioned VAE for Speech Generation
the University of Manchester(曼彻斯特大学);ShanghaiTech University(上海科技大学);University of Cambridge(剑桥大学);Shanghai Jiao Tong University(上海交通大学);Tsinghua University(清华大学);University College London(伦敦大学学院);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));University of Shanghai for Science and Technology(上海理工大学)
MeloDISinger: Melody-Aware & Duration-Preserving Singing Voice Editing with Audio Infilling
Graduate School of Artificial Intelligence, KAIST, South Korea(韩国釜山科学技术院人工智能研究生院);Graduate School of Culture Technology, KAIST, South Korea(韩国釜山科学技术院文化科技研究生院)
Towards Automatic Evaluation and High-Quality Pseudo-Parallel Dataset Construction for Audio Editing: A Human-in-the-Loop Method
College of Computer Science, Nankai University(南开大学计算机科学学院);College of Electronic Information and Optical Engineering, Nankai University(南开大学电子信息与光工程学院)
Object-AVEdit: An Object-level Audio-Visual Editing Model
Institute of Artificial Intelligence (TeleAI)(人工智能研究院);China Telecom(中国电信);Gaoling School of Artificial Intelligence(东城区人工智能学院);Renmin University of China(中国人民大学);Beijing University of Posts and Telecommunications(北京邮电大学);Tencent Data Platform(腾讯数据平台);Tsinghua University(清华大学);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);Huawei Noah’s Ark Lab(华为诺亚实验室);The University of Hong Kong(香港大学)
EditGen: Harnessing Cross-Attention Control for Instruction-Based Auto-Regressive Audio Editing
Department of Informatics and Telecommunications, UoA NCSR "Demokritos"(信息与电信系,雅典国家科研中心"Demokritos");School of Electrical and Computer Engineering, NTUA(电气与计算机工程学院,国家技术研究院)
DirectAudioEdit: Inversion-Free Text-Guided Audio Editing via Diffusion Prediction Contrast
School of Computer Science and Engineering, Northeastern University, Shenyang, China(东北大学计算机科学与工程学院);Kunming University of Science and Technology(昆明理工大学);NiuTrans Research, Shenyang, China(新译研究)
RFM-Editing 2: Text-Guided Audio Editing with Rectified Flow Matching and Coarse-to-Fine Diffusion Transformers
Centre for Vision, Speech and Signal Processing (CVSSP), University of Surrey(萨里大学视觉、语音与信号处理中心);School of Artificial Intelligence, Beijing University of Posts and Telecommunications(北京邮电大学人工智能学院);Fisheries College, Ocean University of China(中国海洋大学水产学院);College of Information and Electrical Engineering, China Agricultural University(中国农业大学信息与电气工程学院)