The audio research community depends on open generative models as foundational tools for building novel approaches and establishing baselines. In this report, we present Woosh, Sony AI's publicly released sound effect foundation model, detailing its architecture, training process, and an evaluation against other popular open models. Being optimized for sound effects, we provide (1) a high-quality audio encoder/decoder model and (2) a text-audio alignment model for conditioning, together with (3) text-to-audio and (4) video-to-audio generative models. Distilled text-to-audio and video-to-audio models are also included in the release, allowing for low-resource operation and fast inference. Our evaluation on both public and private data shows competitive or better performance for each module when compared to existing open alternatives like StableAudio-Open and TangoFlux. Inference code and model weights are available at https://github.com/SonyResearch/Woosh. Demo samples can be found at https://sonyresearch.github.io/Woosh/.
Can Foundation Models Hear What Made That Sound? A Tiered Benchmark of Audio-Language Models and Traditional Classifiers for Closed-Set Sound Source Identification
Patient-level validation of a foundation-model pipeline for pediatric lung sounds: physician-annotated adventitious events are recognized, disease groups are not reliably predicted
Department of Pediatric Intensive Care Medicine, Life Support Center, Hacettepe University(儿科重症医学科、生命支持中心、哈切塔佩大学);Department of Pediatric Emergency Medicine, Life Support Center, Hacettepe University Faculty of Medicine(儿科急诊医学科、生命支持中心、哈切塔佩大学医学院)
A Generalist Audio Foundation Model for Comprehensive Body Sound Auscultation
Shanghai Jiao Tong University(上海交通大学);Xinhua Hospital Affiliated To Shanghai Jiao Tong University School of Medicine(上海交通大学医学院附属新华医院);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);Tongji University(同济大学)
CoughPhase-CLR: Designing an acoustics-informed foundation model for coughing sound classification
Chair of Health Informatics at the TUM University Hospital(慕尼黑工业大学大学医院健康信息学教席);Munich Center for Machine Learning(慕尼黑机器学习中心);Munich Data Science Institute(慕尼黑数据科学研究所);University Hospital Augsburg at the University of Augsburg(奥格斯堡大学奥格斯堡大学医院);Medical Faculty, Ludwig Maximilians University of Munich(慕尼黑路德维希-马克西米利安大学医学院);Group on Language, Audio, & Music at Imperial College London(帝国理工学院语言、音频与音乐组);Technical University of Munich(慕尼黑工业大学)
Pediatric Asthma Detection with Googles HeAR Model: An AI-Driven Respiratory Sound Classifier
Kent State University(肯特州立大学);Northeastern University(东北大学);Department of Computer Science, Cleveland State University(计算机科学系,克利夫兰州立大学);Khoury College of Computer Science, Roux Institute at Northeastern University(计算机科学学院,东北大学罗克斯研究所)
Towards the Vision-Sound-Language-Action Paradigm: The HEAR Framework for Sound-Centric Manipulation
School of Automation and Intelligent Sensing, Shanghai Jiao Tong University and Shanghai Key Laboratory of Navigation and Location Based Services(自动化与智能感知学院,上海交通大学,导航与基于位置的服务重点实验室);Department of Engineering, Cambridge University(工程系,剑桥大学)
Leveraging tropical reef, bird and unrelated sounds for superior transfer learning in marine bioacoustics
University College London(伦敦大学学院);Google DeepMind(谷歌DeepMind);Zoological Society of London(伦敦动物学会);Conservation Metrics Inc.(保护指标公司);University of New Hampshire(新罕布什尔大学);K. Lisa Yang Center for Conservation Bioacoustics, Cornell Lab of Ornithology, Cornell University(康奈尔大学鸟类学实验室K. Lisa Yang保护生物声学中心);Department of Public and Ecosystem Health, Cornell University(康奈尔大学公共与生态系统健康系);Sound Ocean Science(海洋声学科学);University of Exeter(埃克塞特大学);University of Bristol(布里斯托大学);IPB University(IPB大学);Google Research(谷歌研究院)
ChildVox: A Speech, Audio, and Large Audio-Language Model Benchmark in Understanding and Characterizing Sound across Childhood
University of Southern California(南加州大学);The Ohio State University(俄亥俄州立大学);University of California, Los Angeles(加州大学洛杉矶分校);Harvard University(哈佛大学);Boston University(波士顿大学);University of Miami(迈阿密大学)
SS-DPPN: A self-supervised dual-path foundation model for the generalizable cardiac audio representation
Department of Computer Science and Engineering(计算机科学与工程系);BRAC University(布拉克大学);Department of Electricial and Electronic Engineering(电气与电子工程系);Department of Biomedical Engineering(生物医学工程系);Linköping University(林肯堡大学);Department of Electrical and Computer Engineering(电气与计算机工程系);University of Maryland(马里兰大学);School of Electrical Engineering, Computing and Mathematical Sciences(电气工程、计算与数学科学学院)
A Multimodal Symphony: Integrating Taste and Sound through Generative AI
Centro di Sonologia Computazionale (CSC)(计算声学中心);Department of Information Engineering University of Padova(信息工程系帕多瓦大学);Center for Mind/Brain Sciences (CIMeC)(心智/大脑科学中心);University of Trento(特伦托大学);SoundFood s.r.l.(SoundFood公司)
Audio-FLAN: An Instruction-Following Dataset for Unified Audio Understanding and Generation of Speech, Music, and Sound
The Hong Kong University of Science(香港科学与技术大学);Inner Mongolia University(内蒙古大学);Beihang University(北京航空航天大学);Queen Mary University of London(伦敦玛丽女王大学);The Chinese University of Hong Kong(香港中文大学);National University of Singapore(新加坡国立大学);University of Surrey(萨里大学);University of Rochester(罗切斯特大学);Independent Researcher(独立研究者)
GRAM: Spatial general-purpose audio representation models for real-world applications
Donders Institute, Radboud University, Nijmegen, The Netherlands(多纳茨研究所,拉德堡德大学,尼姆韦根,荷兰);Mortimer B Zuckerman Institute, Columbia University, New York, United States(莫蒂默·B·祖克erman研究所,哥伦比亚大学,纽约,美国)