本文分析 MixCheck studio 平台中 30 种流派的混音与母带音频指标，发现响度和动态问题最常见，而母带在压缩、立体声场及相位方面表现更优。
摘要：
我们对一个音频指标数据集进行了分析，该数据集还包含由网络平台 MixCheck studio 提供的关于混音和母带的审美考量。该平台专为教育目的设计，主要面向业余音乐制作人，旨在在他们的录音发布之前对其进行分析。分析聚焦于以下数据点：30 种用户指定音乐流派中的积分响度、单声道兼容性、削波和相位问题的存在情况、压缩以及音调特征。分析同时涵盖混音音频（mixes）和母带音频（masters），其中混音是指各独立音轨的初步组合与平衡，母带则是指为发行而优化的最终精炼版本。结果表明，与响度相关的问题以及动态问题最为普遍，尤其在母带音频中。然而，母带音频在压缩方面比仅经过混音的音频表现出更好的结果。此外，结果显示母带音频在立体声场和相位问题上的百分比较低。
英文摘要：
We present an analysis of a dataset of audio metrics and aesthetic considerations about mixes and masters provided by the web platform MixCheck studio. The platform is designed for educational purposes, primarily targeting amateur music producers, and aimed at analysing their recordings prior to them being released. The analysis focuses on the following data points: integrated loudness, mono compatibility, presence of clipping and phase issues, compression and tonal profile across 30 user-specified genres. Both mixed (mixes) and mastered audio (masters) are included in the analysis, where mixes refer to the initial combination and balance of individual tracks, and masters refer to the final refined version optimized for distribution. Results show that loudness-related issues along with dynamics issues are the most prevalent, particularly in mastered audio. However mastered audio presents better results in compression than just mixed audio. Additionally, results show that mastered audio has a lower percentage of stereo field and phase issues.
We introduce an approach to convert mono audio recorded by a 360 video camera into spatial audio, a representation of the distribution of sound over the full viewing sphere. Spatial audio is an important component of immersive 360 video viewing, but spatial audio microphones are still rare in current 360 video production. Our system consists of end-to-end trainable neural networks that separate individual sound sources and localize them on the viewing sphere, conditioned on multi-modal analysis of audio and 360 video frames. We introduce several datasets, including one filmed ourselves, and on…
We tackle the problem of generating audio samples conditioned on descriptive text captions. In this work, we propose AaudioGen, an auto-regressive generative model that generates audio samples conditioned on text inputs. AudioGen operates on a learnt discrete audio representation. The task of text-to-audio generation poses multiple challenges. Due to the way audio travels through a medium, differentiating ``objects'' can be a difficult task (e.g., separating multiple people simultaneously speaking). This is further complicated by real-world recording conditions (e.g., background noise, reverbe…
TeleAntiFraud 2.0: A Refreshable, Profile-Grounded, and Audio-Based Benchmark for Telecom Fraud Detection
JD Technology（京东科技）；Chongqing Ant Consumer Finance Co., Ltd., Ant Group（重庆蚂蚁消费金融有限公司，蚂蚁集团）；Meta；University of Science and Technology of China（中国科学技术大学）；Northeastern University（东北大学）
This computer science master thesis aims at modelling the nonlinearities of a loudspeaker. A piecewise linear approximation is initially explored and then we present a nonlinear Volterra model to simulate the behavior of the system. The general theory of continuous and discrete Volterra series is summarised. A Normalized Least Mean Square algorithm is used to determine the Volterra series to third order. We also present as inverted system which is trained with the same algorithm. Training data for the models were collected measuring a physical speaker using a laser interferometer. Results indi…
A central task of a Disc Jockey (DJ) is to create a mixset of mu-sic with seamless transitions between adjacent tracks. In this paper, we explore a data-driven approach that uses a generative adversarial network to create the song transition by learning from real-world DJ mixes. In particular, the generator of the model uses two differentiable digital signal processing components, an equalizer (EQ) and a fader, to mix two tracks selected by a data generation pipeline. The generator has to set the parameters of the EQs and fader in such away that the resulting mix resembles real mixes created b…
Patch-Mix Contrastive Learning with Audio Spectrogram Transformer on Respiratory Sound Classification
KAIST AI（韩国科学技术院人工智能学院）；Department of AI, Kyungpook National University（庆北国立大学人工智能系）；SmartSound（SmartSound公司）；Dongguk University（东国大学）；MODULABS（MODULABS机构）
University of Modena and Reggio Emilia（摩德纳-雷焦艾米利亚大学）；Istituto Italiano di Tecnologia（意大利理工学院）
In this paper, we address the problem of enhancing the speech of a speaker of interest in a cocktail party scenario when visual information of the speaker of interest is available. Contrary to most previous studies, we do not learn visual features on the typically small audio-visual datasets, but use an already available face landmark detector (trained on a separate image dataset). The landmarks are used by LSTM-based models to generate time-frequency masks which are applied to the acoustic mixed-speech spectrogram. Results show that: (i) landmark motion features are very effective features fo…
Considering the bimodal nature of human speech perception, lips, and teeth movement has a pivotal role in automatic speech recognition. Benefiting from the correlated and noise-invariant visual information, audio-visual recognition systems enhance robustness in multiple scenarios. In previous work, audio-visual HuBERT appears to be the finest practice incorporating modality knowledge. This paper outlines a mixed methodology, named conformer enhanced AV-HuBERT, boosting the AV-HuBERT system's performance a step further. Compared with baseline AV-HuBERT, our method in the one-phase evaluation of…
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Mitsubishi Electric Research Laboratories（三菱电机研究实验室）
State-of-the-art approaches for visually-guided audio source separation typically assume sources that have characteristic sounds, such as musical instruments. These approaches often ignore the visual context of these sound sources or avoid modeling object interactions that may be useful to better characterize the sources, especially when the same object class may produce varied sounds from distinct interactions. To address this challenging problem, we propose Audio Visual Scene Graph Segmenter (AVSGS), a novel deep learning model that embeds the visual structure of the scene as a graph and seg…
