分身:音效及其合成孪生体
Doppelganger: Sound Effects and Their Synthetic Twins
AI总结:
介绍Doppelganger基准,用于跨合成-真实边界匹配音效。通过配对真实与合成音频,测试不同训练方式对音频编码器跨边界匹配的影响,发现特定训练可提升匹配准确率,且该匹配具有生成器特异性。
摘要:
音频条件生成器能从真实录音中生成合成音效,导致真实与合成版本共存。但尚无基准衡量能否将合成片段与生成它的特定真实录音匹配。本文引入Doppelganger基准,测试现成音频编码器跨边界匹配情况,发现特定训练方式有效,且匹配具有生成器特异性,人类注释基线也有研究。
英文摘要:
Audio-conditioned generators now produce synthetic sound effects from real recordings, so the real and synthetic versions of an event increasingly coexist in sound libraries and in the corpora used to train audio models -- yet no benchmark measures whether a representation can match a synthetic clip to the specific real recording it was generated from. I introduce Doppelganger, a benchmark for matching sound effects across the synthetic-real boundary, pairing 10,420 real clips across 34 everyday sound events each with an audio-conditioned synthetic twin, alongside a controlled 7-class corpus. Off-the-shelf audio encoders do not cross the boundary cleanly. Making the embedding ignore the boundary the standard way -- training it on sound-event labels -- works on familiar sounds but backfires on new ones, dropping below the untrained encoder. Training on the pairs instead -- a clip and its own synthetic twin -- generalizes. On sound events held out of training, it recovers the exact real source about 80% of the time (up from 61% untrained; chance 0.03%), whereas no objective meaningfully improves category-level recognition on those unseen events. The learned matching is specific to one generator -- it survives changes to that generator's settings but not a switch to a different generator, and collapses for the text-only generators tested. A human annotation baseline (49 listeners) lands well above chance but below the models on the same trials. Synthetic twins fool people into calling them real about 29% of the time, yet a generator-specific detector separates these audio-conditioned twins from real recordings perfectly.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。