AcoustiClaim：具有仪器真值的数值声明基准
AcoustiClaim: A Numeric Claim Benchmark with Instrument Ground Truth
- Carnegie Mellon University（卡内基梅隆大学）
机构由 AI 辅助整理，请以论文原文为准。
中文总结 AI 辅助
AcoustiClaim提出数值声明基准，用仪器真值评分音频模型输出，弃权策略降低错误率，但部分量仍难准确预测。
中文摘要 AI 辅助
音频语言模型会陈述声学量的数值，但无论是人类意见还是评判模型，都无法判断该数值是否与信号相符。AcoustiClaim从自由文本中提取每个数值声明，根据定义该量的仪器对其进行评分，并按参考来源的位置对每个量进行分类。四个开放权重系统和一个封闭模型，在两个语料库上以五种方式询问十个量，填充了207个单元格。其中，49个单元格输出的不同值少于五个，在可排名的158个单元格中，有八个超过了我们设定的0.3的秩相关阈值，其中三个的置信区间明显高于该阈值，五个来自一个封闭模型读取音高的情况。除三个单元格外，所有可排名单元格的错误率均达到或高于常数预测器的下限。我们训练的解码器在95%的混合语音上对五个语音量以散文形式弃权（不执行），且未保留任何内容，并在干净孪生语音上陈述这些量，仅从音频中复现其目标规则。使用校准阈值后，弃权在混合语音上平均降低了所有十个量的错误率，并在每次分割的八个量上降低了错误率，而随机选择器最多只能降低0.6%。线性基线的排序效果至少与我们相当。F0标准差和微扰仍高于常数下限。
英文摘要
Audio language models state numbers for acoustic quantities, and neither human opinion nor a judge model says whether such a number is true of the signal. AcoustiClaim extracts each numeric claim from free text, scores it against the instrument that defines the quantity, and classes each quantity by where its reference can be read. Four open-weight systems and one closed model, asked for ten quantities five ways on two corpora, fill 207 cells. Of these, 49 emit fewer than five distinct values, and eight of the 158 cells that can be ranked exceed a rank correlation of 0.3, the bar we set, three with an interval clear of it, five of them one closed model reading pitch. Error sits at or above a constant-predictor floor in every ranked cell but three. The reference decoder we train declines the five voice quantities in prose on 95% of mixtures, with nothing withheld, and states them on the clean twins, reproducing its targets' rule from audio alone. With a calibrated threshold, withholding lowers error on all ten quantities on the mixtures in the mean and on eight at every split, against at most 0.6% from a random selector. A linear baseline orders errors at least as well as ours. F0 s.d. and shimmer stay above the constant floor.
