低通滤波提升视觉模型的行为对齐
Low-Pass Filtering Improves Behavioral Alignment of Vision Models
AI总结:
低通滤波通过去除高频信息提升视觉模型与人类行为的一致性,实现新的最佳性能。
摘要:
尽管深度神经网络在计算机视觉基准测试中表现出色,但它们在模拟人类视觉行为方面仍存在不足,这体现在误差一致性和形状偏倚上。最近的研究假设通过生成性而非判别性的分类器可以大幅提高行为对齐度,这对人类视觉模型有深远影响。本文则表明,生成模型中看似无害的尺寸调整操作实际上起到了低通滤波器的作用。在一系列受控实验中,我们发现从判别模型如CLIP中去除高频空间信息会显著提高其行为对齐度。在测试时简单地模糊图像而不是在训练时使用模糊图像,能够在模型与人类对比基准上取得新的状态-of-the-art分数,将DNN与人类观察者之间的对齐差距减半。此外,低通滤波器可能是最优的,我们通过直接优化滤波器来证明这一点。为了说明最优滤波器的性能,我们计算了所有可能的帕累托最优解的前沿,这此前是未知的。我们通过观察最优高斯滤波器的频率谱大致匹配人类视觉系统实现的带通滤波器的谱来解释我们的发现。我们还展示了对比敏感度函数,描述人类检测正弦波光栅所需的对比度阈值的倒数随空间时间频率的变化,可以很好地用特定宽度的高斯滤波器近似,该宽度也最大化了误差一致性。
英文摘要:
Despite their impressive performance on computer vision benchmarks, Deep Neural Networks (DNNs) still fall short of adequately modeling human visual behavior, as measured by error consistency and shape bias. Recent work hypothesized that behavioral alignment can be drastically improved through \emph{generative} -- rather than \emph{discriminative} -- classifiers, with far-reaching implications for models of human vision. Here, we instead show that the increased alignment of generative models can be largely explained by a seemingly innocuous resizing operation in the generative model which effectively acts as a low-pass filter. In a series of controlled experiments, we show that removing high-frequency spatial information from discriminative models like CLIP drastically increases their behavioral alignment. Simply blurring images at test-time -- rather than training on blurred images -- achieves a new state-of-the-art score on the model-vs-human benchmark, halving the current alignment gap between DNNs and human observers. Furthermore, low-pass filters are likely optimal, which we demonstrate by directly optimizing filters for alignment. To contextualize the performance of optimal filters, we compute the frontier of all possible pareto-optimal solutions to the benchmark, which was formerly unknown. We explain our findings by observing that the frequency spectrum of optimal Gaussian filters roughly matches the spectrum of band-pass filters implemented by the human visual system. We show that the contrast sensitivity function, describing the inverse of the contrast threshold required for humans to detect a sinusoidal grating as a function of spatiotemporal frequency, is approximated well by Gaussian filters of the specific width that also maximizes error consistency.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。