发表机构
Korea Advanced Institute of Science and Technology; Center of Complex Systems, Korea Advanced Institute of Science and Technology（韩国科学技术院; 韩国科学技术院复杂系统中心）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
该研究将数据集复杂度与神经网络损失地形几何关联，改编自旋玻璃理论的Franz–Parisi构造定义局部熵，经合成与真实图像实验发现数据集复杂度会改变低损失邻域的收缩位置，标签随机化可放大该效应。
AI 中文摘要
有限数据集可拥有相同的规模和低阶统计量，但结构复杂度差异显著。我们通过将邻域尺度上的局部标签混合与训练好的神经网络解周围的局部熵相结合，建立了这种数据集复杂度与损失地形几何之间的联系。局部熵改编自旋玻璃理论中的Franz–Parisi构造，用于测量距参考点每一处距离上，低损失、类解参数配置的有效体积。我们使用自适应序贯蒙特卡洛在有限网络中估计该量。在受控合成扫描中，更高的数据集复杂度会导致参考点附近局部熵的更大幅度下降；在更远的位置，其径向导数变弱且在不同条件下几乎一致。因此，数据集复杂度改变的是有效解体积收缩的位置，而非使其均匀更快地下降。对真实图像数据的实验显示出相同的定性趋势，标签随机化进一步放大了该效应。这些结果表明，数据集结构决定了训练好的解在有限距离内低损失邻域的组织方式。
英文摘要
Finite datasets can share the same size and low-order statistics while differing strongly in structural complexity. We connect this dataset complexity to loss-landscape geometry by pairing local label mixing across neighborhood scales with local entropy around trained neural-network solutions. Adapted from the Franz--Parisi construction in spin-glass theory, local entropy measures the effective volume of low-loss, solution-like parameter configurations at each distance from a reference. We estimate it in finite networks using adaptive sequential Monte Carlo. In a controlled synthetic sweep, greater dataset complexity produces a larger decrease in local entropy near the reference. Farther away, its radial derivative becomes weak and nearly common across conditions. Dataset complexity therefore changes where the effective solution volume contracts, rather than making it decrease uniformly faster. Experiments on real image data show the same qualitative trend, with label randomization further amplifying the effect. These results show that dataset structure shapes how low-loss neighborhoods are organized across finite distances from trained solutions.
Comments18 pages; 39 pages including supplementary material
