泛化信号未必是模型选择信号
A Generalisation Signal Need Not Be a Model-Selection Signal
浏览论文内容
中文总结 AI 辅助
本研究探讨计算生物学中模型选择信号，发现基于Hessian的几何代理虽与泛化差距相关，但无法可靠识别部署最优模型，表明泛化信号不等于模型选择信号。
中文摘要 AI 辅助
计算生物学中的模型选择通常依赖于从训练机制中抽取的验证数据，即使部署环境位于该机制之外。当验证不再能保持哪个模型最优时，一种自然的替代方案是利用训练网络自身的属性对候选模型进行排序。我们使用一种新颖的、仅前向的代理指标（该指标受Hessian矩阵范数的启发）以及常见的Hessian度量，在分子性质、蛋白质适应度和药物反应任务上测试了这一想法。与我们的假设相反，随着验证Spearman相关性的恶化，几何度量并未变得更有用：增强验证有助于某些分布偏移，但显著损害其他偏移。更令人惊讶的是，即使在Hessian迹和最大特征值关系较弱或反转的情况下，该代理指标在大多数任务上仍与泛化差距相关，然而这一信号并不能可靠地识别部署最优模型。曲率界限不一定保持跨模型的排序，且低几何分数甚至可能偏向于坍缩的预测器。因此，泛化信号未必是模型选择信号。
英文摘要
Model selection in computational biology often relies on validation data drawn from the training regime, even when deployment lies outside it. When validation no longer preserves which model is best, a natural alternative is to rank candidates using properties of the trained network itself. We test this idea using a novel, forward-only proxy motivated by the norm of the Hessian, alongside common Hessian measures, across molecular property, protein fitness, and drug-response tasks. Contrary to our hypothesis, geometry does not become more useful as validation Spearman correlation deteriorates: augmenting validation helps some shifts but significantly harms others. More surprisingly, the proxy still correlates with generalisation gap on most tasks even when Hessian trace and top-eigenvalue relationships are weak or reversed, yet this signal does not reliably identify the deployment-best model. A curvature bound need not preserve cross-model rankings, and low geometric scores can even favour collapsed predictors. Thus, a generalisation signal need not be a model-selection signal.
发表机构
- BITS Pilani, K K Birla Goa Campus（比拉理工学院K K Birla果阿校区）
- LexisNexis Legal & Professional（律商联讯法律与专业）
- Mahindra University（马恒达大学）
机构由 AI 辅助整理，请以论文原文为准。
