监督学习中表示可辨识性的纤维准则
A Fiber Criterion for Representation Identifiability in Supervised Learning
AI总结:
本文提出纤维准则,通过投影映射的纤维常数性来形式化监督学习中表示-头部分解的可辨识性,并指出仅凭监督预测行为无法唯一确定表示。
摘要:
监督学习通过输入-输出行为评估预测器。当预测器实现为复合函数 $f=c\circ h$ 时,监督证据约束了复合映射 $f$,但未必确定表示-头部因子分解 $(h,c)$。本文形式化了由此产生的表示级可辨识性问题:对于一类可接受的表示-头部对,当且仅当表示属性在投影 $(h,c)\mapsto c\circ h$ 的纤维上为常数时,它可从诱导的预测器中辨识,等价于它下降为预测器的良定义属性。保持预测器的增广给出了一个规范障碍:辅助信息可以附加到表示上而头部忽略它,保持预测器不变但改变诸如极小性、压缩、不变性、等变性、干扰信息或语义可访问性等属性。这种构造将表示可辨识性与优化和有限样本估计分离开来。有限样本诊断说明了而非证明了该准则:精确代数见证在改变表示诊断时保持预测器固定,而匹配性能的Waterbirds模型表明不同约束可以在相似的监督性能下选择不同的表示。结果阐明,表示级声明需要超越监督预测行为本身的假设、目标、测量或归纳偏置。
英文摘要:
Supervised learning evaluates predictors through their input-output behavior. When a predictor is implemented as a composition $f=c\circ h$, supervised evidence constrains the composite map $f$ but need not determine the representation-head factorization $(h,c)$. This paper formalizes the resulting representation-level identifiability problem: for a class of admissible representation-head pairs, a representation property is identifiable from the induced predictor exactly when it is constant on the fibers of the projection $(h,c)\mapsto c\circ h$, equivalently when it descends to a well-defined property of the predictor. Predictor-preserving augmentation gives a canonical obstruction: auxiliary information can be appended to a representation while the head ignores it, leaving the predictor unchanged but altering properties such as minimality, compression, invariance, equivariance, nuisance information, or semantic accessibility. This construction separates representation identifiability from optimization and finite-sample estimation. Finite-sample diagnostics illustrate, rather than prove, the criterion: exact algebraic witnesses hold the predictor fixed while changing representation diagnostics, and matched-performance Waterbirds models show that different constraints can select different representations at similar supervised performance. The results clarify that representation-level claims require assumptions, objectives, measurements, or inductive biases beyond supervised predictive behavior alone.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。