AI 中文总结
本文将分类预测变量的嵌入维度选择转化为约束分配问题，基于近似-估计权衡推导闭式分配规则，提升了有限样本下的预算效率与预测性能。
AI 中文摘要
分类预测变量的嵌入维度通常通过启发式调参选择，尽管它直接影响模型复杂度、近似质量和有限样本泛化能力。本文将嵌入维度选择表述为约束分配问题，核心贡献是证明可根据显式的近似-估计权衡，将嵌入容量分配给异质分类预测变量。我们通过潜在类别表示的奇异值尾部刻画近似误差，而估计误差随总嵌入复杂度增加。在固定全局嵌入预算和易处理近似模型下，这推导出闭式分配规则：分配给每个预测变量的维度与其近似值相对于参数代价的平方根成正比。模拟实验验证了所提近似-估计解释，显示该分配规则相较于标准均匀和基于基数的启发式方法提升了预算效率，尤其在预算紧约束且预测变量异质性显著时效果更明显。真实数据医疗应用进一步表明其在预测准确率和概率校准方面的改进。总体而言，研究结果确立了嵌入维度分配是一个有原则的有限样本优化问题，而非纯粹的启发式建模选择。
英文摘要
The embedding dimension of categorical predictors is usually selected through heuristic tuning, although it directly affects model complexity, approximation quality, and finite-sample generalization. This paper formulates embedding dimension selection as a constrained allocation problem. The main contribution is to show that embedding capacity can be allocated across heterogeneous categorical predictors according to an explicit approximation-estimation tradeoff. We characterize approximation error through the singular-value tail of the latent category representation, while estimation error increases with total embedding complexity. Under a fixed global embedding budget and a tractable approximation model, this leads to a closed-form allocation rule in which the dimension assigned to each predictor is proportional to the square root of its approximation value relative to its parameter cost. Simulation experiments support the proposed approximation-estimation interpretation and show that the allocation rule improves budget efficiency relative to standard uniform and cardinality-based heuristics, particularly when the budget is binding and predictor heterogeneity is substantial. A real-data healthcare application further shows improvements in predictive accuracy and probabilistic calibration. Overall, the results establish embedding-dimension allocation as a principled finite-sample optimization problem rather than a purely heuristic modeling choice.
CommentsKeywords: Machine Learning in OR, Constrained Optimization, Nonlinear Programming, Multivariate Statistics, Predictive Models
