带结构损失函数的数据驱动超参数多重调优的紧界
Tight Bounds for Data-driven Multiple Hyper-parameter Tuning with Structured Loss Function
AI总结：
该研究针对带结构损失函数的数据驱动超参数多重调优，利用实代数几何建立紧伪维界，提出多区域下界框架并扩展拓扑框架，解决了现有界宽松及缺乏全面下界的问题。
AI中文摘要：
数据驱动算法设计将超参数调优视为统计学习问题，但由于模型性能对超参数存在隐式、非平滑依赖，建立泛化保证仍具挑战性。现有基于分段多项式假设的多维界在理论上仍较宽松，且缺乏全面的下界。我们通过为多维数据驱动调优建立紧伪维界解决了这一问题。首先，我们利用实代数几何改进学习理论上界；通过分析块消元过程中的不变连通符号单元而非孤立符号向量，避免拓扑过度计数，从而推导更严格的样本复杂度。其次，我们提出一种多区域下界框架，将组合容量与代数容量解耦；通过在不同区域构造可击碎的问题实例，证明我们的上界是紧饱和的。最后，我们扩展拓扑框架以适应一般双层验证损失调优及更广泛的半代数应用。
英文摘要：
Data-driven algorithm design frames hyperparameter tuning as a statistical learning problem, but establishing generalization guarantees remains challenging due to the implicit, non-smooth dependence of model performance on hyperparameters. Existing multi-dimensional bounds under piecewise-polynomial assumptions remain theoretically loose and lack comprehensive lower bounds. We resolve this by establishing tight pseudo-dimension bounds for multi-dimensional data-driven tuning. First, we refine the learning-theoretic upper bound using real algebraic geometry; by analyzing invariant connected sign cells during block elimination rather than isolated sign vectors, we avoid topological over-counting to derive strictly sharper sample complexities. Second, we present a multi-regime lower-bound framework that disentangles combinatorial and algebraic capacities. By constructing shattered problem instances across distinct regimes, we prove our upper bounds are tightly saturated. Finally, we extend our topological framework to accommodate general bi-level validation-loss tuning and broader semi-algebraic applications.
