排名可靠的教师引导适应度近似用于昂贵进化优化：一项TinyML架构搜索研究
Rank-Reliable Teacher-Guided Fitness Approximation for Expensive Evolutionary Optimization: A TinyML Architecture Search Study
浏览论文内容
中文总结 AI 辅助
针对昂贵进化优化，提出TGL-NSGA-II框架，利用教师引导的低保真度评分与高斯过程代理融合，在TinyML架构搜索中实现排名可靠且高效的候选评估。
中文摘要 AI 辅助
昂贵的进化搜索并不总是需要对每个候选个体进行精确的适应度估计。它通常需要一个更简单问题的可靠答案：哪个候选个体更好？我们通过教师引导学习的NSGA-II（TGL-NSGA-II）来解决这一需求，这是一个用于约束型微型机器学习（TinyML）神经架构搜索的低保真度框架。一个预训练的教师根据难度和类别共同定义的层级对样本进行分层组织。然后，每个候选个体在紧凑的训练集上经历KD-Lite，一个简短且有上限的知识蒸馏过程，之后在一个独立的分层评估集上进行评分。这种教师引导的评分与高斯过程代理模型融合，以选择候选个体进行全面评估。对于固定的候选种群，我们分析了评估方差、评分集中度、成对排名反转、期望Kendall-τ、第一前沿识别和超体积扰动。我们还推导了一个方差感知的融合权重和一个容量自适应的蒸馏规则。在关键词识别和鸟类叫声分类任务上，测得的Kendall-τ值分别为0.74和0.62，超过了相应的预测下限0.60和0.46。与随机评估相比，联合分层将代理评分的方差降低了41%。相比之下，选择性教师不匹配增加了差异偏差，并将Kendall-τ降低至0.41。在受限的评估预算下，TGL-NSGA-II在关键词识别上实现了最大的平均超体积和最小的世代距离，在BirdCLEF上记录了最低的平均误报率，并且比完整的NSGA-II运行快2.2倍。这些保证适用于种群级别的低保真度评估，并不建立完整进化轨迹的收敛性。
英文摘要
Expensive evolutionary search does not always need an exact fitness estimate for every candidate. It often needs a reliable answer to a simpler question: which candidate is better? We address this need through Teacher-Guided Learning NSGA-II (TGL-NSGA-II), a low-fidelity framework for constrained Tiny Machine Learning (TinyML) neural architecture search. A pretrained teacher organizes samples into strata defined jointly by difficulty and class. Each candidate then undergoes KD-Lite, a short and capped knowledge-distillation procedure on a compact training set, before being scored on a separate stratified evaluation set. This teacher-guided score is fused with a Gaussian-process surrogate to select candidates for full evaluation. For a fixed candidate population, we analyse evaluation variance, score concentration, pairwise rank inversion, expected Kendall-$τ$, first-front identification, and hypervolume perturbation. We also derive a variance-aware fusion weight and a capacity-adaptive distillation rule. On keyword spotting and bird-call classification, the measured Kendall-$τ$ values are 0.74 and 0.62, exceeding the corresponding predicted lower bounds of 0.60 and 0.46. Joint stratification reduces proxy-score variance by 41% relative to random evaluation. Selective teacher mismatch, in contrast, increases differential bias and reduces Kendall-$τ$ to 0.41. Under a constrained evaluation budget, TGL-NSGA-II achieves the largest mean hypervolume and smallest generational distance on keyword spotting, records the lowest mean false-positive rate on BirdCLEF, and runs 2.2x faster than full NSGA-II. These guarantees apply to population-level low-fidelity evaluation and do not establish convergence of the complete evolutionary trajectory.
发表机构
- National Institute of Technology Durgapur（杜尔加布尔国立理工学院）
机构由 AI 辅助整理，请以论文原文为准。
