Expensive evolutionary search does not always require an exact fitness estimate for every candidate; it often only needs a reliable answer to the simpler question of which candidate is better. To meet this need we introduce Teacher‑Guided Learning NSGA‑II (TGL‑NSGA‑II), a low‑fidelity framework for constrained Tiny Machine Learning (TinyML) neural architecture search. A pretrained teacher partitions samples into strata jointly defined by difficulty and class. Each candidate then undergoes KD‑Lite—a short, capped knowledge‑distillation step on a compact training set—before being scored on a separate stratified evaluation set. The teacher‑guided score is fused with a Gaussian‑process surrogate to select candidates for full evaluation. For a fixed candidate population we analyze evaluation variance, score concentration, pairwise rank inversion, expected Kendall‑$\tau$, first‑front identification, and hypervolume perturbation. We also derive a variance‑aware fusion weight and a capacity‑adaptive distillation rule. In keyword spotting and bird‑call classification experiments the measured Kendall‑$\tau$ values are 0.74 and 0.62, exceeding the predicted lower bounds of 0.60 and 0.46. Joint stratification reduces proxy‑score variance by 41% compared with random evaluation, while selective teacher mismatch increases differential bias and drops Kendall‑$\tau$ to 0.41. Under a constrained evaluation budget, TGL‑NSGA‑II achieves the highest mean hypervolume and smallest generational distance on keyword spotting, records the lowest mean false‑positive rate on BirdCLEF, and runs 2.2× faster than full NSGA‑II. These guarantees apply only to population‑level low‑fidelity evaluation and do not imply convergence of the full evolutionary trajectory.
Review