Production recommender systems exhibit a feature‑lifecycle asymmetry: adding a feature must improve at least one metric, while removing a feature must never degrade any metric. In practice, prioritizing deprecation is difficult because a feature may interact with others, its importance can change daily, it cannot be evaluated before model updates or data drift, it competes for A/B test slots, it may cause regressions in online experiments, or it is consumed by multiple downstream models that need coordinated removal.
FORTE is a model‑agnostic production framework designed to discover, validate, and safely retire features. It leverages an efficient perturbation engine to enable daily feature evaluation and uses grouped ablation to assess candidate feature sets constructed from all low‑rank features, thereby accounting for interactions. Offline results are sufficiently predictive of online A/B tests, allowing deprecations to proceed with confidence. A lineage‑tracing component automatically identifies all downstream consumer models, ensuring that the full savings of feature removal are realized.
Empirical validation on two large‑scale recommender systems shows that FORTE reliably identifies deprecable feature sets without causing online regressions. For a search‑relevance model, FORTE produced three nested candidate sets of 32, 48, and 59 features. Online guardrails rejected the largest set; the 48‑feature set was selected, removing 43.3% of the model’s features. On a large ranking model, FORTE identified eight removal candidates; retraining and A/B testing confirmed no regression. The launch reduced trainable parameters by 29.9%, improved serving throughput by 19.8%‑25.5%, and cut annual training costs by hundreds of thousands of dollars.
Review: FORTE demonstrates that daily perturbation, grouped ablation, and lineage tracing can together provide an efficient, reliable, and interpretable solution for feature deprecation, establishing a reproducible best practice for industrial recommender systems.