In practice, industries need to pick a small subset of columns from tabular data to cut training cost, improve interpretability, or respect clinical test constraints. Existing feature‑selection benchmarks rely on outdated models, ad‑hoc evaluation or synthetic data, offering little guidance for real tasks. To bridge this gap we built SelectArena, gathering 71 manually curated real‑world tabular datasets that span classification and regression, and evaluating them with state‑of‑the‑art tabular foundation models such as TabNet and FT‑Transformer. Fifteen widely used selection methods are compared across four axes: predictive performance, runtime, stability and validity. Results show that tree‑based CART, RFImportance and the statistical F‑Test lie on the Pareto front for performance‑time trade‑off; for stability and validity, F‑Test, information‑theoretic and distance‑based methods lead, with F‑Test emerging as the most balanced overall. Search‑based approaches are slow, unstable and achieve low performance. SelectArena turns these findings into practical guidance and offers an extensible platform for future research.
Review