Automated feature engineering (AutoFE) for tabular learning can be naturally framed as a program synthesis problem, aiming to discover predictive feature transformations from an exponentially large search space.
Recent advancements in large language models (LLMs) have enhanced the expressiveness of AutoFE by enabling feature program generation beyond predefined operator libraries. However, existing LLM-based methods remain fundamentally limited by stateless generation and homogeneous search: feature proposals are produced from static prompts without accumulating search experience, while single-population exploration quickly converges to dominant transformation patterns, rarely discovering complementary feature compositions across transformation families.
We propose TOPOFE, a topology-aware multi-island evolutionary framework for LLM-guided feature engineering. TOPOFE combines family-specialized exploration, adaptive prompt memory, and topology-guided knowledge transfer to efficiently discover diverse and compositional feature programs.
Experiments on 29 public tabular datasets demonstrate consistent improvements over state-of-the-art AutoFE methods across classification and regression tasks. Beyond predictive performance, TOPOFE discovers more diverse and transferable feature programs that generalize across multiple downstream predictors and LLM backbones.
Blogger's Review: TOPOFE significantly enhances feature engineering outcomes by introducing a topology-aware multi-island evolutionary strategy. This innovation not only increases the diversity of feature combinations but also provides better adaptability for various downstream models, showcasing the potential of LLMs in feature generation. Its universal performance across different datasets suggests promising directions for future research in feature engineering.