Abstract
Recent generative models can produce high-quality synthetic images, offering scalable training data for data-hungry models. Existing approaches typically involve 1) training or fine-tuning generators, or 2) using lightweight post-hoc adaptation like prompt engineering or inference-time guidance, making them generator-specific and expertise-intensive.
We study a complementary question: can downstream utility be improved purely by selecting an informative subset from a fixed pool of generated images? The answer is yes. Effective selection must counter a structural bias of modern generators: they tend to over-produce canonical modes of each class while underrepresenting intra-class variation.
Building on this insight, we split each real class into a canonical Homogeneous (HO) subset and a non-redundant Heterogeneous (HE) subset, then score synthetic images by a fidelity-diversity criterion that rewards semantic alignment while penalizing canonical redundancy. This method is generator-agnostic and requires no retraining.
Across multiple benchmarks, it consistently outperforms state-of-the-art data selection baselines and matches real-data performance with up to 40% fewer synthetic samples. The same criterion remains effective when applied on top of stronger task-tuned generators, with gains on both classification and segmentation tasks. Thus, post-generation selection is not a substitute for better generators, but a complementary mechanism for improving the utility of synthetic data.
Blogger's Review: This paper introduces an innovative post-generation image curation method that significantly enhances the utility of synthetic data through homogeneous-heterogeneous splitting. By avoiding dependency on generators, it opens new avenues for the application of synthetic data. Its superior performance across various benchmarks underscores the importance of selection strategies, warranting further exploration in future research.