In the field of text-to-audio generation, existing models can produce high-quality audio but often struggle with instructions involving multiple sound events and temporal order. This gap arises because current evaluation and training signals primarily emphasize global similarity or perceptual quality, with limited supervision on instruction-level correctness. To address this, we propose an instruction-level framework using audio-aware large language models (ALLMs) as fine-grained judges to verify the presence of target events and their temporal relations in generated audio.
After validating ALLM judgments through benchmarks and human verification, we construct preference pairs from their feedback for direct preference optimization. Furthermore, we introduce S3Bench, a narrative benchmark for evaluating multi-event temporal instruction following. Experimental results demonstrate that our method improves event completeness, temporal ordering, and joint instruction-following accuracy across existing benchmarks and S3Bench while maintaining audio quality.
Blogger's Review: This study significantly enhances the performance of text-to-audio models under complex instructions by introducing a fine-grained feedback mechanism, showcasing the potential of audio-aware language models in multimodal tasks. Future research could further explore applications of this technology in other types of generative models.