Music Structure Analysis (MSA) aims to uncover the high-level organization of musical pieces. While state-of-the-art methods often rely on supervised deep learning, they are bottlenecked by the need for heavily annotated data and inherent structural ambiguities. This paper proposes an unsupervised evaluation of nine open-source, generic pre-trained deep audio models on MSA.
For each model, we extract barwise embeddings and segment them using three unsupervised segmentation algorithms: Foote's checkerboard kernel, spectral clustering, and Correlation Block-Matching (CBM), focusing exclusively on boundary retrieval. Our results demonstrate that modern, generic deep embeddings generally outperform traditional spectrogram-based baselines, although not systematically. Furthermore, our unsupervised boundary estimation methodology consistently yields stronger performance than recent linear probing baselines. Among the evaluated techniques, the CBM algorithm consistently emerges as the most effective downstream segmentation method.
Finally, we highlight the artificial inflation of standard evaluation metrics and advocate for the systematic adoption of 'trimming' or even 'double trimming' annotations to establish more rigorous MSA evaluation standards.
Blogger's Review: This paper explores MSA through an unsupervised approach, breaking free from the dependency on annotated data, showcasing the potential of deep learning in audio processing. The effectiveness of the CBM algorithm offers new directions for future research, making it a topic worth following.