The task of automated subject indexing in libraries can be understood as a multi-label classification task, particularly when the label set is large, fitting the Extreme Multi-Label Classification (XMLC) objective. This study applies a selection of specialized supervised XMLC methods to the test case of subject indexing contemporary German scientific literature collected at the German National Library (DNB). We contrast these results with a classical lexical matching baseline and three of our recently developed LLM-based methods.
Algorithms are evaluated and compared across several metrics, including binary relevance comparisons with previously indexed materials and graded relevance ratings by professional subject librarians. A challenge for all methods is to reliably suggest from the long tail of the subject vocabulary.
We find that supervised XMLC algorithms relying on transformer-based dense features yield the best results in terms of overall binary relevance metrics. However, when focusing on graded relevance and performance in the long tail of our subject vocabulary, the LLM-based generative methods outperform, making them a promising alternative for future productive use.
Blogger's Review: This article highlights the advantages of generative AI over traditional XMLC methods, particularly in handling complex and long-tail data. It provides significant insights for future research and practical applications, emphasizing the need to closely monitor the advancements and applications of generative AI.