Abstract
Thematic indexing, the practice of assigning structured conceptual labels to sections of text, is crucial for scholarly access in large-scale literary and historical editions, yet it remains a largely manual and labor-intensive process. This paper explores the application of machine learning for automatic thematic indexing, using two substantial sub-corpora of Voltaire's Complete Works as a test case:
- Essai sur les mœurs et l'esprit des nations
- Questions sur l'Encyclopédie
The task is framed as a multi-label classification problem, where the model must assign the set of index entries that a professional indexer would apply to a given page of text. We compare a range of approaches, from encoder-based models with classification heads to generative large language models (LLMs) fine-tuned via Low-Rank Adaptation (LoRA), spanning model sizes from approximately 3 to 120 billion parameters. Our best-performing model, from the Mistral family in a 4-bit quantised configuration, achieves F1 scores of up to 0.67; we argue that these figures represent lower bounds, given the inherent subjectivity of professional indexing and the frequency with which model predictions prove semantically valid despite diverging from the print index.
We further evaluate cross-corpus generalisation and conduct a detailed qualitative analysis of model behaviour on literary and rhetorical features of the source texts that prove particularly resistant to automated treatment. Our findings have implications for the broader challenge of providing structured thematic access to large-scale literary and historical corpora.
Blogger's Review: This study showcases an innovative application of machine learning in traditional fields, particularly in the automation of literary indexing. By delving into Voltaire's works, it highlights model performance and limitations, revealing promising potential for handling large-scale literature despite the challenges of subjectivity and accuracy.