This work investigates the evolution of topics in the “future work” sections of scholarly papers, aiming to separate genuine thematic shifts from effects of document coverage, topic assignment, and writing style. We built a snapshot dataset covering 5,655 ACL papers published between 2012 and 2022, of which 4,005 contain full future‑work text.
For topic modeling we compared Non‑negative Matrix Factorization (NMF) with Latent Dirichlet Allocation (LDA). Both methods aggregate topic presence at the paper level and compute normalized prevalence, exploratory trend statistics, assignment stability, and agreement between the input text and the cited references.
Results show that NMF achieves a higher mean normalized pointwise mutual information (0.1387 vs. 0.1280) and better pairwise assignment stability, while LDA yields greater keyword diversity (0.8900 vs. 0.7750). Under a seed‑zero configuration, we observe increasing prevalence of broad model‑training and question‑related vocabulary, alongside substantial citation and document‑structure content. The average overlap between input‑derived and reference topic sets is 0.5191 for NMF and 0.4924 for LDA; temporal rank agreement varies widely, indicating that the two models do not agree on prevalence levels.
These findings provide a descriptive account of the corpus, explicitly noting limitations such as missing references, incomplete runs, and unverified reference provenance. The contribution is an empirical measurement study of longitudinal scholarly signals, not a new topic‑modeling algorithm or forecasting system.
Review