Predictive code completion significantly enhances developer productivity. However, in spreadsheets, such auto-completion features are virtually absent despite their common usage. This paper introduces a benchmark for systems that observe user actions in spreadsheets and predict future actions.
Two main challenges are identified:
- The lack of edit histories in public spreadsheet corpora.
- The complexity of spreadsheet actions (spatial, temporal, composite).
To address the first challenge, we manually curated 52 sequences of 12K actions to recreate spreadsheets from public corpora, seeded by parameterized heuristics and LLM refinement. For the second challenge, we proposed an online evaluation method that expects a prediction after each user action, accepts or rejects that prediction, updates future actions upon acceptance, and continues until the target spreadsheet is achieved.
We utilized multiple baseline predictors, including zero-shot LLMs, fine-tuned SLMs, and classical models, analyzing various properties revealed by our benchmark, such as properties of saved actions, false positives, efficiency, user profile effects, trigger effects, and context effects.
Blogger's Review: This research addresses the gap in intelligent prediction for spreadsheet operations, providing a systematic framework and benchmark that could drive advancements in related technologies. By analyzing user behavior patterns, future spreadsheet tools can achieve smarter auto-completion, greatly enhancing user experience and productivity.