For Large Language Models (LLMs) to reliably answer user queries, users must clearly specify requirements, context, and constraints. However, in practice, user queries often lack specificity, forcing models to infer unstated assumptions that may misalign with actual user intent.
Existing prompt engineering guidelines aim to mitigate this issue but are typically generic and task-agnostic, limiting their practical utility. Moreover, existing guidelines are often formed manually and in a non-systematic way.
To address this, we investigate prompt guideline optimization: the problem of automatically generating task-specific guidelines that assist in writing better-specified prompts for a given task and model. Our key observation is that existing (completed) task examples (i.e., reference answers) often implicitly encode the missing information needed to complete underspecified queries, including behavioral constraints, contextual assumptions, and evaluation criteria.
Therefore, we propose AGOPS, an automatic approach that evolves task-specific guidelines through an optimization scheme involving a prompt LLM writer, a solver LLM, and prompt evolution, maximizing downstream effectiveness on a set of examples (user queries with reference answers). At inference time, our guidelines help users write well-specified prompts, boosting the effectiveness of LLMs.
We demonstrate that in mathematical reasoning, medical question answering, and coding tasks, prompt underspecification leads to significant drops (up to 95.3%) in downstream task performance compared to well-specified prompts. Perhaps more importantly, this drop is hardly recoverable by existing prompt optimization techniques. Users following AGOPS guidelines can consistently regain this loss (increasing performance between 15.5% to 81.7% on average) across all benchmarks.
Blogger's Review: The AGOPS method significantly enhances LLM application performance by automating the generation of task-specific prompt guidelines, showcasing an innovative approach in the field of prompt engineering. This study not only addresses the issue of prompt underspecification but also provides a more systematic guide for future LLM applications, offering substantial practical value.