Large language models (LLMs) are emerging as potential numerical optimizers, yet their true optimization power and limits, especially in high dimensions, remain unclear. This work proposes an evaluation framework that separates solution quality from execution failures such as numerical inaccuracies, constraint violations, and output‑format errors. Representative tasks from convex optimization, fractional programming, and constrained projection are used to compare LLM outputs with deterministic reference solvers across varying dimensions. Results show that LLMs can capture useful optimization structures and produce near‑optimal solutions, even when the raw outputs fail to meet numerical or formatting standards. Reliability, however, degrades as problem dimension grows and depends strongly on the underlying optimization structure. To probe these limits, we introduce an iterative refinement loop that supplies targeted feedback on feasibility and solution quality. The refinement markedly improves reliability and extends the dimensional range where LLMs yield valid solutions, yet persistent numerical and structural errors prevent them from serving as standalone optimizers. These insights clarify the capabilities and boundaries of LLM‑based optimization and motivate hybrid approaches that combine LLM reasoning with deterministic numerical verification.
Review