Large Language Models (LLMs) have shown strong abilities in automatically generating travel itineraries, yet real‑world travel planning is inherently uncertain: transportation delays, crowd fluctuations, and unexpected stochastic disruptions often invalidate otherwise feasible schedules. Existing benchmarks such as TravelPlanner and TripCraft assume deterministic environments and only evaluate static constraint satisfaction, ignoring the robustness of generated plans under uncertainty. To fill this gap, we introduce UTP‑Bench¹, a large‑scale benchmark for uncertainty‑aware travel planning. The dataset spans 504 Indian cities and includes attractions, restaurants, accommodations, and multimodal transportation networks. Empirical delay distributions and crowd‑density patterns collected from major cities are integrated, enabling stochastic evaluation of itineraries.
We propose three evaluation metrics:
- Buffer Adequacy Score (BAS): @@@MATH_BLOCK0@@@\text{CATS}=\frac{1}{N}\sum{i=1}^{N}\left(1-\frac{\text{Crowd density}_{i}}{\text{Threshold}}\right)@@@MATH_BLOCK1@@@\text{TDAS}=1-\frac{\sum{j}\text{Actual delay}{j}}{\sum{j}\text{Planned buffer}_{j}}$$, assessing tolerance to transportation delays.
Experiments with state‑of‑the‑art LLMs such as GPT‑5, Qwen‑3, Mistral, and Phi‑4 compare model‑generated itineraries against human‑crafted references. Results reveal substantial gaps, especially in temporal buffering, delay‑aware scheduling, and crowd‑sensitive planning, with models struggling to maintain robustness in densely populated cities. UTP‑Bench provides a unified platform for future research, encouraging the development of more reliable LLM‑driven travel planning.
Review: UTP‑Bench is the first benchmark that systematically incorporates uncertainty into travel planning, and its three metrics offer clear, quantitative insight into itinerary robustness. It sets a solid foundation for advancing LLM applicability in realistic travel scenarios, and future extensions could include cross‑border trips and weather‑related uncertainties.