Air traffic control (ATC) communication is a safety‑critical dialogue that remains largely human‑driven, even though other parts of air traffic management have been semi‑automated. This paper experimentally evaluates whether large language models (LLMs) can generate operationally realistic ATC transmissions. A general‑aviation flight over San Francisco’s “Bay Tour” route was hand‑transcribed to serve as ground‑truth (P0). Using a pilot‑in‑the‑loop approach, we designed five prompt structures (P1‑P5) with increasing constraint and embedded them in a stateful multi‑turn pipeline where the model plays ATC to a fixed pilot transcript while conditioning on the accumulating dialogue history. Across nine open‑ and closed‑source LLMs we varied the prompt, the presence of a worked transcript from a different experimental flight as an in‑context example, and whether the model conditions on its own prior replies or on injected ground‑truth history. Turns were scored with lexical, structural, and semantic similarity metrics and by an LLM‑as‑judge (GPT‑5.5) validated against human expert annotation. Supplying a worked example improves similarity, but tightening the prompt does not: the lightest prompts perform best, while the most heavily scripted one collapses as its own errors accumulate through the dialogue, which injecting correct history repairs. These results outline a concrete path and its current limits toward LLM‑assisted ATC.
Blogger's Review: Overall, the study demonstrates the promise of LLMs in the ATC domain, yet challenges remain in maintaining dialogue consistency and preventing error propagation, calling for further investigation.