NeFut Logo NeFut
中 Admin Login

[CS.AI] Trajectory-Grounded Instruction Translation for Frozen Aerial VLN Agents

Published at: 2026-10-09 22:00 Last updated: 2026-10-12 00:49
#algorithm #AI #Machine Learning

Aerial vision-and-language navigation (VLN) agents are usually trained on detailed, trajectory‑aligned commands, while real users tend to issue short, intent‑driven instructions. This instruction gap drops the success rate on a frozen OpenFly navigator from $31.03\%$ to $11.33\%$. To narrow the gap, the authors first prompt a language model with human‑written examples to convert original commands into intent‑centered weak commands, raising the success rate to $15.27\%$. They then introduce the Trajectory‑Grounded Instruction Translator (TGIT), which keeps the navigator frozen and learns from its trajectory outcomes to translate weak inputs into executable commands. After weak‑command training, TGIT lifts the weak‑input success rate to $37.93\%$ and achieves zero‑shot improvement on real human instructions from $11.33\%$ to $32.51\%$. It also boosts performance on unseen OpenFly scenarios from $4.95\%$ to $20.79\%$ and shows recovery on CityNav and AirVLN.

Review

Original Source: https://arxiv.org/abs/2610.10635

[h] Back to Home