GuidedNav

GuidedNavShaping Vision-Language Navigation Representations
through Pre-Action Attention and Spatial Guidance

Jingyu Guo1,2Jiaxin Huang2,3,†Chengxing Lin2,4Xiaoyu Luo2,5Cheng Wen2Shanshan Ye3Tingjin Chu1Shaoli Huang2,*Mingming Gong1,3,*
1University of Melbourne2AgiBot3Mohamed bin Zayed University of Artificial Intelligence4University of Electronic Science and Technology of China5University of Macau

† Project leader   ·   * Corresponding authors

Paper Code Datasets coming soonWatch demos
GuidedNav overview: subtask guidance identifies the active instruction, landmark guidance grounds visual evidence, and spatial guidance represents scene geometry.
GuidedNav combines subtask, landmark, and spatial guidance for vision-language navigation.

Abstract

Vision-and-language navigation (VLN) requires an agent to track progress through an extended instruction, identify landmarks relevant to its current objective, and understand the surrounding spatial structure to act. End-to-end navigation policies typically rely on action supervision to implicitly learn these capabilities. Without explicit guidance on the evidence underlying each decision, however, policies can overfit to correlations predictive of training actions rather than learn the task-relevant grounding and geometry needed for generalization. Our insight is to guide the representations that connect instructions to actions: identifying what remains to be done determines which visual evidence matters, while understanding the geometry around that evidence informs how to move. We introduce GuidedNav, a unified framework that explicitly shapes these representations through training-time guidance. Subtask and landmark guidance steer native pre-action attention toward the active instruction segment and relevant visual regions, grounding action prediction in the current objective. Spatial guidance aligns the policy's internal representations with geometric features that capture long-horizon context and local spatial structure. Together, these objectives strengthen navigation capabilities while avoiding the inference overhead of intermediate textual reasoning and online geometric encoding. We also introduce Guided-R2R and Guided-RxR, two annotation-enriched navigation datasets that support more interpretable VLN research. GuidedNav establishes state-of-the-art performance among the compared monocular RGB policies, improving success rates by 8.1% and 13.1% relative to the strongest prior results on R2R-CE and RxR-CE val-unseen, respectively, while achieving faster inference. Extensive experiments further demonstrate the effectiveness and real-world transferability of our approach.

Method

GuidedNav architecture: native pre-action attention guidance and training-time spatial supervision.
Subtask and landmark guidance shape native pre-action attention, while spatial guidance transfers geometry into the policy. The learned policy predicts actions directly at inference.

Landmark-Centric Data Engine and Visualization

Data pipeline: subtask decomposition and temporal alignment, landmark selection and localization, and per-frame landmark mask generation.
Subtask decomposition and temporal alignment → landmark selection and localization → per-frame mask generation.
Guided-R2R

Step-aligned instruction spans

Reuses FGR2R subtasks and maps them onto continuous trajectories using recorded poses.

Guided-R2R example connecting instruction spans, trajectory intervals, and landmark masks.
Guided-RxR

Longer instructions, finer alignment

Decomposes English RxR instructions and refines temporal intervals before localizing landmarks and propagating masks.

Guided-RxR example connecting multiple subtasks to observation intervals and landmark masks.

Dataset videos show expert observation sequences and generated annotations. Missing or unreliable labels disable the corresponding auxiliary loss while preserving action supervision. These annotations are used during training and are not required as policy inputs at inference.

Main result

Original Table 1 from the paper: full monocular RGB comparison on R2R-CE and RxR-CE val-unseen, including all metrics, external training data and footnotes.
Original Figure 4 and Table 2 from the paper: navigation performance by path length and inference efficiency on an NVIDIA RTX 5090.

Simulator and Real-world Rollouts

Simulator Rollouts

walk towards yellow chair and turn right, walk past couch and straight through doorway towards dining table, walk on left side of dining table through double doors, walk over one doormat, stop on the second doormat.
Go into the arch and walk straight down the hallway until you get to the last door on the left. Go into the laundry room on the left and wait near the washer and dryer.

Real-world Rollouts

Go straight and stop in front of the stairs.
You are facing a row of lockers. Turn left and walk to the end of the lockers. Then turn right and stop in front of the vending machine.
You are facing several sofas. Walk straight past the sofas, turn left at the end, and stop in front of the blue sofa.
Walk past the orange chairs in front of you, then go straight and stop in front of the coffee machine.

CITATION

BibTeX

@misc{guo2026guidednav,
  title={GuidedNav: Shaping Vision-Language Navigation Representations
         through Pre-Action Attention and Spatial Guidance},
  author={Jingyu Guo and Jiaxin Huang and Chengxing Lin and Xiaoyu Luo
          and Cheng Wen and Shanshan Ye and Tingjin Chu
          and Shaoli Huang and Mingming Gong},
  year={2026},
  eprint={2610.13122},
  archivePrefix={arXiv},
  primaryClass={cs.CV},
  url={https://arxiv.org/abs/2610.13122}
}