Vision-Language Navigation (VLN)

Visual-Language Action model Research for robotic navigation in unseen environments.

Introduction

Built on NaVILA (Cheng et al), a universal Vision-Language-Action (VLA) model that lets legged robots navigate via natural-language instructions, this project deploys and evaluates VLN on a Unitree A1 quadruped. Below are indoor deployment demonstrations; see VL-N3RD-Bench for our benchmark of 3D reconstruction pipelines for sim-to-real deployment.

Simple VLN tasks following natural-language navigation instructions
Long-Horizon VLN task

System Requirements

We install the conda environment for NaVILA and run the VLN model on the server with RTX 5090 GPU. Since the paper used 40-series for the setup, there's some incompatibility with the 50-series. For a detailed setup guide, please refer to this instruction written by Richard Wang.