DEEP Robotics presents

DR-VLALearning Human-Like Quadruped Navigation
from Egocentric Web Videos via
Domain-Invariant Scene Representations

Jun Tian^1,2Yingjie Niu^1,*,†Ruo Chen³Hongyi Lin^4,5Chao Li^1,*

¹Deep Robotics²University of Illinois at Urbana-Champaign³Zhejiang University⁴Tsinghua University⁵Massachusetts Institute of Technology

^* Corresponding authors · ^† Project Leader

Paper arXiv Videos

Learning human navigation priors for wheeled-legged robots.
Grounded in demonstrations from real-world deliveries.

Trajectory visualizations

Navigation in motion

DR-VLA trajectory predictions over recorded robot observations.

Scene 0120 seconds

Along the sidewalk

Predicted paths through a tree-lined urban scene.

Scene 0230 seconds

Beside parked vehicles

Trajectory predictions in a scene with vehicles and pedestrians.

Muted loops · Use the player controls to pause, seek, or view fullscreen.

From web video to robot navigation

Overview · Figure 1

Human egocentric videos and four-view robot observations train three experts to predict scene structure and robot trajectories. — DR-VLA connects action-free human video pretraining with robot action learning through a shared representation of scene structure.

10,000 h

Egocentric web videos

2,000

Robot training clips

0.884

Quadruped Navigation Score

92.1

NAVSIM v1 PDM score

The research

Abstract

Quadruped robots navigating shared outdoor spaces must understand complex scenes, anticipate change, and act in socially compliant ways. Egocentric web videos contain abundant human navigation experience, yet transferring it to robots is difficult because humans and robots observe and move through the world differently.

We introduce DR-VLA, a Mixture-of-Transformers framework that transfers this experience through a domain-invariant scene representation (DISR). An understanding expert reasons about the scene, a world expert predicts its future structure, and a flow-matching action expert generates robot trajectories. Using EgoLynx, we first learn from 10,000 hours of human videos without action labels, then adapt to 2,000 teleoperated robot training clips. On the held-out robot test split, DR-VLA improves the Quadruped Navigation Score by approximately 5% and reduces final heading error by 14.6%, each relative to the strongest competing result. A separately trained NAVSIM model reaches 92.1 PDMS without web-video pretraining.

01 / Motivation

Human navigation experience,
shared with robots.

People already demonstrate how to move through crowds, navigate around obstacles, and respond to changing scenes—in everyday videos.

An abundant source of experience

Robot demonstrations require teleoperation and safety oversight. Human egocentric videos provide a much broader range of streets, weather, and social interactions, offering a scalable way to learn navigation priors.

Two gaps to bridge

A human’s head-mounted camera differs from a robot’s low-mounted fisheye views. Human motion also lacks executable robot action labels. Transferring scene understanding and future prediction requires an interface that can span these differences.

Scene structure provides a shared interface:
what is present, where it is, and how it relates to its surroundings.

02 / Method

Understand. Anticipate. Act.

DR-VLA predicts the future in a domain invariant scene representation, then grounds that prediction in a robot trajectory.

Semantic structure

What is in the scene?

Segmentation provides a shared vocabulary of entities, including pedestrians and vehicles.

Spatial structure

Where is it?

Relative depth captures geometry while reducing scale differences caused by camera height.

Relational structure

How does it relate?

Causal spatiotemporal attention encodes relationships across entities, geometry, and time.

EXPERT 01

Understanding

A vision-language model interprets observations and the navigation instruction, learning scene reasoning through caption supervision.

EXPERT 02

World

Predicts future DISR latents to learn how scene structure evolves, using human videos without robot action labels.

EXPERT 03

Action

Uses understanding and world-expert features to generate executable trajectories through flow matching.

03 / EgoLynx

Learn at scale.
Adapt to the robot.

Complementary human and robot data support two stages of training under a shared annotation format.

Stage 1 · EgoLynx-Web

10,000 hours

Human videos, without action labels

Curated egocentric videos cover urban and rural environments, different times of day, and diverse weather. The understanding and world experts learn scene reasoning and future DISR prediction.

Stage 2 · EgoLynx-Robot

2,000 training clips

Grounding in real robot motion

Approximately 17 hours of teleoperated wheeled-legged robot navigation from real-world delivery settings. All three experts are fine-tuned, with robot action supervision and continued scene understanding and prediction.

Navigation instructionsScene captionsDISR targetsRobot actions · Stage 2

Robot evaluation uses a held-out test split from the same teleoperation collection, with no clips shared with the training split. Robot adaptation addresses the remaining viewpoint and embodiment differences.

04 / Results

Better navigation through
structured future prediction.

Evaluated on the held-out EgoLynx-Robot test split and, with an independently trained model, NAVSIM v1.

~5%

Higher QNS
0.884 vs. NaVILA’s 0.843

14.6%

Lower final heading error
0.041 vs. NaVILA’s 0.048

92.1

NAVSIM v1 PDM score
Trained without web pretraining

Quadruped navigation

Quadruped Navigation Score · higher is better

DR-VLA0.884
NaVILA0.843
StreamVLN0.833
π₀.₅0.819
LingBot-VLA 2.00.809
OpenVLA0.802
DepthVLA0.789
InternNav0.764
OmniNav0.743

Values from Table 1. All methods are trained on EgoLynx-Robot; DR-VLA additionally uses EgoLynx-Web. QNS combines safety, instruction compliance, comfort, and progress through pseudo-closed-loop evaluation.

Human video pretraining matters.

Pretraining improves QNS from 0.823 with robot data alone to 0.884 with EgoLynx-Web.

The prediction target matters, too.

Alternative world-expert targets, Table 2a
Prediction target	QNS ↑
No world expert	0.841
Cosmos latent	0.845
DINOv3	0.856
V-JEPA 2.1	0.861
Depth + DINO video	0.866
FLARE-style	0.853
DISR (ours)	0.884

Beyond quadruped navigation

A model trained separately on NAVSIM v1 reaches 92.1 PDMS. Removing the world expert lowers the score to 89.5.

Without world expert89.5

PDMS

With DISR target92.1

Full quadruped results · Table 1

All values transcribed from manuscript Table 1. Up means higher is better; down means lower is better.
Model	ADE ↓	FDE ↓	AHE ↓	FHE ↓	NC ↑	WAC ↑	TTC ↑	CL ↑	EP ↑	IC ↑	QNS ↑
InternNav	0.079	0.098	0.038	0.062	0.945	0.955	0.871	0.815	0.893	0.818	0.764
NaVILA	0.030	0.059	0.028	0.048	0.968	0.975	0.912	0.838	0.941	0.893	0.843
OmniNav	0.090	0.152	0.033	0.054	0.939	0.949	0.863	0.811	0.866	0.803	0.743
StreamVLN	0.036	0.067	0.032	0.053	0.964	0.972	0.905	0.843	0.933	0.884	0.833
LingBot-VLA 2.0	0.044	0.068	0.038	0.054	0.958	0.966	0.893	0.829	0.926	0.863	0.809
DepthVLA	0.056	0.074	0.032	0.049	0.952	0.961	0.882	0.825	0.915	0.842	0.789
π₀.₅	0.036	0.075	0.032	0.052	0.961	0.969	0.899	0.834	0.922	0.874	0.819
OpenVLA	0.041	0.068	0.030	0.051	0.956	0.964	0.888	0.823	0.928	0.855	0.802
DR-VLA w/o web	0.034	0.062	0.030	0.052	0.960	0.970	0.905	0.832	0.924	0.878	0.823
DR-VLA	0.027	0.052	0.024	0.041	0.982	0.979	0.928	0.842	0.945	0.921	0.884

ADE/FDE: average/final displacement error; AHE/FHE: average/final heading error; NC: no collision; WAC: walkable-area compliance; TTC: time-to-collision; CL: clearance; EP: ego progress; IC: instruction compliance. The “w/o web” variant uses EgoLynx-Robot only.

05 / Qualitative results

Human priors meet robot grounding.

Predicted trajectories on replayed observations from four unseen scenes, comparing web-only, robot-only, and jointly trained variants.

Three rows of predicted trajectories in four fisheye scenes, trained on web data only, robot data only, and both sources. — Figure 3 · Top: EgoLynx-Web only. Middle: EgoLynx-Robot only. Bottom: both data sources. These are replay-based trajectory predictions.

Web-only

Steers around obstacles in these examples, but remains poorly calibrated to the robot’s metric scale. This ablation uses DPVO pseudo-trajectory supervision.

Robot-only

Predicts more suitably scaled motion, but follows socially compliant routes less consistently in the shown scenes.

Joint training

Combines human navigation priors with robot grounding. In the third scene, the predicted path avoids an obstacle and returns to the sidewalk.

Evaluation scope. The manuscript reports open-loop and pseudo-closed-loop evaluation with real-robot replay. Large-scale closed-loop deployment remains future work, and transfer still requires adaptation on robot data.

Reference

Citation

@misc{tian_dr_vla,
  title  = {DR-VLA: Learning Human-Like Quadruped Navigation
            from Egocentric Web Videos via
            Domain-Invariant Scene Representations},
  author = {Tian, Jun and Chen, Ruo and Lin, Hongyi
            and Niu, Yingjie and Li, Chao},
  note   = {Manuscript}
}

Preliminary manuscript citation. Publication metadata will be updated when the preprint is available.

DR-VLALearning Human-Like Quadruped Navigation from Egocentric Web Videos via Domain-Invariant Scene Representations