Can Jev Nav?

The recent launch of Typesafe’s Jev have left many re-evaluating the ability for language models to play a role in real-time control. Are “System One” models running at 2-5 hz now able to apply its internet-scale pretraining to robotics tasks? Can they compete with algorithmic, deterministic approaches to path and motion planning? Or is the language-only nature of Jev and similar models an impossible barrier.

We put this to the test across both real and simulated environments, giving models short, medium, and long navigation tasks in a variety of environments on a quadruped robot.

The test / model matrix is a follows:

Scene selection

Scenes are 134 homes from the Habitat Dataset [1], run in Habitat-Sim [2][3], with doors removed so every room is reachable; homes were classified by navigable floor area into small, medium and large categories, under 77 m², 77 to 207 m², and over 207 m². Then categorized by how crowded the floor is, so we have an even distribution between cramped and open environments.

Task Selection

Tasks are all navigation objectives from start point to a target object. Objects are pulled from the scene graphs. In each environment, rooms are defined by flood-filling the navmesh. To be an eligible goal point, every object must have a navigable standing point within 1 m of its footprint and a viable navmesh route from the start point. One goal per room, up to six per home, qualified by sitting at least 3 m apart, adding at least 5 m of new route to the task, and overlapping no other task by more than 60%.

Agents were provided with environments in JSON/string format, as Jev can only receive language as input. We define this abstraction as WorldState. We use the below method to convert from XML scenegraphs to WorldState, which is a string representation of the environment, objects, and obstacles.

Jev needs language

Jev performs poorly when WorldState provides numerical values relative to world frame. For example:

"robot": {"position": {"x": 1.2, "y": -0.4}, "yaw_deg": 92},
"objects": [{"label": "chair", "position": {"x": 3.1, "y": 0.8},
             "distance_m": 2.3, "bearing_deg": 31}, ... 20 objects]

"objects": [{"target": true, "label": "chair", "bearing": "ahead_left",
             "distance": "near", "bearing_deg": 31}, ... 5 nearest],
"way_to_target": {"state": "blocked", "blocked_by": "wall",
             "open_sides": [{"side": "right", "kind": "doorway", ...}]},
"robot": {"recent": {"pattern": "advancing"}}

The above two experiments were run on a limited environment and task set of 11 scenes and 84 total tasks. The only change was in the input WorldState format as described: world state → robot state with natural language helpers

We found task completion rates increased from 40% to 90% when instead Jev received STRING values in robot frame, and with helper worlds in natural language such as ahead_left, behind_right, near, far, touching, blocked, tight, clear, doorway, corner, stuck and moving_without_getting_closer as well as bearings in units of degrees out of 360.

Jev is stateless, meaning each call is independent unless otherwise prompted. The literature has yet to evaluate how Jev’s implicit memory degrades over time (its context limit is 32K tok for state and 64K tok including questions). With that low ceiling, it remains to be seen if memory can be injected effectively over long periods. However, for each model, WorldState gives the following values that cache short-term memory:

We score trajectories by first defining a geodesic path G for every task which is the shortest navigable path from starting point to target object

F the navigable free space of the home, s the start, g the standing point beside the target; ℓ is the length of the shortest path inside F.

Then we use a variant of SPL [4] which is the success weighted by path length and delta from the geodesic G. This is calculated per run for each model.

S arrival, 1 if the run ended within 1 m of the target with line of sight, else 0; ℓ the geodesic length; p the length the robot actually drove. A run that arrives along the geodesic scores 1; every metre of detour lowers it.

This penalizes paths that diverge too far from the ideal path. Since S with SPL is binary success, S ∈ {0, 1}, we also grade with a SoftSPL [5] in which S is continuous as a function of final arrival distance from target object. Worth noting that current trajectory grading does not include time, smoothness, and other parameters.

d_final the navmesh geodesic from where the robot stopped to the goal, so a run that stops halfway keeps half the credit; an arrived run has d_final = 0 and the same value as SPL. (·)₊ clips at zero.

Grading

In addition to trajectory scoring, we grade across the following:

All models get a fresh dimOS instance and Habitat sim per task. The dimOS instance includes a Perception module DemoObjects which publishes a stream of Detection3DArray as a stand-in for real perception input from the robot. This is to normalize inputs for all models in the test matrix.

RawRobotBridge JSON (as described above) is then built from the dimOS instance with robot pose (PoseStamped), detections (Detection3DArray), 2D detections (Detection2DArray), pointcloud (PointCloud2), goal (str), the robot's own motion block (RobotState), and optionally a Memory object (Memory).

Jev

TypeSafeAgent calls WorldState at 2hz and sends output to Jev. Jev outputs the following at frequency f defined as 1 / max(0.5 s, l_j) where l_j is latency per jev call inference call. In plain language, if Jev answers in under 0.5 s the loop runs at 2 Hz. It may run slower than 2hz depending on inference time but speed is capped at 2hz.

Jev raw output

Output then converted to Drive object with signed values for x, y, and yaw, converted to dimOS Twist vector and then sent to the robot. As an aside there is additional logic we apply in the Twist conversion that helps smooth the application of velocity, but not worth elaborating on in depth here.

All other Agents

All agents with harnesses are not permitted to use dimOS tools, enforced by their only access to dimOS instance being RawRobotBridge. RawRobotBridge calls RawRobotBridge at 2hz and all agents access the current world state via the robot/world_state/json Zenoh published topic. All agents successfully wrote their own simple client, typically a Python script over eclipse-zenoh.

All agents for this run use Pi, except for Dimensional which runs dimcode. Tools provided Pi's native coding tools only: read, bash, edit, write, plus grep, find, ls.

RawRobotBridge exposes velocity commands, which agent could publish to, clamped to 1 m/s and 1.5 rad/s.

When brought into the real-world, multi-room navigation was a bottleneck. We believe that this is a data problem rather than a true limitation imposed by Jev, as the WorldState relied on clean labeling of walls and doorways which is more challenging outside of a simulated environment with ground truth.

What is clear is that there is opportunity for a control loop sitting comfortably at around 4hz - either by providing coarse inputs or by signalling for help to the slower but more intelligent LLM outer loops.

1. Astra wins, but Jev and Fable are close behind

2. Jev achieves Fable-level performance on shorter paths, and wins speed and cost

3. Astra has the greatest consistency across path lengths & complexity

If you use Can Jev Nav? in your research, please cite this report.

Stash Pomichter, Ruthwik Dasyam, and Henry Ventura. Can Jev Nav? Dimensional Research. https://research.dimensional.org/system-one-navigation/

@misc{dimensionalcanjevnav,
  author       = {Pomichter, Stash and Dasyam, Ruthwik and Ventura, Henry},
  title        = {{Can Jev Nav?}},
  howpublished = {Online research report, Dimensional Research},
  year         = {2026},
  url          = {https://research.dimensional.org/system-one-navigation/}
}

1. Khanna, M., Mao, Y., Jiang, H., Haresh, S., Schacklett, B., Batra, D., Clegg, A., Undersander, E., Chang, A. X., Savva, M. "Habitat Synthetic Scenes Dataset (HSSD-200): An Analysis of 3D Scene Scale and Realism Tradeoffs for ObjectGoal Navigation." CVPR 2024. https://arxiv.org/abs/2306.11290
2. Savva, M. et al. "Habitat: A Platform for Embodied AI Research." ICCV 2019. https://arxiv.org/abs/1904.01201
3. Szot, A. et al. "Habitat 2.0: Training Home Assistants to Rearrange their Habitat." NeurIPS 2021. https://arxiv.org/abs/2106.14405
4. Anderson, P. et al. "On Evaluation of Embodied Navigation Agents." 2018. https://arxiv.org/abs/1807.06757
5. Datta, S. et al. "Integrating Egocentric Localization for More Realistic Point-Goal Navigation Agents." CoRL 2020 (SoftSPL, Habitat Challenge). https://arxiv.org/abs/2009.03231