Two very different jobs, often confused

People often lump together two kinds of AI model that do different but complementary jobs: a world model and a VLA, short for "vision-language-action". The difference comes down to a simple question: what does the model produce?

So why does everyone mix them up? Partly because both take an "action" as an input — a button press for the game, a movement command for the robot. Partly because they're built from very similar machinery underneath. But mostly because researchers are now deliberately combining them into single systems that do both jobs at once. There's even a name for these hybrids: World Action Models. So the blurring isn't purely a mistake — it's a direction researchers are actively pursuing.

And they're approaching it from opposite ends. World model builders are teaching their models to act: NVIDIA's Cosmos 3 and Odyssey's Odyssey-3 both output robot actions alongside video. VLA builders are teaching theirs to imagine: some now practise predicting the next few camera frames before choosing a move, on the bet that a robot that can picture what happens next will pick better moves. Back in the video game, the engine is learning to play and the player is learning to picture the next screen. In October 2026 Dream Labs, founded by the former NVIDIA researchers who coined the term, launched Dream-1 as a World Action Model anyone can use, built from the world-model end.

That's the outside view of a VLA — what it takes in, what it puts out. Here's what's actually happening between those two points.

What's actually inside a robot's brain

A VLA works as a loop that repeats many times per second:

The three inputs

• What it sees — one or more camera feeds, often including a camera on the wrist so it can see what its hand is doing.
• What it's been told — a plain-English instruction like "put the red block in the bin."
• Where its own body is — the current angle of every joint. This matters more than it sounds: you can't decide how to move your arm without knowing where your arm currently is.

Crucially, nobody builds this understanding from scratch. Researchers start with an existing AI model that already understands images and language from being trained on the internet, then teach it about robot movement on top. That inherited knowledge is why a robot can be told "pick up the fruit" and reach for a banana it's never seen before.

That's what's inside one robot's brain. The industry's big ambition is to make that same brain work across many different robots, which is where a newer label comes in.

Robot foundation models: one brain, many bodies

So what the heck is a robot foundation model? Mostly, it's a new name for a big VLA. "Foundation model" is the AI industry's term for one large model trained on broad data and then adapted to many jobs; the models behind ChatGPT and Gemini are foundation models for language. A robot foundation model is the same idea for robots: a VLA trained on recordings from many different robots and tasks, so it can be adapted to new ones.

The two labels describe the same kind of model from different angles. "VLA" says how it works: vision and language in, actions out. "Robot foundation model" says how broadly it's trained and how widely it's meant to be reused. Physical Intelligence's π0 and Google DeepMind's Gemini Robotics are both. A few companies, like Skild AI, describe their models only as foundation models, but they do the same job.

How is this different from GPT-6 or Claude?

The big labs' general-purpose models are moving into robotics too. OpenAI highlighted robot control when it launched GPT-6 Astra, and in an early independent test across 42 simulated robot tasks, Astra, with no robot-specific training at all, outscored every publicly released robot model on the leaderboard, though at only about 22% average success. Anthropic has taken a different route: in August 2026 it previewed a standard for connecting Claude to robots and lab equipment.

The two are relatives. A VLA starts from a model much like these and is then retrained to produce movement. The difference is what comes out. GPT-6 or Claude answers in words and code: to move an arm, it reasons through the problem, then names a target position for the gripper, and separate software works out the joint movements. That takes seconds per decision. A robot foundation model outputs motor commands directly, many times a second, because it learned from recordings of robots, and people, actually moving.

So each is strong where the other is weak. The same test found Astra good at tasks that depend on understanding the scene and the instruction, and poor at anything needing precision, fast reactions or two hands working together, which is exactly what robot foundation models are trained for. That maps neatly onto the two-speed design covered in Part Four: a general model as the slow planner, a robot foundation model as the fast hands. Google already splits the work this way, with Gemini Robotics ER 2 planning and a separate model doing the moving.

Where the industry is heading

The goal is one model, trained once, that can run a robot arm, a humanoid or a four-legged robot, with only a little extra training for each new body. That's how language AI already works, and robotics companies want the same economics. Skild AI calls its model "omni-bodied" and says it can control robots it has never trained on. Google DeepMind says a single Gemini Robotics model drives two different two-armed robots and a humanoid. Dream Labs says its Dream-1 model can run two-armed robots it has never seen without extra training.

How far that goes depends on which part of a robot's brain carries over to a new body, and which part doesn't.

How a new body gets plugged in

• Train on many bodies at once. Pool recordings from many robot types (the shared datasets in Part Four exist for exactly this), so the model sees lots of variety. Each robot's movements are translated into a common format the model can learn from, and a small body-specific component translates the model's output back into that robot's joints.
• Post-train on the new robot. A person remote-controls the new robot through the task a number of times, and the model is fine-tuned on those recordings. This is the "just post-train it" step, and the amount needed is shrinking: DeepMind says its on-device model adapts to a new robot from fewer than 200 examples (see Gemini Robotics 2 in Part Four).
• Learn to adapt on the fly. Skild says it trains its model to adapt rather than memorise one body, partly by practising in physics simulations, so it can work out what body it's in from how that body responds. It claims this handles damaged limbs, jammed wheels and heavier loads without retraining. Those are the company's own claims, not yet independently tested.

That's why the "one brain" bet is credible: the knowledge is shared, so each new robot needs less data of its own. But plugging in a new body is never free. Part Four covers the work that remains.

When a world model helps a robot

Every job an imagination does for a robot happens at one of two moments: while the robot is training, or in the instant it's deciding what to do.

Training-time uses are the practical win happening today; runtime, in-the-moment decision-making is the harder, less-solved half.

One of the training-time uses above — making up practice material — assumed there was footage to learn from in the first place. Here's where that footage can come from when nobody demonstrated the task on a robot at all.

Learning movement from video nobody labelled

Here's the puzzle. There's an enormous amount of video of people doing things with their hands. It would be extraordinarily useful for training robots — except none of it records what movements were made. You see the outcome, not the instruction.

The solution is a clever trick called a latent action model. "Latent" here just means hidden — it works out the hidden action that must have happened between one frame and the next.

The trick: feed two consecutive frames into an encoder that compresses "whatever changed" into a tiny code. A separate decoder gets only the first frame plus that code — never the real second frame — and has to predict it. Compare the guess to reality, adjust both, repeat millions of times. Because the code is too small to carry a whole image, the encoder is forced to throw away everything that stayed the same and keep only the motion.

Predicting pictures vs. predicting ideas

This is the deepest live disagreement in the field, and it's surprisingly easy to grasp.

Most world models predict what the world will look like — actual images. An alternative approach, championed by Yann LeCun and called JEPA, argues this is fundamentally wasteful. Instead of predicting pictures, it predicts a compressed summary of what's happening — the gist rather than the image. (My later primer, World models, from the ground up, covers both camps and how each is built.)

An example: a ball rolling behind a box

A red ball rolls across a table and disappears behind a box for a moment.

The claim is that this is exactly what a robot needs. To decide what to do, a robot doesn't need a perfect video of the future — it needs to know "if I push here, the cup tips over and spills." That's a fact about consequences, not appearances. It's also cheaper to work out and less easily confused by irrelevant visual changes like a shadow moving.

The argument has real money on both sides. Fei-Fei Li's World Labs is building realistic 3D worlds for robots to practise in; Yann LeCun's AMI Labs skips pictures entirely and predicts consequences in the abstract. Each has raised about $1 billion, and Nvidia has invested in both: opposite philosophies, same cheque. (World models, from the ground up covers both companies in more detail.)

The hard problems every robot has to solve

Whichever model a company bets on, two problems come up for everyone: the robot's thinking is slower than its hands, and every new body needs its own last mile of work.

Slow thinking, fast hands

This is the constraint that shapes robot design more than almost anything else, and it's rarely mentioned in headlines.

The "thinking" part of a robot's brain is big and slow. It can form a fresh understanding of a scene maybe two to five times per second. But hands doing careful work — anything involving touch, grip, or balance — need updates ten to fifty times per second to stay steady. There's a gap, and it's a problem.

The first workaround is to plan a burst rather than a single step. Instead of deciding one tiny movement at a time, the robot plans roughly the next second of motion, called action chunking, and carries it out before checking back in, so it isn't waiting on every thought. The second, which most systems now use, is a two-speed design: a slower part works out the overall plan a few times a second, while a smaller, much faster part handles the split-second adjustments in between. This mirrors how people work — you consciously decide "I'll pick up that mug," but the fine muscle coordination that actually does it runs faster and more automatically, without you thinking through every millimetre.

Engineers often call that faster part the low-level policy or controller: a small, fast model — sometimes little more than straightforward maths — whose only job is turning the slow part's rough intention into a real stream of joint-by-joint motor commands, tens of times a second. It doesn't understand language or images at all; it just executes.

The last mile to a new body

Robot foundation models (Part One) make it cheaper to bring up a new robot, but "just post-train it" still undersells the work:

• The data still has to come from that body. Even a few hundred demonstrations means someone operating the new robot by hand, task by task.
• Hands are the hardest jump. A simple gripper opens and closes; a five-fingered robot hand can have twenty or more joints to coordinate. Moving a model between two arms is a much smaller step than moving it from a gripper to a hand, or from an arm to a walking humanoid.
• The model isn't the whole robot. Cameras, calibration, safety limits and the fast control loop above still have to be built and tuned for each body. A 2026 survey calls this the embodiment gap: the work that remains between a model that generalises and one that actually runs on your robot, which a headline success rate doesn't show.

Every company below is working around those problems in its own way. Here's who's building what.

Who's building what

Robot brains (VLAs and other robot foundation models)

World models

For a fuller, more recent look at who is building world models, and how, see World models, from the ground up.

Practice environments

These are the virtual worlds where robots do most of their learning:

• MuJoCo — the academic default, free and open. Especially good at simulating touch and contact, which makes it the standard for manipulation research. Nearly every open robot model works with it out of the box.
• Isaac Lab / Isaac Sim — NVIDIA's industrial-grade option. Much faster on their hardware, and the usual choice for training walking robots or generating simulated footage at scale.
• Genesis — a newer entrant that can also simulate soft materials and fluids, not just rigid objects.
• Gazebo, PyBullet, Drake, Webots — established general-purpose tools, still widely used.

Shared training data

Because collecting robot data is so expensive, the field pools it. The main collections:

• Open X-Embodiment — the big combined dataset spanning many different robot types. The usual starting point.
• DROID — one robot recorded across a huge variety of settings, which makes it especially useful for testing whether a robot copes with unfamiliar surroundings.
• RH20T — 110,000+ recordings across 7 robots and 147 tasks, strong on tasks involving touch and contact.
• RoboCasa — kitchen scenarios, and probably the closest thing the field has to a standard benchmark environment.
• LeRobot (Hugging Face) — not a dataset but a shared format and toolkit, an attempt to get everyone's robot data working together. Hundreds of community datasets use it, though quality varies enormously.

Glossary