A LANGUAGE FOR THE PHYSICAL WORLD
A LANGUAGE FOR THE PHYSICAL WORLD
A LANGUAGE FOR THE PHYSICAL WORLD
A LANGUAGE FOR THE PHYSICAL WORLD
A LANGUAGE FOR THE PHYSICAL WORLD
A LANGUAGE FOR THE PHYSICAL WORLD
Objects
Geometric structure
Containment
Contact relations
Relative pose
Support relations
Physical constraints
Interaction topology
This representation becomes a common interface between perception, planning, learning, and execution.
PHYSICAL TOKENS
Physical Tokens: The Vocabulary of Physical Intelligence.
The physical world can be described as a sequence of structured tokens.
Unlike language models that operate on words, robots require tokens that describe physical entities, relationships, and state transitions.
Our tokenizer extracts these tokens directly from perception and updates them continuously during execution.
Object Tokens
<Object>
<Tool>
<Container>
<Liquid>
<Rigid>
<Deformable>
Relation Tokens
<Contact>
<Support>
<Inside>
<Above>
<Below>
<LeftOf>
Constraint Tokens
<Hinge>
<Slider>
<Elastic>
<Fixed>
<Revolute>
<Prismatic>
Motion Tokens
<TranslationChanged>
<RotationChanged>
<SlidingMotion>
<HingeRotation>
<ElasticDeformation>
<Compression>
Force Tokens
<ContactForce>
<NormalForce>
<InsertionForce>
<PullForce>
<PushForce>
<Torque>
Together these tokens form a machine-readable language for describing physical reality.
PHYSICAL PROGRAMS
Physical Programs Instead of Motion Programs.
Physical programs specify desired changes in the physical world.
Instead of telling the robot how to move, we describe which physical relationships should appear, disappear, or change.
The robot is free to choose its own motions while preserving the intended physical outcome.
Instead of
MoveJoint1()
MoveJoint2()
MoveJoint3()
We write
ContactCreated()
SupportRemoved()
TranslationChanged()
InsideCreated()
SupportCreated()
Different robots may execute different trajectories.The physical program remains identical.
GOALS GRAPHS
Every Task Becomes a Goal Graph.
Tasks are represented as graphs of desired world states rather than demonstrations or trajectories.
Each node describes a physical configuration.
Each edge represents the physical transitions required to reach the next state.
This representation separates task specification from execution.
Goal Graph can be generated from multiple sources.
Human demonstration
Natural language
Manual programming
Large Language Models
Existing robot programs
One Representation. Multiple Ways to Program.
GENERALIZATION BY DESIGN
We don't force the model to learn an infinite number of options. We gradually collapse the space of variations.
At each level, some uncertainty disappears.
Pixels → color, texture, lighting, and background disappear.
Object States → specific forms of interaction disappear.
Physical Tokens → specific trajectories disappear.
Goal Graph → the method for defining the task disappears.
As a result, we receive almost "pure physics": a set of objects, their relationships, constraints, and desired changes.
The generalization becomes not a side effect of the model's size, but a consequence of the system's architecture.
Perception removes appearance variability
The first challenge is visual diversity. The same object may appear with different shapes, sizes, textures, colors, poses, camera viewpoints. Our perception system reconstructs an object-centric representation of the scene. Once an object has been identified, its visual appearance no longer determines how it is manipulated.
Physical Language removes action variability
End-to-end models learn trajectories. Trajectories are highly specific to robot morphology, object position, object orientation, workspace layout. Our system instead represents manipulation as changes in the physical world. The program describes what changes, not how to move.
Goal Graph removes demonstration variability
A task is represented as a desired world configuration. The same Goal Graph can be produced from: human demonstrations, manual programming, natural language, LLMs. Different inputs converge to the same semantic representation.
Planning adapts automatically
Because execution is driven by semantic constraints rather than recorded trajectories, the planner can automatically adapt to new object poses, new object instances, new robot embodiments, new environments while preserving the desired physical outcome.
APPLICATION
One Language. Unlimited Applications.
Physical Language describes changes in the physical world — not domain-specific tasks.
The same representation can power robots across completely different environments.
The same Physical Program can describe infinitely many tasks.
Physical Program():
Pick(Object)
SupportRemoved(Object, Source)
Translate(Object)
Orient(Object)
Contact(Target, Object)
Support(Target, Object)
Release(Object)
Home
Object = Apple
Target = Bowl
Constraction
Object = Brick
Target = Wall
Manufacturing
Object = Electronic Component
Target = Fixture
Warehouse
Object = Package
Target = Shipping Box
Laboratory
Object = Sample Tube
Target = Rack
Agriculture
Object = Fruit
Target = Crate
Healthcare
Object = Medication
Target = Dispensing Tray
Our
Vision
Today we program computers with software.
Tomorrow we will program robots with physical programs.
We believe that physical language will become the abstraction layer between perception and action, enabling robots to understand, share, generate, and execute tasks independently of any specific robot or environment.
Toward a Programming Language for the Physical World.