Tau Robotics
Building toward general-purpose humanoid robots
How do you teach a humanoid robot to clean someone else's home?
A humanoid robot is shaped like a person: a torso, a head, two arms and two legs. The point is often practical. A body like ours can use human tools and human spaces, and most of the world is built for people.
The motors or pistons that move a robot's joints, doing the job muscles do for us. do the work of muscles. They can be electric, pneumatic or hydraulic, and the wish list is always the same: high power, low mass, small size. Electric actuators are the most popular, but a single one may not be strong enough for a human-sized joint, so it is common to gang several onto one joint.
Then there is staying upright. Sensors that track the robot's own body: joint angles, orientation and speed. tell the robot the position, orientation and speed of its body and joints, the job our inner ear and muscles do for us. To keep its balance while walking, a robot also needs to know the contact forces under it and how it is moving compared with how it means to move. The standard idea here is the The point on the ground where the forces from a walking robot's contact balance out; keeping it under the feet keeps the robot from tipping..
Further reading Humanoid robot (Wikipedia)
- i.
Too many joints
More joints make a robot more capable and harder to run. Extra The independent ways a robot can move, roughly one for each joint axis. open up more tasks, and they bring more complexity and new problems to planning and control. Control methods that coordinate all of a robot's joints together toward several goals at once. is the field that tries to coordinate all those joints at once, so a robot can do several things simultaneously in a sensible order of priority, like wiping a counter without tipping over.
- ii.
Homes are not factories
A factory arm stays bolted down in a highly structured environment. A humanoid moves around and deals with whatever is there, so it has to worry about colliding with itself, finding a path and avoiding obstacles. Legs add a risk of falling, which is why some humanoids are only an upper body on a wheeled base.
- iii.
No internet of chores
Language models learned from a web full of text. There is no web full of robots doing housework. Large, broad datasets of robot interaction are hard to come by, and the ones that exist are often narrow: one environment, one set of objects or a small range of tasks.
Further reading Humanoid robot (Wikipedia)Open X-Embodiment: Robotic Learning Datasets and RT-X Models (arXiv)
Tau wants robots that do the work nobody misses doing. For now that means a humanoid cleaning service in real San Francisco homes: wiping surfaces, vacuuming, taking out the trash and picking up clutter.
The longer aim is the model underneath. Tau is building AI that it hopes will eventually do the job without a human operator.
Further reading Tau Robotics (Tau Robotics)Service Privacy Policy (Tau Robotics)
- Step 1: A person at the controls
Each cleaning is done by a humanoid operated in real time by a trained person in another location, who sees the home through the robot's cameras. This is A person operating a robot from a distance, seeing through its sensors., and it lets the service do real work before the robot can do it alone.
- Step 2: Every visit is data
The robot records video from cameras on its head and wrists, along with joint positions, forces and balance state. That footage, the telemetry and the operator's commands become training data for AI models that take camera images and a text instruction and output robot movements directly., which take in images and an instruction and put out low-level robot actions. The reasoning is blunt: "the only way we know to get there is to learn from real work in real homes."
- Step 3: Several ways to learn
The research spans world models, reinforcement learning, imitation learning and pretraining, and simulation and evaluation, so the real-home data is only one ingredient.
Further reading Service Privacy Policy (Tau Robotics)Vision-language-action model (Wikipedia)Careers (Tau Robotics)
Does experience on one robot transfer to another?
When 21 institutions pooled data from 22 different robots, a large model trained on the lot improved several of the robots by drawing on experience from the others. Whether that holds for a whole humanoid body doing housework is less clear.
When does a robot need to feel?
A recent survey found that outside a few tasks such as pouring, peg-in-hole insertion and handling delicate objects, imitation-learning models don't yet work at a level where force truly matters. Cleaning, with its scrubbing and pressing, may be a good test.
Further reading Open X-Embodiment: Robotic Learning Datasets and RT-X Models (abstract) (arXiv)Towards Forceful Robotic Foundation Models: a Literature Survey (arXiv)
Tau Robotics is building toward general-purpose humanoid robots — machines that can perform a wide range of real-world tasks without being reprogrammed for each one. The company is developing the underlying model that will make this possible, using humanoid robots with human teleoperators to collect the real-world interaction data required to train it.
The team's approach centers on whole-body control and physical learning: the belief that generality in robotics requires robots that can act with their full embodiment, not just their hands. Progress is methodical and grounded in what the data actually shows.
- Actuators
- The motors or pistons that move a robot's joints, doing the job muscles do for us.
- Proprioceptive sensors
- Sensors that track the robot's own body: joint angles, orientation and speed.
- Zero Moment Point
- The point on the ground where the forces from a walking robot's contact balance out; keeping it under the feet keeps the robot from tipping.
- degrees of freedom
- The independent ways a robot can move, roughly one for each joint axis.
- Whole-body control
- Control methods that coordinate all of a robot's joints together toward several goals at once.
- teleoperation
- A person operating a robot from a distance, seeing through its sensors.
- vision-language-action models
- AI models that take camera images and a text instruction and output robot movements directly.
- 1Humanoid robot · Wikipedia
- 2Open X-Embodiment: Robotic Learning Datasets and RT-X Models · arXiv
- 3Tau Robotics · Tau Robotics
- 4Service Privacy Policy · Tau Robotics
- 5Vision-language-action model · Wikipedia
- 6Careers · Tau Robotics
- 7Open X-Embodiment: Robotic Learning Datasets and RT-X Models (abstract) · arXiv
- 8Towards Forceful Robotic Foundation Models: a Literature Survey · arXiv