When “More Data” Isn’t Enough: The Quality Crisis in Robotics
The lesson of the AI boom has been that scale wins.
Feed models with more tokens, more images, more parameters, and they get smarter; and that story has held largely true for large language models. But in robotics, the equation isn’t as simple.
Robotics isn’t about processing words or images on a screen. It’s about making decisions in the messy, physical world, where data is noisy, inconsistent, and full of variance. In this context, quantity alone does not unlock autonomy, it clutters the path toward it. Robots need training fuel that is both high quality and abundant to succeed. What matters in robotics isn’t the balance between scale and quality; rather, it’s the ability to scale quality itself.
What Quality Really Means
While there isn’t any exact definition for data quality in robotics, in general, it’s the combination of factors that determine whether a robot can actually learn skills that transfer into the real world. Unlike text or images, where noise is tolerable, robots need data that is sharp, varied, and dependable. At its core, data quality in robotics comes down to three attributes: accuracy, relevance, and consistency.
Accuracy
Accuracy is the foundation of data quality in robotics. It answers the simplest, but most critical, question: did the robot actually do what was intended? Unlike text or images, where noise can be tolerated or smoothed over, robotics data is unforgiving. A placement that’s off by a centimeter may be the difference between setting a cup safely on a counter or watching it shatter on the floor.
Accurate data means that motions are recorded with high fidelity to the intended action. This does not mean that every trial must succeed. In fact, recording intentional failures can make datasets more robust by teaching robots how to recover or avoid errors. Instead, what matters is that both successes and failures are true to how the robot should behave in the real world. A failed grasp is still valuable if it reflects an authentic, physically plausible attempt; while a trajectory that drifts into impossible or erratic motion is not.
In short: accuracy ensures that every individual recorded motion, whether success or failure, is faithful to real robot behavior. Without it, data risks teaching the wrong lessons—and in robotics, the wrong lesson often leads to the wrong outcome.
Relevance
Beyond accuracy, data must be relevant. Accuracy focuses on the integrity of each motion; relevance ensures that the dataset as a whole reflects the world the robot will operate in. That starts with environments, such as varied object positions, backgrounds, people, and lighting. Robots need exposure to the types of motions and conditions they will actually encounter, collected directly from the robot itself. A dataset built only in one controlled lab isn’t enough. Robots trained on spotless white tables may look great in demos, but they stumble the moment conditions change. Put that same robot in a real kitchen with crumbs, shadows, and a cluttered counter, and suddenly it can’t place an object reliably.
Below is a video of Physical Intelligence’s robot which has been trained in various environments.
Relevance also means that the dataset must align with the actual use case a robot is being trained for. If you’re teaching a warehouse robot to stack boxes, showing it household chores won’t help. And if a home-assistant robot never practices in a kitchen, it won’t succeed at the very tasks it was built for. Data that looks varied on the surface but doesn’t connect to the real job just adds noise instead of building useful skill.
In other words, accuracy makes sure each datapoint is trustworthy, while relevance ensures the dataset as a whole is useful and generalizable.
Consistency
Finally, there’s consistency. Consistency isn’t just about clean labels. It is about the data itself being uniform in format, motion type, and robot type. Bounding boxes, trajectories, and action annotations need to follow the same format across datasets. Otherwise, every batch requires manual cleanup before it can be used. Robot motion follows the same smoothness and patterns. If every new batch comes in inconsistent or messy, the model absorbs that noise, and overall quality degrades.
Taken together, accuracy, representativeness, and consistency turn demonstrations from scattered recordings into true training fuel. Without them, more data is just more noise. With them, data becomes a foundation that robotics can actually scale on.
The Hidden Cost of Low Quality Data
For AI systems that learn from text, data is cheap and abundant. Billions of sentences can be collected online, and the sheer scale covers up small errors.
Robotics is different. Every demonstration costs time, labor, and equipment wear. Data does not come from scraping the internet. It comes from people controlling robots in real environments, one task at a time. And unlike text, motion data is fragile. If an operator hesitates, tires, or makes even small mistakes, those mistakes are baked directly into the robot’s learning.
That is why a dataset of one million noisy demonstrations is not an asset. It’s a liability. Large, messy datasets don’t just fail to generalize, they actively weaken models, leaving them brittle and inconsistent in the real world. The result is a vicious cycle. Train a robot on noisy demonstrations and it learns inconsistent strategies, forcing engineers into endless debugging loops. To compensate, teams collect even more data, which only compounds the noise and deepens the fragility. Companies burn through time and capital without breaking free.
Imagine teaching a child baseball with a bent bat and a lopsided ball. No matter how many swings they take, the result will not be progress, only reinforced mistakes. Robotics faces the same trap: without high quality demonstrations, more data means worse robots, not better ones.
Why Simulation Alone Will Not Save Us
A potential solution that’s tossed around in the robotics community is simulation— generate virtual demonstrations at scale and sidestep the bottleneck. Simulations are invaluable for prototyping and stress testing, but they still fall short in crucial ways.
Contact rich tasks such as grasping a slippery object, screwing in a bolt, or folding fabric behave differently in the real world than in a physics engine. Materials bend, surfaces degrade, and environments add noise. Capturing those details at high fidelity would require simulations so expensive and complex that they erase the supposed advantage.
And even when simulations are good, they are often too perfect. Real world data is messy and inconsistent, exactly the qualities robots must learn to handle. Dynamics like motor slippage and uneven pressure are extremely difficult to capture in simulation. In the end, simulated data ends up as lower quality, lacking the real-world consistency needed for generalization.
Simulation accelerates research. But without high quality real world data, robots will not achieve the nuance needed to leave the lab.
A Shift in Mindset
The robotics industry stands at a turning point. In AI, scale became transformative only after labs paired it with curated, higher quality data. Robotics has the opposite problem. Teams are piling on demonstrations in the hope to unlock generalization, but without enough high quality data available, scale just amplifies inconsistency.
The path forward isn’t about choosing between scale and quality, but recognizing how they depend on one another. Scale without quality produces robots that end up in a “master of none” scenario. Quality without scale produces demonstrations that are hyper focused. Progress depends on building both in tandem so robots can generalize across the messiness of the real world.
Data is central to this. It isn’t just fuel to be consumed once, but an ongoing system that needs to be designed, maintained, and improved—more like infrastructure than a consumable resource. Until the industry builds the means to make data both abundant and dependable, robots will remain stuck in limited pilots. But with the right infrastructure, quality and quantity can reinforce each other, and robotics can move toward broader, everyday deployment.
Video credits: Physical Intelligence
Thumbnail credits: Robot Report