It has been a long, long time since Xiaomi stopped being a mobile company. Today the company’s tentacles reach all types of sectors, from mobile and household appliances until cars, chip design and, from now on, robotics. And the Chinese company has just presented its first vision, language and action model for robotics. Its name: Xiaomi-Robotics-0.
What is this about?. Xiaomi-Robotics-0 an open-source model whose code can be found in GitHub and HugginFace. As the company explains, this model has been optimized to offer “high performance, speed and smoothness in real-time executions.” We should not think of this model as an AI capable of making a robot run and jump like a human, but rather one capable of making a “simple” robot understand its surroundings and know how to make the optimal decision without, for example, destroying whatever it has in its hands.
About the robots. When we talk about AI applied to robotics we are not just talking about a robot being able to move. The device must know and understand that it should not apply the same force when holding a brick as it does when holding a cat, for example. In that sense, there has to be an understanding of the visual, an understanding of what is being seen and an appropriate execution of actions: this is a brick > it is a heavy object > I have to apply more force to hold it and move it from one side to the other.
Xiaomi-Robotics-0 results in the benchmarks | Image: Xiaomi
The benchmarks. Xiaomi has achieved, as detailed on the project website, very good results in the benchmarks I RELEASE (measures knowledge transfer), SimplerEnv (measures performance in real simulations) and CALVIN (measures performance in tasks conditioned by language). According to the company, Xiaomi-Robotics-0 “achieves high success rates and robust results in two challenging two-handed tasks: disassembling LEGOs and folding towels.”
The fun of training. Every AI model draws from a training dataset. In the case of Xiaomi-Robotics-0, a 4.7 billion parameter model, the dataset consists of 200 million time steps of robot trajectories and more than 80 million samples of general vision-language data, including 338 hours of LEGO disassembly videos and 400 hours of towel folding videos.
The results. The company claims in the paper that its model is capable of disassembling complex LEGOs of up to 20 pieces, adapting the grip in real time to avoid errors, using only one hand to place the towel correctly and folding it or, if you pick up two towels from the basket, take one of them, leave it in place and fold only one. This demonstrates an interesting capacity for adaptation and learning that, although it may seem trivial on paper, has its value if we think about industrial and even domestic robots.
Beyond. What this model is demonstrating is being able to adapt to complex and unpredictable geometries, such as that of a towel thrown in a basket, and to understand the, let’s say, “soft physics.” On a towel it may seem like a small thing, but let’s think about manipulating human tissues in an intervention, for example. Same with LEGOs. It’s not just disassembling them, it’s understanding the position of the blocks, how they fit together, what force to apply and at what angle so as not to break them. Let’s think about a robot that removes debris.
An industrial robot has historically been programmed with fixed coordinates, that is, moving something from point A to point B. A robot with AI like the one proposed by Xiaomi would be much more versatile. The first robot learns movements, the second robot learns tasks, and the difference is a world. If we think about a distant future in which there are domestic robots, a robot cleaning dust from a shelf will not be the same as knowing how to identify objects, decorations, etc., and understanding that it must move them to avoid throwing them away and cleaning them thoroughly.
Cover image | Xiaomi
GIPHY App Key not set. Please check settings