A robot that can learn on its own shows promise for future AI

A journalist visited the offices of Generalist AI in Cambridge, Massachusetts, to witness robots performing complex tasks with minimal training. The robots, equipped with robotic arms, were able to complete tasks such as stacking cups and moving objects into bowls after watching short instructional videos. One robot even adapted by using a dustpan as a brush when the original tool was removed. Another robot successfully unzipped a purse and retrieved notes, switching hands when necessary. These demonstrations highlight the robots' ability to improvise and apply learned skills to new situations. Pete Florence, CEO of Generalist AI, compared the robots' capabilities to those of GPT-3, noting that the models can be prompted to perform new tasks with a high chance of success. The company focuses on teaching robots the physics of the world, inspired by human intuition. This approach allows robots to transfer knowledge between different scenarios, similar to how children experiment and adapt when learning new tasks. Generalist AI’s team includes former employees of Google DeepMind and Boston Dynamics, and they have developed a unique training method that involves collecting large-scale physical interaction data without being tied to a single robot. The company has also built its AI models from scratch rather than relying on open-source language models. Danfei Xu, a roboticist at Georgia Tech, praised the startup for its execution and scientific approach, noting that their work suggests a path toward deployable robots in real-world settings. Karen Liu, a roboticist at Stanford University, highlighted the company’s data-driven approach as a strong bet for future robotic capabilities. However, Generalist AI acknowledges that its models are not yet fully reliable, with a success rate of around 59% for completing tasks. The company is still working on improving generalization across different tasks and environments. Despite these challenges, the potential for robots to quickly learn and adapt in industries like manufacturing is significant. One engineer observed a robot independently stacking cups after being shown the task, demonstrating the system’s ability to learn and apply new skills in real-time.