NVIDIA Unveils Project Gr00t for Robotic AI, Apple Vision Pro Integration

Riding high on its soaring stock price and the red-hot artificial intelligence boom, Nvidia on Friday announced several initiatives to further capitalize on AI, including a coalition to build better humanoid robots, a new Blackwell computer chip, and a part to play in the development of Apple’s Vision Pro spatial computing platform.

The announcements flowed from a two-hour keynote delivered Monday by NVIDIA co-founder and CEO Jensen Huang at the annual GPU Technology Conference (GTC) in San Jose, Calif.

Starting with the company’s foundational technology, Huang noted that the needs of AI development will exponentially increase the amount of data involved.

”We’re gonna have to build even bigger GPUs,” he said, noting that its existing Hopper line is great before unveiling “a very, very big GPU.”

Named after mathematician David Blackwell, Huang said the new hardware is more than just a GPU; it is a platform.

”People think we make GPUs. And we do, but GPUs don’t look the way they used to.”

“We are starting to do some really great work in the next generation of robotics,” Huang said. “The next generation of robotics will likely be humanoid robotics;—we now have the necessary technology to imagine generalized human robotics.”

To get there, Huang announced Project GR00T, a backronym invoking the iconic Marvel Comics character that stands for Generalist Robot 00 Technology. On stage, Huang presented several AI-powered robots to the audience, such as the Jetson Thor computing platform designed to perform complex tasks and interact naturally with humans.

Huang said Nvidia is developing a complete system for training humanoid robots that can learn from visual content, including videos and virtual reality, pulling together a number of related divisions and projects within the chipmaker.

“We developed Nvidia Isaac Lab to train GR00T at scale and built NVIDIA Osmo, a compute orchestration service that coordinates the training and inference workflows across Nvidia DGX systems for training, Nvidia OVX systems for simulation, and Nvidia IGX and Nvidia AGX systems for hardware-in-the-loop validation,” the company said.

Jetson Thor is part of NVIDIA’s updates to the Isaac robotics platform, and Huang said they are training these robots for real-world adaptation in its Isaac reinforcement learning gym.

“In a way, human robotics is likely easier,” Huang said. “The reason for that is that we have a lot more imitation training data that we can provide the robots because we are constructed in a very similar way. It is very likely that humanoid robotics will be much more useful in our world because we created the world to be something that we can interoperate and work well in.”

The virtual gym for AI robotics training is called Omniverse, and Huang showed a demonstration that he descrived as “kind of insane, but it’s going to be very, very close to tomorrow.”

The demo showed a robotics building described as a “digital twin of a 100,000-square-foot warehouse” populated by autonomous systems—humans and vehicles—that interact with each other. The environment will allow rapid development of both physical plant engineering and design but better coordinate the work of both humans and robots within. And it’s more than theoretical.

”Siemens is the world’s largest industrial engineering and operations platform, Siemens is building the industrial metaverse, and today we’re announcing that Siemens is connecting their crown-jewel Xccelerator to Nvidia Omniverse.”

Apple’s new Vision Pro headset is also part of the picture.

“Omniverse Cloud streams to the Vision Pro… Vision Pro connected to Omniverse portals you into Omniverse—it is really, really quite amazing,” Huang announced. “And because all of these CAD tools and all these different design tools are now integrated and connected to Omniverse, you can have this type of workflow.”

The automotive industry was an obvious place to harness AI innovation, Huang said.

“Everything that moves will be robotic, there’s no question about that, it’s safer, it’s more convenient,” he explained. ”One of the largest industries is going to be automotive—we build the robotic stack from top to bottom, from the computer system in the case of self-driving cars” to manufacturing.

Huang said Jetson Thor can be deployed in autonomous vehicles. He added that it will be shipped next year and included in Mercedes and Jaguar Land Rover vehicles.

The Nvidia CEO also revealed that the company is building artificial intelligence platforms for other firms, including 1X Technologies, Agility Robotics, Apptronik, Boston Dynamics, Figure AI, Fourier Intelligence, Sanctuary AI, Unitree Robotics, and XPENG Robotics.

“Building foundation models for general humanoid robots is one of the most exciting problems to solve in AI today,” Huang said. “The enabling technologies are coming together for leading roboticists around the world to take giant leaps towards artificial general robotics.”

Edited by Ryan Ozawa.