A New Era of AI Learning
Artificial intelligence is poised to move beyond the boundaries of human-derived data. A groundbreaking collaboration between NVIDIA and Ineffable Intelligence, the London-based AI lab founded by renowned AlphaGo architect David Silver, aims to build the foundational infrastructure for a new breed of AI systems: those that learn continuously from their own experience, rather than relying solely on pre-existing datasets. This partnership, announced as Ineffable emerges from stealth mode, signals a strategic shift toward reinforcement learning at an unprecedented scale.
The Vision: From Pretrained Knowledge to Self-Discovery
Reinforcement learning agents—AI systems that learn through trial and error—are uniquely capable of converting raw computation into actionable knowledge. Unlike traditional pretraining, which feeds a fixed corpus of human data into a model, reinforcement learning requires the system to generate its own data through interaction with an environment. This distinction is central to the collaboration between NVIDIA and Ineffable Intelligence.
David Silver's Pursuit of Superlearners
David Silver, a pioneer in reinforcement learning, has long advocated for what he calls "superlearners"—systems that discover new knowledge autonomously. “Researchers have largely solved the easier problem of AI: how to build systems that know all the things humans already know,” Silver stated. “But now we need to solve the harder problem: how to build systems that discover new knowledge for themselves. That requires a very different approach—systems that learn from experience.” This vision aligns with NVIDIA CEO Jensen Huang’s description of the next frontier: “superlearners—systems that learn continuously from experience.”
Engineering Challenges of Real-Time Learning
Building a training pipeline for reinforcement learning at scale presents unique engineering hurdles. Unlike static datasets, reinforcement learning workloads generate data on the fly. The system must act, observe, score, and update in tight, repeating loops. This places intense demands on interconnect speeds, memory bandwidth, and inference serving—pressures that are absent in conventional pretraining workflows.
The Tight Feedback Loop
Every cycle in reinforcement learning requires rapid data generation and processing. The agent interacts with an environment (simulated or real), receives a reward signal, and adjusts its policy. This continuous feedback loop must be orchestrated with minimal latency to allow for efficient learning. NVIDIA and Ineffable are jointly engineering a pipeline that can sustain these loops at massive scale, ensuring that the system never waits idly for data.
Beyond Human Data
Another critical aspect is the type of data these systems will train on. Reinforcement learning can exploit rich forms of experience that differ markedly from human language or labeled images. These may include complex simulated physics, multi-agent interactions, or entirely novel domains. As a result, the infrastructure must accommodate not only new algorithms but also novel model architectures that can process these diverse streams of experience efficiently.
NVIDIA's Role: Hardware and Software Co-design
The collaboration leverages NVIDIA’s cutting-edge hardware platforms to accelerate the development of this next-generation reinforcement learning infrastructure. Engineers from both companies are working together to explore optimal ways to create the training pipeline.
Starting with Grace Blackwell, Exploring Vera Rubin
The initial work will be performed on the NVIDIA Grace Blackwell superchip, which combines high-performance Arm-based CPUs with NVIDIA’s most advanced GPUs. This platform is designed for the massive parallel computation required by modern AI workloads. Furthermore, the team will be among the first to explore the upcoming NVIDIA Vera Rubin platform, which promises even greater capabilities. The goal is to understand the hardware and software requirements as the AI world shifts from learning from human data to learning through simulation and direct experience.
The Promise: Unlocking Breakthroughs Across Domains
Getting the infrastructure right is crucial. If successful, this collaboration will unlock reinforcement learning at an unprecedented scale in highly complex and rich environments. Agents will be able to discover breakthroughs across all fields of knowledge—from scientific research to robotics, drug discovery, and beyond. As Huang remarked, “We are thrilled to partner with Ineffable Intelligence to codesign the infrastructure for large-scale reinforcement learning as they push the frontier of AI and pioneer a new generation of intelligent systems.”
By combining Ineffable’s deep expertise in reinforcement learning algorithms with NVIDIA’s proven hardware and software stack, this partnership aims to lay the foundation for the next paradigm of artificial intelligence—one where machines truly learn from their own experience.