Google Research: Deep Learning Is an Illusion. The Reality Is “Nested Learning.”

A new paper, “The Illusion of Deep Learning Architectures,” argues that AI’s power doesn’t come from stacking layers, but from nesting learning processes that mimic the human brain.

For the last decade, we’ve been told a simple story about AI: to make models more powerful, you make them deeper.

Stack more layers, add more parameters, and voilà, you get smarter AI. It’s a beautifully simple idea that gave us everything from AlexNet to the massive Transformers powering ChatGPT and Gemini today.

But what if that whole story is just.. an illusion?

What if the “depth” we’ve been chasing isn’t about stacking layers of architecture, but stacking layers of learning processes happening at different speeds?

Hmmm…

A new research paper from Google Research, titled “Nested Learning: The Illusion of Deep Learning Architectures,” is making this exact argument. The authors: Ali Behrouz, Meisam Razaviyayn, Peiling Zhong, and Vahab Mirrokni, are proposing a radical new way to see deep learning.