✨🔬 Acceleration though AI-automated R&D: My chat (+transcript) with researcher Tom Davidson

My fellow pro-growth/progress/abundance Up Wingers in America and around the world:

What really gets AI optimists excited isn’t the prospect of automating customer service departments or human resources. Imagine, rather, what might happen to the pace of scientific progress if AI becomes a super research assistant. Tom Davidson’s new paper, How Quick and Big Would a Software Intelligence Explosion Be?, explores that very scenario.

Today on Faster, Please! — The Podcast, I talk with Davidson about what it would mean for automated AI researchers to rapidly improve their own algorithms, thus creating a self-reinforcing loop of innovation. We talk about the economic effects of self-improving AI research and how close we are to that reality.

Davidson is a senior research fellow at Forethought, where he explores AI and explosive growth. He was previously a senior research fellow at Open Philanthropy and a research scientist at the UK government’s AI Security Institute.

In This Episode

* Making human minds (1:43)

* Theory to reality (6:45)

* The world with automated research (10:59)

* Considering constraints (16:30)

* Worries and what-ifs (19:07)

Below is a lightly edited transcript of our conversation.

Making human minds (1:43)

. . . you don’t have to build any more computer chips, you don’t have to build any more fabs . . . In fact, you don’t have to do anything at all in the physical world.

Pethokoukis: A few years ago, you wrote a paper called “Could Advanced AI Drive Explosive Economic Growth?,” which argued that growth could accelerate dramatically if AI would start generating ideas the way human researchers once did. In your view, population growth historically powered kind of an ideas feedback loop. More people meant more researchers meant more ideas, rising incomes, but that loop broke after the demographic transition in the late-19th century but you suggest that AI could restart it: more ideas, more output, more AI, more ideas. Does this new paper in a way build upon that paper? “How quick and big would a software intelligence explosion be?”

The first paper you referred to is about the biggest-picture dynamic of economic growth. As you said, throughout the long run history, when we produced more food, the population increased. That additional output transferred itself into more people, more workers. These days that doesn’t happen. When GDP goes up, that doesn’t mean people have more kids. In fact, the demographic transition, the richer people get, the fewer kids they have. So now we’ve got more output, we’re getting even fewer people as a result, so that’s been blocked.

This first paper is basically saying, look, if we can manufacture human minds or human-equivalent minds in any way, be it by building more computer chips, or making better computer chips, or any way at all, then that feedback loop gets going again. Because if we can manufacture more human minds, then we can spend output again to create more workers. That’s the first paper.

The second paper double clicks on one specific way that we can use output to create more human minds. It’s actually, in a way, the scariest way because it’s the way of creating human minds which can happen the quickest. So this is the way where you don’t have to build any more computer chips, you don’t have to build any more fabs, as they’re called, these big factories that make computer chips. In fact, you don’t have to do anything at all in the physical world.

It seems like most of the conversation has been about how much investment is going to go into building how many new data centers, and that seems like that is almost the entire conversation, in a way, at the moment. But you’re not looking at compute, you’re looking at software.

Exactly, software. So the idea is you don’t have to build anything. You’ve already got loads of computer chips and you just make the algorithms that run the AIs on those computer chips more efficient. This is already happening, but it isn’t yet a big deal because AI isn’t that capable. But already, one year out, Epoch, this AI forecasting organization, estimates that just in one year, it becomes 10 times to 1000 times cheaper to run the same AI system. Just wait 12 months, and suddenly, for the same budget, you are able to run 10 times as many AI systems, or maybe even 1000 times as many for their most aggressive estimate. As I said, not a big deal today, but if we then develop an AI system which is better than any human at doing research, then now, in 10 months, you haven’t built anything, but you’ve got 10 times as many researchers that you can set to work or even more than that. So then we get this feedback loop where you make some research progress, you improve your algorithms, now you’ve got loads mor