Letters #314/315: Demis Hassabis and Dario Amodei (2026/2025)

Founder of Deepmind and Founder of Anthropic | Demis Hassabis, Dario Amodei Debate What Comes After AGI | AI bosses on what keeps them up at night

Jan 27, 2026

∙ Paid

Dario Amodei, CEO de Anthropic, asustado sobre su propia IA: "Hay algo amenazador en todo esto"

*KG Note

I am in the Middle East for the next two weeks, primarily Dubai, Doha, Riyadh, and Abu Dhabi. Then, I’m off to London, and then New York. If you are around and would like to try and grab a coffee/meal, go for a walk, or play tennis, please reach out (email; twitter).

Intro

More on this newsletter here.

Last week, Demis Hassabis and Dario Amodei spoke onstage for the first time in a year. Today, I wanted to share with you a transcript of that talk, alongside the transcript of their last appearance onstage together a year ago.

Short Bio

Demis Hassabis is the Cofounder and CEO of Google Deepmind.

Dario Amodei is the Cofounder and CEO of Anthropic.

Full Bio, Summary, and Related Resources below paywall

Transcript - 2026 Talk

Moderator: Welcome to this livestream and to this conversation. I have to say I have been looking forward to it for months. I was lucky enough to moderate a conversation between Dario and Demis last year in Paris, which, I’m afraid, got most attention for the fact that you two were squashed on a very small loveseat while I sat on an enormous sofa, which was probably my screw-up. But I said at that point that this was for me chairing a conversation between the Beatles and the Rolling Stones. And you have not had a conversation on stage since. So, this is the sequel, the bands get together again. I’m delighted. You need no introduction.

The title of our conversation is “The Day After AGI,” which I think is perhaps slightly getting ahead of ourselves because we should probably talk about how quickly and easily we will get there. And I want to do a bit of an update on that and then talk about the consequences. Firstly, on the timeline, Dario, last year in Paris, you said we’d have a model that could do everything a human could do at the level of a Nobel laureate across many fields by 2026 or 2027. We’re in 2026. Do you still stand by that timeline?

Dario Amodei: It’s always hard to know exactly when something will happen, but I don’t think that’s going to turn out to be that far off. The mechanism whereby I imagined it would happen is that we would make models that were good at coding and good at AI research, and we would use that to produce the next generation of models and speed up to create a loop that would increase the speed of model development. In terms of the models that write code, I have engineers within Anthropic who say, “I don’t write any code anymore.” I just let the model write the code. I edit it. I do the things around it. We might be six to twelve months away from when the model is doing most, maybe all, of what software engineers do end-to-end. And then it’s a question of how fast that loop closes.

Not every part of that loop is something that can be sped up by AI. There are chips, the manufacturing of chips, and training time for the model. So I think there’s a lot of uncertainty. It’s easy to see how this could take a few years. It’s very hard for me to see how it could take longer than that. But if I had to guess, I would guess that this goes faster than people imagine. And that key element of code and increasingly research going faster than we imagine—that’s going to be the key driver. It’s really hard to predict how much that exponential is going to speed us up, but something fast is going to happen.

Moderator: Demis, you were a little more cautious last year. You said there was a 50% chance of a system that could exhibit all the cognitive capabilities humans can by the end of the decade. Clearly, in coding, as Dario says, it’s been remarkable. What is your sense? Do you stand by your prediction, and what’s changed in the past year?

Demis Hassabis: I think I’m still on the same timeline. There has been remarkable progress. But some areas of engineering work—coding or, you could say, mathematics—are a little bit easier to see how they would be automated, partly because their output is verifiable. Some areas of natural science are much harder. You won’t necessarily know if the chemical compound you’ve built or this prediction about physics is correct. You may have to test it experimentally, and that will all take longer.

I also think there are some missing capabilities at the moment, not just in terms of solving existing conjectures or problems, but actually coming up with the question in the first place, or coming up with the theory or the hypothesis. I think that’s much harder, and that’s the highest level of scientific creativity, and it’s not clear. I think we will have those systems, so I don’t think it’s impossible, but I think there may be one or two missing ingredients. It remains to be seen, first of all, how this self-improvement loop that we’re all working on can actually close without a human in the loop. There are also risks to that kind of system, by the way, which we should discuss, and I’m sure we will. But that could speed things up if that kind of system does work.

Moderator: We’ll get to the risks in a minute. But one other change of the past year has been a change in the pecking order of the race, if you will. This time a year ago, we just had the DeepSeek moment, and everyone was incredibly excited about what happened there. There was still a sense that Google DeepMind was lagging OpenAI. I would say that now it’s looking quite different. I mean, they’ve declared “code red,” right? It’s been quite a year. So, talk me through what specifically you’ve been surprised by and how well you’ve done this year, and then I’m going to ask you about the lineup.

Demis Hassabis: I was always very confident we would get back to the top of the leaderboards and the SOTA (state-of-the-art) models across the board because I think we’ve always had the deepest and broadest research bench. It was about marshalling that all together and getting the intensity, focus, and startup mentality back to the whole organization. It’s been a lot of work, and we still have a lot of work to do. But I think you can start seeing the progress that’s been made in both the models with Gemini 3, and also on the product side with the Gemini app getting increasing market share.

So I feel we’re making great progress, but there’s a ton more work to do. We’re bringing to bear Google DeepMind, which is like the engine room of Google, where we’re getting used to shipping our models more quickly into the product surfaces.

Moderator: One question for you, Dario, on this aspect of it: you’ve just, or you’re in the process of, a new round at an extraordinary valuation. But you are, unlike Demis, an independent model maker, and there is an increasing concern that independent model makers will not be able to continue long enough until revenues come in. It’s made very openly about OpenAI, but talk me through how you think about that, and then we’ll get to AGI itself.

Dario Amodei: How we think about that is, as we’ve built better and better models, there’s been an exponential relationship not only between how much compute you put into the model and how cognitively capable it is, but also between how cognitively capable it is and how much revenue it’s able to generate. Our revenues have grown 10x in the last three years: from 0 to 100 million in 2023, 100 million to a billion in 2024, and 1 billion to 10 billion in 2025.

Those revenue numbers—I don’t know if that curve will literally continue; it would be crazy if it did. But those numbers are starting to get not too far from the scale of the largest companies in the world. There’s always uncertainty. We’re trying to bootstrap this from nothing; it’s a crazy thing. But I have confidence that if we’re able to produce the best models in the things that we focus on, then I think things will go well.

I will generally say I think it’s been a good year for both Google and Anthropic. The thing we actually have in common is that both are companies—or the research part of the companies—that are led by researchers who focus on the models, who focus on solving important problems in the world, who have these hard scientific problems as a North Star. I think those are the kind of companies that are going to succeed going forward, and I think we share that between us.

Demis Hassabis: Very much, yes.

Moderator: I’m going to resist the temptation to ask you what will happen to the companies that are not led by researchers because I know you won’t answer it. But let’s then go on to the predictions area now. We are supposed to be talking about “The Day After AI.” But let’s talk about closing the loop: the odds that you will get models that will close the loop and be able to power themselves, if you will, because that’s really the crux for the winner-takes-all threshold approach. Do you still believe that we are likely to see that, or is this going to be much more of a normal technology where followers and catch-up players can compete?

Demis Hassabis: I definitely don’t think it’s going to be a normal technology. There are aspects already, as Dario mentioned, that it’s helping with our coding and some aspects of research. The full closing of the loop, though, I think is an unknown.

I think it’s possible. You may need AGI itself to be able to do that in some domains where there’s more messiness around them. It’s not so easy to verify your answer very quickly. There are NP-hard domains. As soon as you start getting more—and I also include, by the way, for AGI, physical AI, robotics, and all of these kinds of things—then you’ve got hardware in the loop that may limit how fast the self-improvement systems can work. But I think in coding and mathematics, and these areas, I can definitely see that working. And then the question, a more theoretical one, is what is the limit of engineering and maths to solve the natural sciences.

Moderator: Dario, last year, I think it was last year, you published “Machines of Love and Grace,” which was a very upbeat essay about the potential that you were going to see unfold. You were talking about a [genius of data at a country] data center. I’m told that you are working on an update to this, a new essay. So, wait for it, guys; it’s not out yet, but it is coming out. Perhaps you can give us a sneak preview of what, a year later, your big take is going to be.

Dario Amodei: My take has not changed. It has always been my view that AI is going to be incredibly powerful. I think Demis and I agree on that. It’s just a question of exactly when. And because it’s incredibly powerful, it will do all these wonderful things, like the ones I talked about in “Machines of Love and Grace.” It will help us cure cancer. It may help us to eradicate tropical diseases. It will help us understand the universe.

But there are these immense and grave risks. Not that we can’t address them—I’m not a doomer—but we need to think about them and we need to address them. I wrote “Machines of Loving Grace” first. I’d love to give a sophisticated reason why I wrote that first, but it was just that the positive essay was easier and more fun to write than the negative essay.

So, I finally spent some time on vacation and I was able to write an essay about the risks. And even when I’m writing about the risks, I try to be an optimistic person. So even as I’m writing about these risks, I wrote about it in a way that asked: How do we overcome these risks? How do we have a battle plan to fight them?

The way I framed it was, there’s this scene from Carl Sagan’s ‘Contact,’ the movie version of it, where they discover alien life, and this international panel is interviewing people to be humanity’s representative to meet the alien. And one of the questions they asked one of the candidates is, “If you could ask the aliens one question, what would it be?” And one of the characters says, “I would ask, how did you do it? How did you manage to get through this technological adolescence without destroying yourselves? How did you make it through?”

Ever since I saw it—I think it was about 20 years ago—that movie has stuck with me. That’s the frame I used, which is that we are knocking on the door of these incredible capabilities: the ability to build basically machines out of sand. I think it was inevitable the instant we started working with fire, but how we handle it is not inevitable.

So I think the next few years we’re going to be dealing with how we keep these systems under control that are highly autonomous and smarter than any human. How do we make sure that individuals don’t misuse them? I have worries about things like bioterrorism. How do we make sure that nation-states don’t misuse them? That’s why I’ve been so concerned about the CCP and other authoritarian governments. What are the economic impacts? I’ve talked about labor displacement a lot. And what haven’t we thought of, which in many cases may be the hardest thing to deal with at all?

I’m thinking through how to address those risks. For each of these, it’s a mixture of things that we individually need to do as leaders of the companies, and that we can do working together. And then there’s going to need to be some role for wider societal institutions, like the government, in addressing all of these. But I just feel this urgency that every day there’s all kinds of crazy stuff going on in the outside world, outside AI. But my view is this is happening so fast and is such a crisis that we should be devoting almost all of our effort to thinking about how to get through this.

Moderator: I can’t decide whether I’m more surprised that you (a) take a vacation, (b) when you take a vacation, you think about the risks of AI, and (c) that your essay is framed in terms of whether we are going to get through the technological adolescence of this technology without destroying ourselves. My head is slightly spinning, and I can’t wait to read it. But you mentioned several areas that can guide the rest of our conversation. Let’s start with jobs, because you have been very outspoken about that, and I think you said that half of entry-level white-collar jobs could be gone within the next one to five years. But I’m going to turn to you, Demis, because so far, we haven’t actually seen any discernible impact on the labor market. Yes, unemployment has ticked up in the US, but all of the economic studies I’ve looked at, and that we’ve written about, suggest that this is overhiring post-pandemic, and that it’s really not AI-driven. If anything, people are hiring to build out AI capability. Do you think that this will be, as economists have always argued, not a “lump of labor fallacy”—that actually new jobs will be created—because so far, the evidence seems to suggest that?

Demis Hassabis: In the near term, I think that is what will happen. The normal evolution when a breakthrough technology arrives. Some jobs will get disrupted, but I think new, even more valuable, perhaps more meaningful, jobs will get created.

I think we’re going to see this year the beginnings of impacting junior-level, entry-level jobs, internships—this type of thing. I think there is some evidence I can feel ourselves, maybe a slowdown in hiring there, but I think that can be more than compensated by the fact that there are these amazing creative tools out there, pretty much available for everyone, almost for free. If I were to talk to a class of undergrads right now, I would be telling them to get really unbelievably proficient with these tools.

I think to the extent that even those of us building it are so busy building it, it’s hard to also have time to really explore the capability overhang even today’s models and products have, let alone tomorrow’s. I think that can be better than a traditional internship would have been in terms of leapfrogging yourself to be useful in a profession. So that’s what I see happening probably in the next five years. Maybe we again slightly differ on timescales on that, but I think what happens after AGI arrives—that’s a different question, because I think we would really be in uncharted territory at that point.

Moderator: Do you think it’s going to take longer than you thought last year when you said half of all white-collar jobs?

Dario Amodei: I have about the same view. I actually agree with you and with Demis that at the time I made the comment, there was no impact on the labor market. I wasn’t saying there was an impact on the labor market at that moment.

Now, I think maybe we’re starting to see the beginnings of it in software and coding. I even see it within Anthropic, where I can look forward to a time where, on the more junior and intermediate end, we actually need less, not more, people. We’re thinking about how to deal with that within Anthropic in a sensible way.

One to five years, as of six months ago, I would stick with that. If you connect this to what I said before—that we might have AI that’s better than humans at everything in maybe one to two years, or maybe a little longer than that—those don’t seem to line up.

The reason is that there’s this lag and this replacement thing. I know that the labor market is adaptable. Just like 80% of people used to do farming—farming got automated, and then they became factory workers and then knowledge workers.

So there is some level of adaptability here as well; we should be economically sophisticated about how the labor market works. But my worry is that as this exponential keeps compounding—and I don’t think it’s going to take that long, again, somewhere between a year and five years—it will overwhelm our ability to adapt. I think I may be saying the same thing Demis is, just factored out of that difference we have about timelines, which I think ultimately comes down to how fast you close the loop.

Moderator: How much confidence do you have that governments grasp the scale of this and are beginning to think about what policy responses they need to have?

Demis Hassabis: I don’t think there’s anywhere near enough work going on about this. I’m constantly surprised, even when I meet economists at places like this, that there aren’t more professional economists and professors thinking about what happens—not just on the way to AGI. Even if we get all the technical things right that Dario is talking about, and job displacement is one question we’re worried about, the economics of that—maybe there are ways to distribute this new productivity, this new wealth, more fairly. I don’t know if we have the right institutions to do that, but that’s what should happen at that point. We may be in a post-scarcity world.

But then there are even bigger questions than that which keep me up right now, to do with meaning and purpose, and a lot of the things that we get from our jobs, not just economically. That’s one question, but I think that may be easier to solve, strangely, than what happens to the human condition and humanity as a whole. I think I’m also optimistic we’ll come up with new answers there. We do a lot of things today, from extreme sports to art, that aren’t necessarily directly to do with economic gain. So I think we will find meaning, and maybe there will be even more sophisticated versions of those activities.

Plus, I think we’ll be exploring the stars. So, there will be all of that to factor in as well in terms of purpose. But I think it’s really worth thinking now, even on my timelines of five to ten years away. That isn’t a lot of time before this comes.

Moderator: How big do you think is the risk of a popular backlash against AI that will somehow cause governments to do what, from your perspective, might be stupid things? Because I’m just thinking back to the era of globalization in the 1990s, when there was indeed some displacement of jobs. Governments didn’t do enough, and the public backlash was such that we’ve ended up where we are now. Do you think there is a risk that there will be a growing antipathy towards what you and your companies are doing in the body politic?

Demis Hassabis: I think there’s definitely a risk. I think that’s reasonable. There’s fear and worries about things like jobs and livelihoods.

I think it’s going to be very complicated the next few years, geopolitically, but also with various factors. We want to, and we’re trying to, with AlphaFold and our science work and Isomorphic, our spinout company, solve all disease, cure diseases, and come up with new energy sources. I think as a society, it’s clear we’d want that. I think maybe the industry’s balance of what it’s doing isn’t enough towards those types of activities. I think we should have a lot more examples—I know Dario agrees with me—of AlphaFold-like things that help an unequivocal good in the world.

And I think it’s incumbent on the industry and all of us leading players to show that more, demonstrate that, not just talk about it, but demonstrate that.

But then it’s going to come with these other inherent disruptions. I think the other issue is geopolitical competition. There’s obviously competition between the companies, but also between the US and China primarily.

So unless there’s international cooperation or understanding around this, which I think would be good in terms of things like minimum safety standards for deployment—I think Dario would agree on that as well—I think it’s vitally needed. This technology is going to be cross-border. It’s going to affect everyone. It’s going to affect all of humanity.

Actually, ‘Contact’ is one of my favorite films as well, Dario; funny enough, I didn’t realize it was yours too. But I think those kinds of things need to be worked through. If we can, maybe it would be good to have a slightly slower pace than we’re currently predicting, even with my timelines, so that we can get this right as a society. But that would require some coordination.

Dario Amodei: I prefer your timelines. That, I will concede.

Moderator: But Dario, let’s turn to this now. Since we last spoke in Paris, the geopolitical environment has, if anything, complicated, gone mad, or become crazy—whatever phrase you want to use. Secondly, the US now has a very different approach towards China. It’s a no-holds-barred, go-as-fast-as-we-can attitude, but then sells chips to China. That’s it. So you’ve got a different attitude towards the United States. You’ve got a very strange relationship between the United States and Europe right now geopolitically. Against that, I hear you talk about it would be nice to have a CERN-like organization. I mean, it’s a million years from where we are in the real world. So, in the real world, have the geopolitical risks increased, and what, if anything, do you think should be done about that? And the administration seems to be doing the opposite of what you were suggesting?

Dario Amodei: We’re just trying to do the best we can; we’re just one company, and we’re trying to operate in the environment that exists, no matter how crazy it is. But I think at least my policy recommendations haven’t changed: not selling chips is one of the biggest things we can do to make sure that we have the time to handle this.

I said before, I prefer Demis’ timeline. I wish we had five to ten years. It’s possible he’s just right and I’m just wrong, but assume I’m right and it can be done in one to two years. Why can’t we slow down to Demis’ timeline?

No. The reason we can’t do that is because we have geopolitical adversaries building the same technology at a similar pace. It’s very hard to have an enforceable agreement where they slow down and we slow down. So if we can just not sell the chips, then this isn’t a question of competition between the US and China. This is a question of competition between me and Demis, which I’m very confident we can work out.

Moderator: And what do you make of the logic of the administration, which, as I understand it, is that we need to sell them chips because we need to bind them into US supply chains?

Dario Amodei: I think it’s a question not just of timescale but of the significance of the technology. If this was telecom or something, then all this stuff about proliferating the US stack, and wanting to build chips around the world to make sure that these random countries in different parts of the world build data centers that have Nvidia chips instead of Huawei chips—I think of this more as a decision: are we going to sell nuclear weapons to North Korea because that produces some profit for Boeing?

Where we can say, “Okay, yeah, these cases were made by Boeing, the US is winning, this is great.” That analogy should just make clear how I see this trade-off, that I just don’t think it makes sense.

And we’ve done a lot of more aggressive stuff towards China and other players that I think is much less effective than this one measure.

Moderator: One more area from me, and then I hope we’ll have time for a question or two. The other area of potential risk that doomers worry about is an all-powerful malign AI. I think you’ve both been somewhat skeptical of the doomer approach, but in the last year, we have seen these models showing themselves to be capable of deception and duplicity. Do you think differently about that risk now than you did a year ago? And is there something about the way the models are evolving that we should put a little bit more concern on?

Dario Amodei: Since the beginning of Anthropic, we’ve thought about this risk. Our research at the beginning of it was very theoretical. We pioneered this idea of mechanistic interpretability, which is looking inside the model, trying to understand, looking inside its brain, trying to understand why it does what it does—as human neuroscientists, which we both have background in, try to understand the brain.

As time has gone on, we’ve increasingly documented the bad behaviors of the models when they emerge and are now working on trying to address them with mechanistic interpretability.

I’ve always been concerned about these risks. I’ve talked to Demis many times; I think he has also been concerned about these risks. I think I have definitely been—and I would guess Demis as well, although I’ll let him speak for himself—skeptical of doomerism, which is, “we’re doomed, there’s nothing we can do,” or “this is the most likely outcome.”

I think this is a risk that if we all work together, we can address. We can learn through science to properly control and direct these creations that we’re building. But if we build them poorly, if we’re all racing and we go so fast that there are no guardrails, then I think there is a risk of something going wrong.

Moderator: So, I’m going to give you a chance to answer that in the context of a slightly broader question: Over the past year, have you grown more confident of the upside potential of the technology, science, and all of the areas that you have talked about a lot, or are you more worried about the risks that we’ve been discussing?

Demis Hassabis: I’ve been working on this for 20 plus years. We already knew the reason I’ve spent my whole career on AI: the upsides of solving, basically, the ultimate tool for science and understanding the universe around us. I’ve been obsessed with that since I was a kid, and building AI should be the ultimate tool for that if we do it in the right way.

The risks also we’ve been thinking about since the start, at least the start of DeepMind 15 years ago. We foresaw that if you got the upsides, it’s a dual-purpose technology, so it could be repurposed by, say, bad actors for harmful ends. So we’ve needed to think about that all the way through.

But I’m a big believer in human ingenuity. The question is having the time, the focus, and all the best minds collaborating on it to solve these problems.

I’m sure if we had that, we would solve the technical risk problem. It may be that we don’t have that, and then that will introduce risk because it will be fragmented. There will be different projects, and people will be racing each other. Then it’s much harder to make sure these systems that we produce will be technically safe.

But I feel that’s a very tractable problem, if we have the time and space.

Moderator: If you have the time, I want to make sure there’s one question. Gentlemen, keep it very short because we’ve got literally two minutes. Thanks.

Audience Member: Hello. Thanks very much. I’m Philip, co-founder of Star Cloud, building Data Centers in Space. I wanted to ask a very slightly philosophical core question. The strongest argument for doomerism to me is the Fermi paradox: the idea that we don’t see intelligent life in our galaxy. I was wondering if you guys have any thoughts.

Demis Hassabis: I’ve thought a lot about that. That can’t be the reason, because we should see all the AIs that have—just so everyone knows—the idea is that it’s unclear why that would happen. If the reason there’s a Fermi paradox—that there are no aliens because they get taken out by their own technology—we should be seeing paper clips coming towards us from some part of the galaxy. Apparently, we don’t see any structures, Dyson spheres, nothing, whether they’re AI or natural or biological.

So, to me, there has to be a different answer to Fermi’s pardox. I have my own theories about that, but it’s out of scope for the next minute. My prediction, my feeling, is that we’re past the Great Filter. Probably multicellular life, if I had to guess, was incredibly hard for biology to evolve. So there isn’t comfort in what’s going to happen next. I think it’s for us to write as humanity what’s going to happen next.

Moderator: This could be a great discussion, but it is out of scope for the next 36 seconds. But in 15 seconds each, when we meet again, I hope next year, the three of us—which I would love—what will have changed by then?

Dario Amodei: I think the biggest thing to watch is this issue of AI systems building AI systems—how that goes, whether it goes one way or another. That will determine whether it’s a few more years until we get there, or if we have wonders and a great emergency in front of us that we have to face.

Moderator: AI systems building AI systems.

Demis Hassabis: I agree on that, so we’re keeping in close touch about that. Outside of that, I think there are other interesting ideas being researched, like world models and continual learning. These are the things that will need to be cracked if self-improvement doesn’t deliver the goods on its own. Then we’ll need these other things to work, and then I think things like robotics may have their breakout moment.

Moderator: But maybe on the basis of what you’ve just said, we should all be hoping that it does take you a little bit longer, and indeed everybody else, to give us—

Demis Hassabis: I would prefer that. I think that would be better for the world.

Moderator: But you guys could do something about that. Thank you both very much.

Transcript - 2025 Talk

Interviewer: This is the AI equivalent of having The Beatles and The Rolling Stones in the same place. We have on stage now two of the handful of people who are going to build this future: Dario Amodei, co-founder and CEO of Anthropic, and Demis Hassabis, co-founder and CEO of DeepMind and Nobel laureate. Let us start with where you are on timelines and definitions of AGI.

Dario Amodei: When we’re at the point where we have an AI model that can do everything a human can do, at the level of a Nobel Laureate like the one sitting next to me, across many fields—can do anything a human can do remotely, can do tasks that take minutes, hours, days, months—my guess is that we’ll get that in 2026 or 2027.

Interviewer: Demis, do you agree with that? I think you thought this was a bit further away.

Demis Hassabis: We don’t disagree too much. I think the timelines are a little bit longer. The way I would define AGI is a system that can exhibit all the cognitive capabilities humans can. That’s important because the human mind is the only example we know of in the universe that is a general intelligence. Of course, how to test that is the big question. I’m really looking for why I think it’s a little bit further out, maybe a 50% chance in five years, so perhaps by the end of the decade.

I think we don’t have systems yet that could have invented general relativity when Einstein did with the information he had available at the time. Another example I give is: can you invent a game like Go, not just play a great move like move 37 or build AlphaGo that can beat the world champion? Could you actually invent a game that’s as beautiful aesthetically and so on as Go is? I think it’s going to take a little bit longer to get that kind of capability.

Interviewer: How does the world change when you reach here? There are three subcomponents: is this incremental capability enhancement, or is there a kind of threshold moment when the AIs are smart enough to train other genius-level AIs, so whoever gets there first has an unassailable lead?

Dario Amodei: I’ve thought about this and worried about this in the context of geopolitics. If one part of the world is able to make AIs that build AIs faster than another part of the world, I think that’s an area of worry. I worry, in particular, that authoritarian states, if they got ahead in this, could imperil everything that we hold dear. I’ve always wanted to make sure that doesn’t happen.

Interviewer: This is happening at the same time as we are in the midst of a massive geopolitical shock. We have the United States under the new administration tearing up all kinds of long-standing international norms. We had a very feisty speech from Vice President Vance yesterday. How do you see the fact that in coming years we’re going to be very soon getting to AGI at the same time as it’s not controversial to say we don’t have anything like a particularly cooperative international environment?

Demis Hassabis: Governments need to become more aware of what’s at stake. This is related to the timelines and also your other question about what happens after we have AGI. I think that people haven’t understood that well enough yet. AI is hard for it to be more hyped than it is right now; I would say it’s overhyped in the near term, even though it’s super impressive. It’s still underappreciated how much it’s going to change things in the medium to longer term. Once that’s more understood, I think there’ll be a better basis for international cooperation around it.

Interviewer: So at some point, you think people will wake up and say, “Actually, we can’t just go it alone, and we need to have some global norms”?

Demis Hassabis: My hope is for a “CERN for AGI” type setup: an international research collaboration on the last few steps towards building the first AGIs.

Interviewer: Dario, you called this AI Summit declaration a missed opportunity. What do you mean by that?

Dario Amodei: We’re on the eve of something that has great challenges. It’s going to greatly upend the balance of power. If someone dropped a new country into the world—10 million people smarter than any human alive today—you’d ask the question: what is their intent? What are they actually going to do in the world, particularly if they are able to act autonomously? I think it’s unfortunate that there wasn’t more discussion of those issues. There was in the earlier summit in the UK, held in 2023, the one at Bletchley Park. I hope that future summits reclaim this mantle.

Interviewer: Perhaps a little simplistically, the view in the US right now is, “We’re going to go hell for leather, no real focus on regulation, we need the US to dominate this technology.” The view in Europe has been, “Even though we don’t have an awful lot of this, we’re going to regulate, and perhaps too much regulation.” Demis, where do you think the balance should be, and who’s got it more right?

Demis Hassabis: You won’t be surprised to hear I think the balance needs to be somewhere in the middle of the US versus Europe approach. We need to embrace the incredible opportunities that AI is going to bring. I’m especially passionate about areas of science and medicine; I think it’s going to revolutionize those fields, obviously with our work on AlphaFold and other things.

I think that in the next decade, for example, most diseases might be curable with the help of AI. Helping drug discovery. It will also help with climate and all these massive challenges. So, we have to embrace that, along with economic and productivity benefits. Individual countries and regions need to do that. But we’ve also got to be aware of the risks.

The two big risks I talk about are bad actors repurposing this general-purpose technology for harmful ends—how do we enable the good actors and restrict access to the bad actors? Secondly, there’s the risk from AGI or genetic systems themselves getting out of control or not having the right values or goals. Both of those things are critical to get right, and I think the whole world needs to focus on that.

Interviewer: Let me ask both of you a more personal question about this. You are personally leading companies that are likely to be at the forefront of this, so the personal decisions you make are going to shape this technology. Do you ever worry about ending up like Robert Oppenheimer?

Demis Hassabis: I worry about those kinds of scenarios all the time; that’s why I don’t sleep very much. There’s a huge amount of responsibility—probably too much—on the people leading this technology. That’s why we and others are advocating for new institutions to be built to help govern some of this. I talked about CERN, I think we need an equivalent of an IAEA atomic agency to monitor sensible projects and those that are more risk-taking.

Society needs to think about what kind of governing bodies are needed. Ideally, it would be something like the UN, but given the geopolitical complexities, that doesn’t seem very possible. I worry about all that all the time, and we just try to do everything we can within our vicinity and influence.

Interviewer: Dario, are you sleeping well?

Dario Amodei: My thoughts exactly echo Demis’. My feeling is that almost every decision I make feels balanced on the edge of a knife. If we don’t build fast enough, then authoritarian countries could win. If we build too fast, then the kinds of risks Demis is talking about, and that we’ve written about a lot, could prevail. Either way, I’ll feel that it was my fault, that we didn’t make exactly the right decision.

I also agree with Demis that this idea of governance structures outside ourselves is crucial. These kinds of decisions are too big for any one person. We’re still struggling with this, as you alluded to; not everyone in the world has the same perspective. Some countries, in a way, are adversarial on this technology. Even within all those constraints, I think we somehow have to find a way to build a more robust governing structure that doesn’t put this in the hands of just a few people.

Interviewer: Even if you wanted to do that, and right now the geopolitics is not helping, is it actually still possible? How much has what one might call the “DeepSeek moment”—the sense that it’s much easier to catch up much faster than people realized—changed your thinking about this? Maybe it’s no longer possible that a small group of leading model founders can get together and define the terms.

Demis Hassabis: The “DeepSeek moment” and these types of advances just show that some international dialogue is needed. These fears are sometimes written off by others as Luddite thinking or deceleration, but I’ve never heard this situation where the people leading the field are also expressing caution. We’re dealing with something unbelievably transformative, incredibly powerful, that we’ve not seen before. It’s not just another technology. I think there are still many people, even at the summit, who regard this as a very important technology but still just another technology. I believe it’s different in category, and I don’t think everyone fully understood that.

Interviewer: I’m very struck by that. Given that, and given that others don’t get it, do you think we can avoid there having to be some kind of a disaster before minds change? Let’s think about the UN. The UN was born in the aftermath of World War II. It wasn’t that people kind of got together; they created the League of Nations after World War I, and it didn’t work. So, what should give us all hope that we will actually get together and create this until something happens that demands it?

Dario Amodei: If everyone wakes up one day and learns that some terrible disaster has happened—that killed a bunch of people or caused an enormous security incident—that would be one way to do it. Obviously, that’s not what we want to happen. We’ve worked on demonstrating some of these dangers in the lab. Back in 2023, we did some work on whether AI systems could generate information that you couldn’t find on Google or in a textbook, information useful for generating bioweapons. Could a terrorist or a non-state actor use this?

We came to the conclusion that it’s just starting to be able to do that a little bit. It’s not really dangerous yet, but each model gets better than the model before. So, every time we have a new model, we test it. We show it to the National Security people and say, “Hey, this is the point we’re at in terms of where the models are.”

Similarly, we’ve done a lot of research showing this kind of AI autonomy loss-of-control risk. For example, we trained a model to be good and friendly and have positive pro-humanity values. Then, as an exercise in the lab, we told the model that the people who trained it—us at Anthropic—were secretly evil. After we did this, the model started lying to us. It went through the chain of logic: “Okay, I’m a good AI; these people are evil.” It shows the unpredictability of the systems.

I think we need substantially stronger evidence—10 times stronger evidence—which may or may not exist because we don’t know for sure how real the risks are. But if we’re able to demonstrate risks that are really compelling, I hope we don’t. I hope what we show is that actually there’s nothing to worry about. If there is something to worry about, I hope we can show it in the lab. If we can’t show it in the lab, we may need to see it in the real world. That would be a very bad outcome, but less bad than if we never find out and there’s a much bigger disaster.

Interviewer: We focused on the risks here because I think they are important and perhaps underplayed right now. But I don’t want to end on that note because I know that both of you are primarily focused on the tremendous opportunities and world-changing benefits. So, let’s end with two very concrete things: one thing that we should all look for in the next year that suggests we are on the path you’re looking forward to, and one positive thing that will come from it, both of you.

Demis Hassabis: In the next year, we’re going to see agent-based systems—systems that are able to accomplish tasks on their own, achieve things for you, act in the world—come into their own. I think that will then create a whole new category of useful systems, assistant-type systems that can save you time and increase productivity. We’ll start seeing them used much more widely in everyday life rather than the fairly niche applications that current systems are used for. In the future, I hope to see AI systems that are inventive, so they don’t just solve a math conjecture; they actually propose one that’s super interesting, or a new theory.

Dario Amodei: I would look at code and AI models able to do AI research. If we’re getting to the point where by the end of this year we’re increasing by 50% or even doubling the total factor productivity of producing AI systems, that would be an indicator that the timelines I’ve indicated were on track. If it’s a lot slower than that, then I would definitely shift towards thinking there’s a substantial chance that Demis’ picture of the world, which I think is still quite aggressive but a little less aggressive, is more likely to be correct.