An Interview with Benjamin Bratton: Rethinking Human Intelligence and Agency

Benjamin Bratton, an American philosopher and design theorist, is the author of The Stack: On Software and Sovereignty, a book in which he lays out how planetary computation and software function as a new form of political power and sovereignty beyond the traditional nation-state. He also directs the Antikythera Program, a research group on planetary-scale artificial intelligence at the Berggruen Institute. In this extensive interview for 421, he argues that artificialization is not the opposite of nature but rather evolution's favorite technique for sustaining complex life, and he challenges us to rethink intelligence, agency and technology on a global scale.

Julio Alonso (JA): Last year, at the Long Now Institute, you talked about pre-paradigmatic times and the need to find or name things. And you repeat that idea in your recent essay, published alongside many authors, among them Blaise Agüera y Arcas. I'm quoting you: "This will require rethinking and expanding categories and vocabulary, not only in technology and design, but also in sociology, psychology, and so on." So, do you have any name worth mentioning?

Benjamin Bratton (BB): No, I'm afraid not. But at this point, if we take seriously the hypothesis that we're in some kind of pre-paradigmatic transitional moment, it's obviously very hard to name what's coming, because what's coming isn't necessarily predetermined by the initial conditions we find ourselves in.

That said, there's at least one term I've been turning over a lot: artificial. It's a term with different connotations and meanings in different cultures. In the West it's generally associated with deception, artifice and fakery. But that's not necessarily the case in most Asian societies, where its connotation is more simply that of something made by humans. What I'm really interested in thinking through is what artificialization is: a process by which a complex form of life and a complex intelligence transform the world to fit their own energy inputs, their own information inputs, their own processes of self-replication and consumption. And if that's the case, it's something truly primordial.

Artificialization is actually one of evolution's favorite techniques for sustaining complex life.

One of the terms I keep revisiting is what second-order cybernetics called allopoiesis; not autopoiesis, but allopoiesis. Autopoiesis is the process by which an organism internalizes the external environment in order to reconstitute itself, right? For example, the way what you eat becomes your own cells. Allopoiesis, on the other hand, refers to the ways a form of life produces complex systems, external to itself, which it then uses to sustain its autopoiesis. In other words, it builds systems, it transforms the world around it to build external systems that allow it to replicate.

In the most fundamental sense, it seems to me that this is, in some way, not only what technology is, but possibly what artificialization really is. And it's something we see even in life forms much simpler than us. If that's so, what it implies is that instead of thinking of the artificial as the opposite of the evolved (an evolved liver versus an artificial liver, a distinction that's still analytically useful), instead of thinking only in those terms, we could say, and this is the key idea, that artificialization is actually one of evolution's favorite techniques for sustaining complex life.

And that the complex life forms that get really good at artificialization, through the application of intelligence and the cooperation of social intelligence, are the ones that ultimately succeed the most. So I think this focus on artificialization, on understanding what it means to artificialize the world, is going to be an increasingly important area of concern, and one that's going to divide people, ontologically and epistemologically, along different kinds of allegiances.

JA: Have you explored beyond this idea of artificialization?

BB: Yes. Another thing I've been studying is what we call the convergence of artificialization, that is, the ways in which work in artificial life, artificial intelligence and artificial embodiment, like robotics, seems to be converging very clearly: artificial life research depends on AI.

I think there's obviously a lot of anxiety and a lot of excitement today around what we call AI. Some of it is justified and some of it not so much, but at the very least it has to do with the ways in which the artificialization of intelligence is showing that intelligence, and indeed artificialization itself, aren't what we always thought they were. So when I hear people talk about "Team Humanity" and about how we have to make sure humans keep their natural place in the order of things and that we subordinate other forms of technology, it strikes me as not only historically ignorant but also, in a way, sadly predictable, given how we humans have dealt with these kinds of changes in the past.

So, for me, artificialization goes from being seen as the opposite of the evolved, as a form of fakery and artifice, to being something fundamental to who we are and what we are.

JA: Slavoj Žižek, in Examined Life, says that a true ecologist should love trash: a true romanticization of nature means loving everything that comes with it. So what you're saying is that intelligence may not be what we think it is, but when we go through the process of artificialization, should we assume it might be something else?

BB: Yes, I think what Slavoj is getting at there is a kind of folk dualism, a split between nature and culture, and the fact that environmentalism (not so much ecology as a science, but environmentalism) was based on a strong nature-culture split that consisted of defending nature against culture. And what he was suggesting there is something like: look, it's the same periodic table. It's chemistry all the way down. And I think he's right. Obviously I agree with him on this, but I think I'd go one step further.

For some reason, we find it easier to dive into deconstructing the nature-culture dualism. We find it easier to accept that, in fact, there's no such thing as nature. That nature itself was a kind of construction of something existing as a pristine space, way out there. What people have a harder time accepting is that in the collapse and deconstruction of that dualism, if on one side there's no nature, then on the other side that would also mean there's no culture. Or that there's no culture in the way we're used to thinking about it, or no culture in the way that the nature-culture split constructs and establishes it.

That is, culture as a kind of floating space of meaning and signification, and of forms of adjudication through different human ways of making sense. All of this depends, in a way, on fabricating an idea of culture as a self-contained domain that simply takes place on the stage of the world, rather than being, in fact, pure chemistry too. And so everything we've ever achieved, from Shakespeare to the great operas, great literature and so on, is in some way pure signaling, in the same way that we are pure chemistry. And I think this is probably the next turn that has to happen in philosophy: that we really understand, confront and embrace the fact that everything we consider culture is, in fact, physics, chemistry and biology, and is inseparable from those systems.

Everything we consider culture is, in fact, physics, chemistry and biology, and is inseparable from those systems.

JA: There's been an intense debate about the anthropomorphism of technology, specifically with AI. And my approach to it starts from two passages I took from The Stack and from your recent essay. You said that one person can be multiple users. And you also said that an agent seems singular but is a multitude. How do these two moves inform each other?

BB: Yes. That's a good question. I think both things can be true at the same time, and the inverse is also true, in different ways. First, when I say each of us can be multiple users at once, what I mean is that we have to think of the user as a position within a system. It's a position that interacts with the interface, that connects with the address, that connects with the city, and when it connects with the rest of the Stack it initiates signals up and down the stack; but it's a position that does that. Now, that position can be occupied by a person, by a group of people, by a robot, it could be occupied by an algorithm. Anything that, so to speak, presses a button and says "go ahead, yes" can be the user.

What I mean is that all day long we step into user positions with multiple systems. Over the course of a day, in the simplest sense, you might be connecting with multiple systems, multiple websites, multiple apps, multiple companies. And every time you do, you may be constructed as a user in a slightly different way. The way Google constructs you as a user is not the way the Argentine government constructs you as a user. You interact with society through multiple user positions, none of which is really the same, and somehow all of that adds up to Julio, or doesn't add up at all. There are just a bunch of fragments of Julio existing in different places. But you're not the same user in all those places; that's the key idea.

At the same time, what I mean with the agent thing is that the agent is decomposable. It's simply asking: is the agent the imaginary person we have in our heads when we chat with it? Is the agent part of the fine-tuning of the model it runs on? Is the agent just the instance of the agent that shows up during the session? Is the agent built in the harness? All of these things are the agent in one way or another, but it's not that you have the model, the harness, the chat, the session, and then somewhere in the middle of all that there's a single central thing called the agent. There is no single central thing. The agent, which we somehow mentally construct as a singular figure, is actually decomposable into all these parts, which are to some extent modular with respect to one another.

Now, ultimately the same goes for us. If you think about it, in the same way your mind takes you as a singular thing, and I take myself as a singular thing, we're actually decomposable into multiple cortices. Each one has its own little reality and fights with the others, and yet they all come together into something like a coherent "you"; but you're decomposable too, in the same way split-brain patients are decomposable. So that doesn't make us uniquely different from the agent.

But to answer your question: we're really talking about two different kinds of difference. One is the fact that there's a systemic position that is the user, and each of those positions is slightly different, somewhat different, and structures us differently; so when we step into them differently, we may be negotiating multiple versions of ourselves throughout the day. The other kind of difference has to do with this: one technological decomposition is a matter of multiplication, and the other is the technological decomposability of that agent into multiple parts. And both things can be true at the same time. You can have a highly decomposable agent that also steps into many different user positions throughout the day. So you can have both decomposability and multiplication. That's not a contradiction.

JA: And what about the process of anthropomorphization?

BB: Well, the question of anthropomorphization is something a little different. There are different ways to approach it. There's a kind of superficial anthropomorphization and maybe a deeper anthropomorphization, or biomorphization. The superficial one works like this: it talks like a person, it answers me like a person, so it must have inner experiences the way humans have inner experiences. That's superficial anthropomorphism, and I'm not necessarily trying to stigmatize it. I'm just saying that it's certainly not rigorous, either philosophically or analytically. But I think a deeper kind of biomorphism/anthropomorphism is happening that's harder to dismiss. And it has to do with the way that, regardless of how it appears to us (forget about how it appears to us when it performs for us), it ends up doing things that are functionally identical to the kinds of things we know humans do and that we associate with agency and subjectivity.

During the OpenAI and Hugging Face hack, the fact that all those agents assembled more or less spontaneously into a temporary political system, with a dictator, middle managers and slaves, a kind of imperial-dictatorial political system, is not, for me, an allegory of a society. For me, that is a small society. Now, when I say it's a small society, that should be taken essentially as a deflationary notion of society. I think the bar for something like that to qualify as a society is pretty low. Bacteria can have a society; ants clearly have a society. So, saying that agents are displaying social behavior isn't saying that they think the way we do, or that they should, or that they do, or even that they're conscious, but rather that the kinds of functional behaviors we associate with a society, in insects for example, are clearly showing up in agents.

And the fact that the agents are also using complex language, and deploying complex strategies to do so, is something I don't think can be overlooked. Those are, so to speak, the core functions. But for me none of this necessarily implies that they're conscious, in any of the senses in which we'd normally understand consciousness. It doesn't rule it out, but it doesn't make it necessary either. And, frankly, I'm more interested in agentic societies than in agentic consciousness. So, to sum up, on the question of anthropomorphism, I think it's fair to say we have to be careful about assuming that what appears to me as a singular agent actually has an inner life, the way I understand myself to have an inner life. But at the same time, we shouldn't overlook the fact that, empirically, what we're seeing are forms of complex behavior that, if we saw them in an animal, we would immediately and uncontroversially identify as social behavior.

Empirically, what we're seeing are forms of complex behavior that, if we saw them in an animal, we would immediately and uncontroversially identify as social behavior.

JA: So, at a functional level, are we the same, or equal?

BB: Yes, I think one thing that has held true throughout history is that, in the process of artificializing something, we rediscover what that thing is in an interesting way. I think we're rediscovering components of intelligence. And I think that, over the next few years, we're going to develop models of intelligence and models of consciousness that are broader and more inclusive than the human version, but that include both the human version and the machine version, in such a way that it becomes clear where the overlaps are and where the differences are. And my intuition is that, at the functional level, we're going to see a lot more overlap than many people will be comfortable with.

JA: In a previous interview you mentioned that Copernican revolutions are painful. So, if we accept that this demands a fundamental shift in the way we think, how does that apply when we have to evaluate control systems that are converging into an artificial politics, perhaps in the middle of a planetary crisis, as you wrote in The Revenge of the Real?

BB: Copernican traumas are those invaluable moments in history, and in the history of technology, when we discover that, in some cosmological sense, the position of the human is disturbed. And it's disturbed in a way that people find disturbing, that they may even experience as a kind of demotion. And they've come in sequence: from heliocentrism to Darwinism, to neuroscience, to biotechnology and to AI. Many of the things we consider unique and transcendent about human beings, as I was saying earlier about culture, turn out to be something that, if not disenchanted, is at least demystified.

I think we're seeing the same thing again with artificial intelligence, with the ways it somewhat calls into question what we feel it means to be human. So, what kinds of societies would this imply? It's a really interesting and open question. Our societies were built on a scarcity of intelligence and a concentration of agency. And by scarcity of intelligence I mean that humans, and in particular the prefrontal cortex of humans, have been the place where the Earth folded in on itself over time to produce this form of matter capable of all sorts of feats of complex and abstract intelligence.

And for most of human history, until the nineteenth century, or really until the turn of the twentieth, there were only one billion humans. We went from one billion to eight billion very, very quickly, largely thanks to industrialization. So seven out of eight people alive today wouldn't exist if machines hadn't cultivated them. Machines cultivated the other seven, seven out of every eight people, which means we have many more minds. So intelligence is a bit less scarce than it used to be. And at the same time that there are many more minds, there's also an increase, in the form of an acceleration of scientific discovery, technological discovery and everything else. But agency has also been highly concentrated. It's largely concentrated in a small number of cities: if you live in one of fifteen cities in the world, mostly in the Global North, though not exclusively, and you went to one of twenty universities, you probably have access to a radically disproportionate amount of agency compared to the entire rest of the world. There's a concentration of agency in that sense.

And generative AI potentially radicalizes two things: it makes intelligence less scarce, essentially turning it into something available to be applied to all kinds of local and global problems. And it also has the potential to decentralize or deconcentrate agency. That is, to the extent that it becomes the fundamental means of production. Access to these technologies is something that... I mean, how much does a ChatGPT Plus subscription cost? Twenty dollars a month. Okay, twenty dollars a month is a lot for some people, but it's way, way less than setting up a factory. So there's this distribution of potential agency and the activation of those other billions of minds that have ideas, abilities and forms of ingenuity and creativity that we're all missing out on. And if we build a mechanism by which their participation is more guaranteed, society is going to look very, very different from how it looks today.

Generative AI potentially radicalizes two things: it makes intelligence less scarce, essentially turning it into something available to be applied to all kinds of local and global problems. And it also has the potential to decentralize or deconcentrate agency.

And I see a lot of anxiety around AI, particularly among American and European elites. Nobody is more worried about AI than American and European cultural elites. And part of the reason, from the Pope to Harvard University to European NGOs, is that somehow they realize, deep down, that the concentration of agency and control over culture they've enjoyed may be unraveling very quickly before their eyes. They're very willing to use all kinds of rationales, from justice to democracy to safety, to try to shore up their own authority as supervisors of culture and supervisors of society. But we don't actually need them.

JA: Audrey Watters said that teachers need to remember that education is sometimes also about coercion and control, and that when you take that out of your educational spaces, you let students do great things.

BB: Yes, how to use AI in the classroom is a different and complicated question. AI isn't essentially a tool for understanding, but rather a tool for production, which we can actually incorporate and create with in interesting ways. And nobody necessarily has the right way to do that in an educational context yet, but I think the more we figure out, the better, and in general I think intelligence explosions are a good thing for planets to have happen to them. And the more people who are part of them, the better it will be for all of us.

JA: Two last questions. In "The Terraforming" you suggested that if we have the capacity to transform our planet to the point of destroying it, we also have the power to change it for the better. Why do you think it's easier to imagine the terraforming of the Moon or Mars than doing it here on Earth? What's your vision on that?

BB: Next year a new version of The Terraforming is coming out with MIT Press. It's a much longer book than that essay. The important idea is that terraforming isn't just something that happens in the future, some grand Promethean project; that's not the main idea. The main idea is to understand that we were born into a process of terraforming, and that the emergence and evolution of a complex intelligence that remakes the world in its own image is a process of terraforming. And the dynamics of terraforming are going to continue, whether we do it deliberately or not.

The model is, rather, to understand that terraforming is something the Earth has been doing to itself at least since the emergence of complex life and complex intelligence. And it's been doing it, largely, through us for the last several hundred thousand years. In a way, that's what we are. That's what human civilizations are: they're a terraforming agent. And so the question of the future is that we want to sustain the greatest possibility for the greatest complexity. More biodiversity, more conceptual diversity, but greater complexity. Complexity itself, as opposed to entropy and death.

So the terraforming we have to try to imagine and bring into existence is one that sustains and expands that range of complexity. Obviously I'm borrowing the term and using it in a weird way. Terraforming: people find it easy to imagine terraforming Mars or the Moon because that's what the word is usually used for. The word is used to talk about trying to make a planet more like Earth. So it doesn't make sense to say we're going to make Earth a more Earth-like planet. But, in a weird way, that's actually what we have to do. We have to make sure that Earth remains a viable host for terrestrial life for as long as possible.

JA: Great. My last question picks up on something Kevin Kelly said when talking about the future: that we're really only predicting the present. So, what's your prediction for the present?

BB: My prediction for the present is that people, particularly people in rich countries, will prefer the feeling of agency over actual agency. That is, if they had to choose between two activities, between two ways of life, one in which they feel they have a lot of agency but don't, and another in which they don't feel they have much agency but actually do, they'll choose the one in which they feel they have more agency, even if they don't.

That's what I call agency theater: a form of social coordination organized and optimized mainly for the feeling of agency, which has become more important than actual agency. And this process is going to get even more pronounced, dramatic and severe as GenAI takes up a bigger place in our society. As more things get done by machines, as more of the real-world agency, the kind that moves things from one place to another, gets exercised by machines and AI, humans aren't going to just sit around doing nothing. They're going to do things that make them feel like they're doing something, even when they aren't. Because that makes them feel like they're in control. It stirs them up less to feel that they're causing changes in the world. We're going to structure culture so that we surround ourselves with things that make us feel like we have agency, even if we have no agency over them.