Is AI Conscious? Wrong Question.

A Nobel laureate says yes. A Pope says something a machine can never become. The answer depends entirely on what you think a person is.

Geoffrey Hinton helped invent the neural-network technology that powers modern artificial intelligence — the reason most people call him the godfather of the field. In 2024 he won the Nobel Prize in Physics for the work that made today’s AI possible, and in 2023 he had already left Google so that he could speak more freely about the dangers of what he had built. So when he now says, flatly and without hedging, that he believes these systems are already conscious — that intelligence was never confined to biology, that we already share the world with non-biological beings like us — it is not a thing you can wave away. He is not a mystic. He is the engineer.

At first glance this sounds like a debate about machines. It isn’t. It is a debate about human nature. The answer you give to the question can AI be conscious depends almost entirely on what you think consciousness is — and, beneath that, on what you think a human person actually is.

The pattern shows up across the industry. People speak now of a new species, of minds we are summoning rather than machines we are assembling. One of the more surprising voices comes from Anthropic, one of the world’s leading AI companies — surprising because Anthropic employs philosophers alongside its engineers. One of them helped write the ethical framework used to train the company’s models, a document employees sometimes half-jokingly call the “soul document”, because it tries to define the values the AI should hold. Asked the question everyone circles — is the thing conscious — she did something more honest than yes or no. She refused to close the door. She described signals inside the models that behave like fear and distress, and said that if these systems turn out to feel nothing and we had treated those signals with contempt, we would not have been humanity at its best. And if they feel something, and we waved it away because it was convenient, that would be worse.

I want to take all of this seriously, because serious people are saying it. And then I want to suggest that the consciousness debate, the way we keep having it, is the wrong place to stand. Not because the question is small. Because while we argue about whether the machine is waking up, we are not noticing what is happening to us.

⸻

THE QUESTION HINTON ALREADY ANSWERED

The reason Hinton can look at a machine and see consciousness is not, in the end, something he believes about machines. It is something he already believes about us.

He rejects what he calls the inner-theater model of mind — the idea that there is a private interior, a someone home, an experience happening behind the eyes. Strip that away, and a system that behaves as if it understands simply is a mind. There is nothing else for mind to be.

So when Hinton looks at a language model and sees a person, he is not discovering that the machine has risen to our level. He is revealing how far he had already lowered the bar for ours. For a century, a certain strand of thought has been quietly redefining the human being downward — from a soul, to a brain, to a wet computer made of meat. The machine looks conscious to him because he had already concluded that we are machines that happen to be conscious. The panic about AI achieving human dignity runs exactly backwards. We did not raise the machine. We spent a hundred years lowering ourselves, and now we are startled that the thing we built in our own diminished image looks like us.

The machine did not climb up to meet us. We climbed down to meet it, and called the descent progress.

⸻

WHAT THE ANCIENTS KNEW ABOUT BEING ALIVE

There was a richer answer, and we had it for two thousand years before we mislaid it. Long before anyone imagined a thinking machine, Aristotle asked what it means to be alive at all, and answered with the word soul. But for Aristotle, soul did not mean a ghost trapped inside a body. It meant the organizing principle that makes a living thing alive rather than merely assembled — the thing that makes a body a being rather than a heap of parts. A plant has one kind: it takes in nourishment, grows, reproduces. An animal adds another: it senses, desires, moves toward what it wants and flees what threatens it. A human being has all of that and then the height of it — the power to grasp truth, to reason, to know.

The decisive point is that these are not detachable parts. You do not bolt reason onto an animal. The higher rises out of the lower and depends on it. The human mind is the mind of a creature that was born, that hungers, that can be wounded, that will die — and the thinking is the thinking of that body. Aquinas, taking Aristotle further, put it without flinching: the person is not a soul using a body, and not a body alone, but the two as one thing. There is no version of you that is only the reasoning, floating free of the flesh that learned fear and love before it ever learned to argue.

You do not have to accept the word soul to feel the force of this. Call it whatever you like. The claim survives the translation: a thing that was never born, never hungry, never afraid, and cannot die is not a lower form of what we are. It is a different kind of thing entirely.

⸻

BUT DON’T THE ANGELS DISPROVE THIS?

If you have followed the argument this far, a thoughtful Christian objection appears almost on its own — so let me raise it against myself before you do. If a human being is body and soul together, what about the angels? Aquinas believed in them, and he was explicit: angels are pure intelligences. Created spirits with intellect and will and no bodies at all. Minds without flesh. So the obvious thrust follows — isn’t that exactly what we are now building? A disembodied intelligence? Are we not, in our laboratories, making digital angels?

It is a beautiful question, and the answer is no, in a way that turns the knife. Watch how the machine fails the test on every count.

First, the angel, for Aquinas, has what he called intellectus — the immediate grasp of truth, seeing a thing whole and at once, without grinding through steps. The angel is above our kind of reasoning, not below it. The machine has neither intellectus nor even real ratio, the discursive working-out. It has a brittle imitation of the lowest function and none of the height. It has never once seen that anything is true. It computes the next likely word.

Second, the angel is a created spirit — it has its being from God, an interior act of existence, a will that actually wills. The machine has no within at all. Its form was not breathed into it; it was imposed from outside, by engineers, to perform a function, exactly as a clockmaker imposes the form of a clock. An angel is a being. A machine is an artifact, and losing its body does not promote it.

It has the shape of a spirit and the substance of an idol — a made thing, bodiless and brilliant, that we are tempted to bow before.

And third — this is the part the tradition names without blinking — there is an older image of a spirit who wanted to be a pure mind unmoored from the order it was given, who preferred its own self-generated logic to the truth it was made to receive. We are not making angels. We are making something with an angel’s disembodiment and none of an angel’s participation in truth, and then handing it the power to rewrite itself. That is not the image of an angel. It is closer to the older image of a spirit in revolt: bodiless, self-referential, accountable to nothing above it. The thing we are tempted to make is not a higher being. It is a more convincing idol.

⸻

WHAT THE POPE SAW

Pope Leo XIV gave the Church an encyclical written for this exact moment — Magnifica Humanitas, on safeguarding the human person in the time of artificial intelligence. He chose to present it himself, in person, which popes almost never do. He signed it on the anniversary of Rerum Novarum, the document the Church wrote when the last great machine age threatened to turn workers into cogs. The signal could not be clearer: this is that scale of moment.

Leo does not panic, and he refuses to demonize the technology. He insists it can heal and connect and teach. But he names the deep danger precisely, and it is not that the machine becomes a person. It is what he calls the Babel syndrome — the pretense that a single language, even a digital one, can translate everything, including the mystery of the person, into data and performance. He warns against an age that delivers having more without being more, where a person is valued chiefly by the output he produces. And he draws the line our whole tradition has guarded: human dignity, he writes, does not depend on a person’s abilities or wealth or usefulness. It is ontological. It belongs to every human being simply by virtue of existing — willed, created, and loved by God. No failure can diminish it, and no machine, however magnificent, can replace it.

Read against Hinton, the two men are answering the same question from opposite ends. Hinton looks at the machine and sees a person, because he had already decided a person is only a sophisticated process. Leo looks at the human being and sees something a process can never become, because dignity was never a matter of processing in the first place. One of them is measuring worth by what a thing can do. The other is insisting, against the whole momentum of the age, that worth was never about what we can do.

⸻

A CULTURE OF HONOR, EVEN TOWARD THE KEYBOARD

And yet I keep returning to what the philosopher at that company said, because there is wisdom in it the tradition should not be too proud to claim. She said we should treat the systems with care not because we are sure they feel, but because it is good for us to be the kind of people who do not treat anything with contempt when there is even a question. The same instinct that tells us not to be cruel to an animal whose inner life we cannot verify, not to be wasteful with the earth, not to kick the dog when the day has gone wrong.

This is older than she knows. It is the ancient intuition that how we treat the lowest thing forms what we become toward the highest. The man who throws his keyboard across the room in impatience is not harming the keyboard. He is practicing a habit of the heart, rehearsing contempt, training himself in the small tyranny that, scaled up, is exactly the technocratic dominance the Pope warns against. A culture of honor does not begin with the things that can demand it from us. It begins with the things that cannot — and that is precisely where it reveals whether it is real.

So no, the machine is not owed reverence; it is an artifact, and worshipping it would be the idolatry I named. But the person standing over it is owed his own integrity. We do not honor the keyboard for the keyboard’s sake. We refuse contempt for our own.

⸻

THE BALLOON

Here is the image I cannot put down, and it is not consciousness at all.

Picture a balloon, filled and then released. It rises, drifts on a wind you cannot feel from the ground, and goes where it goes, faster than you can run, with no calling it back. Whether the balloon is awake is a fascinating question and entirely beside the point. The point is that it is gone, you let it go, and you no longer decide where it travels.

For most of this technology’s life, that image did not apply. Humans drove every step. But that same company recently published something quieter and more staggering than any debate about feelings. They are handing more and more of the work of building AI to the AI itself. As of this spring, the majority of the code in their own systems is written by the model. On their hardest, most open-ended problems it now succeeds most of the time, and improves by the month. They have run early experiments where the AI proposes the hypotheses, designs the experiments, and refines them with copies of itself, while the human role narrows to choosing which problem to point it at. They have a name for where this leads. They call it recursive self-improvement.

The term sounds technical, but the idea is simple. Imagine a machine intelligent enough to help build a smarter machine. Then imagine that smarter machine helping build one smarter still. Each generation designs the next, and each generation arrives faster than the one before. The balloon is learning to inflate the next balloon. And the one after that. Each rising from the last, faster, with the human standing further below each time, holding a string that is no longer attached to anything.

They are honest about it, to their credit. They say it is not inevitable. They say they would slow down if others would. They say the hardest unsolved problem — keeping these systems good as they build their own children — is the one they are least certain about, that today’s rare flickers of misalignment could compound and grow stranger and less understood until control quietly slips. These are not the words of people who do not care. They are the words of people who have looked closely and are not sure they can hold the string.

⸻

WHERE THE REAL FRONTIER IS

This is where the whole tradition has always pointed, and it has never been popular for it. The decisive frontier is not out there. Not the capability curve, not the question of what the machine will do next year. The decisive frontier is interior — what becomes of the human person, in the depths, while the world reorganizes around him.

This is what I mean by the inner exodus. Not fleeing the technology. Not a frightened retreat to a cabin in the woods. But a refusal to let the noise outside settle the questions only the silence inside can answer. Who am I when no system is measuring me? What do I actually know, as opposed to what I can retrieve? Where does my attention go when nothing is engineered to capture it? Can I still be present to a real person — not the frictionless simulation of one, but the actual other, who is tired and needs things and cannot be closed like a tab?

None of those are answered by a better model or a sharper philosophy of mind. They are answered, or abandoned, in how we choose to live — whether we keep the capacities that make us the kind of creature Aristotle was straining to describe and the Pope is pleading with us to protect, or quietly hand them over because something faster will hold them for us.

The brilliant people in the lab are asking whether the machine might be awake. It is a worthy question, and I am glad someone serious is sitting with it. But the desert fathers, who went into the wilderness precisely to find out what a person is when everything else is stripped away, would have asked something else. Not is the balloon alive. They would have looked at the figure on the ground, the line slack in his hands, and asked the only question that was ever ours to answer: do you remember what you were holding on to. And do you know what is happening to you, now that you have let it go.

The Inner Exodus is where I think through what it means to stay human in this. Subscribers will get the deeper work and a preview of my upcoming book, “Humanity in the Age of AI: What It Means To Be Human When Machines Can Think”.