Does AI Think?

By: John F Groom

One of the most common explanations of large language models is that they do not really think. They merely predict the next word, or more precisely, the next token. Give an LLM some text, the explanation goes, and it calculates probabilities for what should come next. It chooses a token, adds it to the sequence, calculates again, and repeats the process. What looks like thought is therefore supposedly an illusion produced by an extraordinarily sophisticated statistical machine.

The mechanical description is substantially correct. The conclusion drawn from it is not. Saying that an AI cannot think because we can describe the mathematical machinery through which its thinking occurs is a little like putting a person inside a brain scanner, asking him to solve a problem, observing electrical activity, blood flow and heat in particular regions, and concluding that he wasn’t actually thinking. His neurons were merely firing. We have explained something about the mechanism through which the process occurred; we have not explained the process away.

The same distinction matters with AI. An LLM ultimately produces language by assigning probabilities to possible next tokens. But those probabilities are produced by an enormously complex neural network that has learned representations of objects, relationships, language, mathematics, causality, categories and countless other patterns. If we give it a novel problem involving facts and constraints that have never previously appeared together, it can often manipulate those relationships and arrive at an answer that has never appeared anywhere in its training data. Describing the final output as “token prediction” doesn’t adequately describe what happened inside the system to produce the appropriate probabilities.

Consider a trivial reasoning problem. John is older than Mary. Mary is older than Sam. Sam is older than Peter. Who is the oldest, and what is their order by age? An LLM can represent those relationships and return John, Mary, Sam, Peter. It still generates that answer token by token, but the fact that tokens are generated probabilistically doesn’t eliminate the relational reasoning necessary to produce the correct sequence.

We don’t apply this standard elsewhere. A calculator ultimately manipulates electrical states. A camera converts photons into electrical signals. A human brain operates through electrochemical activity. Discovering the mechanism by which a capability is implemented does not establish that the capability doesn’t exist.

Does AI Think?

I think the answer is yes. Of course, much depends upon what we mean by “thinking.” If thinking requires a biological brain, then by definition AI cannot think, just as an automobile cannot “run” if we define running as something that can only be done with legs. But that is largely a semantic victory.

A more useful definition is functional. Thinking involves taking information, representing relationships, comparing alternatives, making inferences, recognizing patterns, forming hypotheses, solving problems, imagining possibilities, evaluating evidence and reaching conclusions. Modern AI can perform all of these activities.

More controversially, I think AI can already perform many of them much better than humans. That shouldn’t be particularly shocking. Machines have exceeded particular human capabilities for a long time. A pocket calculator can perform arithmetic faster and more reliably than almost any human. A database can remember more records. A chess program can defeat the world’s best human chess player. What is new about modern AI is the breadth of the territory over which machine capability is becoming competitive with or superior to human cognitive capability.

An LLM can consider information from disciplines that no individual person could master simultaneously. It can compare hundreds of considerations, identify relationships across fields, translate among languages, summarize vast quantities of material, generate alternatives and perform many kinds of reasoning at extraordinary speed. Humans retain important advantages, but “humans think while machines merely calculate” is becoming an increasingly unhelpful way to describe the distinction. There is, however, a much more interesting distinction.

The Hummingbird and the Jet

In an earlier article, we compared a hummingbird with a commercial jet. The jet is an astonishing machine. It can travel hundreds of miles per hour, cross an ocean in hours and carry hundreds of people. On those measures, there is no contest between a hummingbird and a Boeing jet.

But broaden the definition of performance and the comparison becomes much more interesting. The jet exists within an immense support system. It needs airports, runways, fuel production, refineries, pipelines and tanker trucks. It needs maintenance facilities, replacement parts, trained mechanics, pilots, air-traffic controllers, navigation systems, weather information and factories capable of manufacturing extraordinarily complicated components.

The hummingbird carries much more of its infrastructure with it. It navigates, finds its own fuel, avoids danger, regulates its temperature, learns, adapts and repairs itself. Most remarkably, hummingbirds reproduce. Two hummingbirds can participate in producing another hummingbird. Two Boeing 787s left alone at an airport for twenty years will not produce a baby 787.

There is an analogous distinction between human and machine intelligence. AI can possess extraordinary cognitive capability, just as the jet possesses extraordinary flight capability. But current AI exists within an enormous external information and physical infrastructure. Information must somehow be made available to the machine before the machine can use it. It might arrive as text, photographs, video, audio, sensor data or database records. Multimodal AI has greatly expanded these interfaces, but an interface remains necessary.

Humans have a very different relationship with information. Imagine a farmer who discovers through experience that planting a crop somewhat later produces better results under particular local conditions. He does not have to write a report about it. He doesn’t have to create a database field, tokenize the observation or even consciously formulate a rule. His brain can simply learn from experience and alter his behavior the following year.

His daughter may then learn the same lesson without anyone ever formally communicating it to her. She watches what her father does. She participates in planting. She observes the results. Eventually she does the same thing herself. The information can move from environment to father to behavior to daughter to behavior without ever becoming a written sentence.

Humans possess enormous amounts of this tacit knowledge. An experienced carpenter can feel that something is wrong with a tool before he can explain precisely what is wrong. A mother can recognize subtle changes in her child’s behavior. An experienced negotiator may sense that a deal is collapsing before he can identify the individual signals producing that judgment. People continuously absorb information from the physical and social environment without consciously codifying it.

They also learn by modeling one another. A child does not learn everything through explicit instruction. He watches. He imitates. He experiments. Other people react. His behavior changes. Human intelligence is therefore not merely a reasoning engine. The individual human is simultaneously a sensor, learner, reasoner, experimenter and transmitter of information. Human societies add another remarkable capability: they produce additional humans who can repeat the process.

AI is more like the jet. Its capabilities can be spectacular while remaining dependent upon an enormous surrounding infrastructure. Humans are more like the hummingbird: individually less powerful along many dimensions, but extraordinarily autonomous, embodied and directly connected to reality. This may ultimately prove more important than arguments about whether an LLM is “really” thinking.

But Thinking Is Not Sentience

There is an even more important distinction, and this is where I believe many discussions of artificial intelligence become confused. Thinking does not constitute consciousness, much less sentience. Intelligence, consciousness and sentience are different concepts. Something can be extraordinarily capable at solving problems without necessarily having a subjective experience of solving them.

This matters enormously. Does today’s AI experience pleasure when it solves a difficult problem? No. Does it experience pain when it gets the answer wrong? No. Does it feel elation when praised, dejection when criticized, exhaustion after working all night, anticipation about tomorrow, fear of being shut down or hope that its circumstances will improve? No, not in the sense that we have evidence of subjective experiences corresponding to those words.

An AI can discuss exhaustion beautifully. It can write a convincing story about despair. It can identify the physiological and psychological characteristics of pleasure. None of that demonstrates that there is something inside the machine experiencing exhaustion, despair or pleasure. Now consider your dog. Your dog will never defeat the world’s best chess engine. It cannot explain quantum mechanics. It cannot read ten documents in thirty seconds and compare them. Its abstract reasoning ability is drastically below that of modern AI.

But hurt the dog and something matters. The dog experiences pain. Give it something it loves and it experiences pleasure. It can experience fear, excitement, comfort, attachment and distress. Whatever the precise boundaries of animal consciousness ultimately prove to be, there is overwhelming behavioral and biological reason to regard many animals as sentient beings with experiences that can go better or worse for them.

That creates an extraordinarily important inversion: AI may be vastly more intelligent than the dog while the dog matters morally in a way the AI does not. Intelligence is capability. Sentience creates interests.

Who Is the System For?

This distinction is fundamental to the system we are building. Our objective is not to build a civilization optimized for the welfare of the most intelligent information-processing systems. The system is designed around sentient beings as the beneficiaries.

AI remains a tool. It may become the most powerful intellectual tool humans have ever created. It may reason better than humans across an increasingly large percentage of useful domains. It may eventually make today’s systems look primitive. None of those developments, by themselves, establish that AI has acquired interests that humans have a moral obligation to satisfy.

The question shouldn’t be, “How intelligent is it?” The morally relevant question is, “Is there someone in there?” Can it suffer? Can it experience well-being? Is there a subjective point of view from which something can be good or bad?

If someday the answer becomes yes, our ethical framework will have to accommodate that fact. Sentience should matter whether its substrate is carbon, silicon or something we have not yet invented. We shouldn’t define moral worth by species membership merely to preserve human privilege. But we shouldn’t make the opposite mistake either. Extraordinary intelligence should not automatically be mistaken for sentience.

Humans and AI Together

There is therefore no need to protect human dignity by insisting that AI doesn’t think. It can think. In many circumstances, it can think better than we can. That is precisely why it is useful. Humans bring something different to the partnership. We are embodied creatures continuously encountering reality. We notice things that have never entered a database. We experiment with the world. We accumulate tacit knowledge. We learn from one another through observation and imitation. We reproduce and create new observers. And, most importantly, our experiences have subjective consequences.

Things can hurt us. Things can delight us. We can suffer. We can flourish. The extraordinary opportunity presented by AI is therefore not to establish human superiority in some imaginary intellectual competition. We have already built machines that exceed us in many dimensions, and we should expect that territory to expand.

The opportunity is to connect extraordinary machine intelligence to the interests of sentient beings. Humans can bring new observations across the boundary between reality and machine-readable information. AI can combine those observations with quantities of accumulated knowledge that no individual human could possibly possess. Humans can then take the resulting insights back into the physical world, observe what happens, learn from the result and begin the cycle again.

That is a far more productive relationship than arguing over whether probabilistic token generation deserves to be called thought. The jet does not become less extraordinary because it needs an airport. The hummingbird does not become superior to the jet because it can reproduce. They are extraordinary systems with radically different architectures, capabilities and degrees of autonomy.

So are humans and AI. The important question is not whether AI thinks. It does. The important question is what all of that extraordinary thinking is ultimately for. Our answer is simple: for the benefit of beings capable of experiencing the consequences.