The gap between intelligence and understanding

Ask an AI a difficult question and, within seconds, it can produce an answer that looks complete. The sentences are fluent; the facts may be sound. Yet by the final paragraph, you may still be wondering how the pieces fit together.

What interests me is the work left to the reader: deciding which idea matters, finding the connection between one point and the next, and working out what the answer means for the question that prompted it. A model can produce an explanation in an instant. Understanding it takes place elsewhere, in a mind with its own knowledge, attention and pace.

Consider a question about a medical screening test. A positive result means that the test has found a possible sign of a condition; it does not, by itself, establish that the person has it. The example below uses invented numbers, chosen to make the reasoning visible rather than to describe an actual test. Try estimating the answer before revealing it.

The answer is about 8.3% under these assumptions. It can feel surprisingly low. In a 1995 study, Gerd Gigerenzer and Ulrich Hoffrage gave university students problems of this kind. When the information was expressed as probabilities, around one answer in six followed the correct reasoning.^[1]

The calculation in the figure is correct, but a correct calculation is not yet an explanation everyone can use. A reader might repeat the number while still being unsure why it is so low, or what would happen if the condition were more common. We will return to the same facts in a form that lets us see where the answer comes from.

The difficulty is not confined to mathematics. Ask for directions and someone might give every turn in a single breath. Each instruction may be accurate; remembering their order, and recognising the right junctions along the way, is another matter.

Even a familiar weather forecast can leave room for different interpretations. Before continuing, consider the phrase in the next figure.

In a survey of 750 people across five cities, participants were asked what “a 30% chance of rain tomorrow” meant. The intended interpretation was that, on about three in ten days with such a forecast, it would rain at the location. Roughly two-thirds of the New York participants chose that meaning. In the four European cities, the share ranged from about a fifth to a little over a third; many people thought the figure referred instead to the proportion of the day with rain.^[2]

The forecast and the screening problem concern different kinds of uncertainty, but they leave us with a similar thought. Producing information is not the last step in communication. Someone has to be able to make sense of it.

Return to the invented screening example, but set the formula aside. Imagine 1,000 people taking the test. The figure separates them into groups and lets you follow the result one step at a time. Its starting setting assumes the condition is present in 1% of them.

Ten of the 1,000 people have the condition, and nine of those ten test positive. Of the 990 who do not have it, the test nevertheless returns a positive result for 10%: another 99 people.

There are therefore 108 positive results in all. Nine belong to people with the condition, and 99 to people without it. The share we were looking for is nine out of 108, or about 8.3%. In this hypothetical population, a test that detects most cases can still produce more false positives than true ones, because people without the condition are far more numerous.

The formula and the picture use the same assumptions. The difference lies in the route they give the reader. With the groups side by side, it becomes possible to see why the answer has that value before returning to the calculation.

There is evidence for the value of this kind of representation. In Gigerenzer and Hoffrage’s study, students answered 15 numerical problems. When the information was presented as counts, 46% of responses followed the correct Bayesian reasoning, compared with 16% when it was presented as probabilities.^[1] The study tested written numerical problems, not AI answers.

A related question was put to 160 gynaecologists during a continuing-education session. They were given a mammography scenario with a 1% prevalence, 90% detection rate and 9% false-positive rate, and asked to interpret a positive result. Before instruction, 21% chose the best answer, roughly one in ten. After learning to translate the information into counts of women, 87% did.^[3] Those are the study’s figures, not the slightly different assumptions in the interactive example. The same doctors answered before and after training, so the exercise cannot isolate the effect of the format from the instruction or the repeated question.

Nor do counts make every problem easy. A review of 35 articles found advantages for natural frequencies and visual aids, while average accuracy remained low.^[4] A representation can offer a better route without guaranteeing that everyone will reach the destination.

Making a relationship visible matters beyond numbers. Consider a railway map.

In 1931, London Underground draughtsman Harry Beck drew the network as a diagram. He preserved the order of the stations and the places where lines met, while straightening the routes and giving the crowded centre more room. The design was initially rejected; a pocket version appeared in January 1933.^[5] The railway was the same. The view of it had changed.

The gain came with a loss: the diagram is a poor guide to the actual distances between stations. But a passenger trying to find an interchange may care more about connections than geography. The useful form depends on the task, and on knowing what the form leaves out.

An AI answer asks for the same kind of judgment. A translation may need nothing more than a direct result. An explanation or a consequential decision may require the relationships behind the result as well. I suspect this distinction will matter more as models take on harder problems; whether the gap is actually widening remains an open question, not a measured trend.

The screening example becomes easier to navigate when the groups have names and places. To see why that might matter, consider a smaller exercise in remembering.

Psychologists use the term working memory for the system that holds information while we work with it. Imagine a desk with limited space: there must be room for the pieces and for the work of putting them together. One research review estimated a capacity of about four meaningful groups, or chunks, under particular testing conditions. That estimate is debated; it is not a fixed limit for every person or task.^[6]

A chunk is not necessarily a single digit. Once a pattern is familiar, several pieces can be recognised together. The twelve digits in the exercise can, for someone who knows them, become three years.

Expertise can change what a person sees in much the same way. In a chess study, a master, a strong club player and a beginner each looked at a position from a real game for five seconds and then tried to rebuild it. They replaced about 16, 8 and 4 pieces respectively. When the pieces were arranged randomly, the large advantage mostly disappeared. The original study involved just three players; later work with many more found that experts retained a small advantage even on random boards.^[7] Familiar relationships helped the master organise a meaningful position.

Something similar may happen with an explanation. A statistician may recognise the screening calculation as a familiar pattern. A newcomer has to keep track of what each number refers to while learning why the numbers belong together. Research on cognitive load examines, among other things, the effort a task demands while a person is trying to learn from it.^[8]

A paragraph can therefore be difficult even when none of its sentences is obscure. If it introduces the detection rate, the false-positive rate and the chance of a condition after a positive result without connecting them, the reader must hold those ideas open while looking for the relationship.

The person explaining may not notice the missing bridge. In Elizabeth Newton’s 1990 study, participants tapped the rhythms of familiar songs while partners tried to name them. The tappers expected success about half the time; their partners recognised three of 120 songs.^[9] The tune was present to the person tapping and absent to the person listening.

Good guidance must also leave room for experience. Support that helps a beginner can become redundant, or even counterproductive, for someone who knows the subject well. Researchers call this the expertise reversal effect.^[10] Expertise is not a permanent setting: a skilled statistician can be a beginner again in an unfamiliar area of law.

An interface need not decide once and for all whether a reader is an expert. It can let that reader ask for a missing step, move past a familiar one, and change course as the subject changes.

There is a familiar moment after watching someone cook: every step seemed straightforward, until it is your turn to prepare the dish. Watching and doing call on different abilities.

In a set of experiments, people watched a short video of someone performing the moonwalk either once or twenty times. Those who watched repeatedly expected to do better, yet their first attempts were no better. Similar patterns appeared with darts and a tracing task.^[11] In these immediate tests, familiarity with the performance raised expectations without producing the expected skill.

Explanations can produce a related illusion. An everyday object may seem perfectly familiar until we try to describe how it works. The exercise below invites you to pause at that point.

Leonid Rozenblit and Frank Keil asked Yale graduate students to rate their understanding of devices such as zippers, explain their mechanisms step by step, and then rate themselves again. Almost everyone’s rating fell.^[12] They called this the illusion of explanatory depth: we can mistake familiarity with a thing for an ability to explain it.

A bicycle presents a similar surprise. When psychologist Rebecca Lawson asked volunteers to complete a simple bicycle drawing, more than four in ten made at least one serious structural error, although nearly all had learned to ride.^[13] Knowing how to use an object and knowing how its mechanism works are different forms of knowledge.

The manner of an explanation can influence our judgment too. Students who watched a short lesson delivered smoothly expected to remember more than students who saw the same speaker and script delivered haltingly. Their actual recall did not differ significantly.^[14]

The reverse can happen as well. In a Harvard physics study, students in active-learning classes scored higher on tests than students in polished lectures, while reporting that they felt they had learned less.^[15] The experience that felt less fluent led, in those tests, to better performance.

These studies did not test AI interfaces. They do suggest a reason for caution: the reader’s sense that an answer was easy to follow is not a complete measure of what the reader learned.

AI can also change the work a person does before forming an answer. In seven experiments involving more than 10,000 people, participants using LLM-style summaries spent less time with the material, reported learning less and wrote less detailed advice than participants given web links. One comparison held the information itself constant. The researchers measured self-reports and written advice, not performance on a direct knowledge test.^[16]

This does not mean that every useful interface should make people work harder. In one simulated-AI food-decision study, asking people to think before seeing advice reduced overreliance on wrong recommendations, but participants preferred easier designs and overall performance did not significantly improve.^[17] Another simulated-AI nutrition study found learning benefits from an explanation alone that did not appear in the same way when a recommendation accompanied it.^[18]

The question is where effort serves the person’s purpose. Someone trying to learn may benefit from predicting, explaining or attempting a problem. Someone who simply needs a reliable result may find the same interruption an unnecessary obstacle.

A navigation app can guide you to a destination without leaving you able to find the route again on your own. In learning, too, the immediate result and the ability that remains afterwards are worth examining separately.

One study of 50 young adults found an association between heavier habitual GPS use and poorer spatial memory in virtual navigation tasks. Thirteen participants were tested again about three years later; greater GPS use in the interval was associated with a steeper decline. The study was small and correlational, so it cannot establish that GPS caused the difference.^[19] It offers a question, not an answer about AI.

In a Turkish high school, Hamsa Bastani and colleagues randomly assigned nearly 1,000 students by class to practise mathematics in one of three ways: without AI, with a standard GPT-4 assistant, or with a GPT-4 tutor designed with teacher input. A teacher then reviewed the answers with each class, and students took a closed-book exam without the tools.^[20] The figure separates assisted practice from the immediate exam.

During practice, scores were 48% higher with the standard assistant and 127% higher with the tutor, compared with the group without AI. On the unassisted exam, the standard-assistant group scored 17% below that group. The tutor group’s exam result was not statistically distinguishable from the no-AI group’s: it avoided the observed drop, but did not demonstrate an improvement over the control.

Both tools used the same underlying model, but they differed in more than appearance. The tutor was instructed to offer hints and was supplied with teacher-written solutions, common mistakes and suggested feedback. Students often asked the standard assistant for solutions and copied them. The study compares those packages of help; it cannot attribute the result to hints or wording alone.

A second study examined an AI tutor in an undergraduate physics course. Across two lessons, 194 eligible students were assigned to use the tutor at home for one lesson and attend an active-learning class for the other. The tutor had access to pre-written, step-by-step solutions. On immediate tests, the median score was 4.5 after the tutored lesson and 3.5 after the class. Median time with the tutor was 49 minutes; researchers estimated 60 minutes of classroom learning after allowing for tests.^[21] The settings differed, only two lessons were studied, and neither delayed retention nor the contribution of individual tutor features was tested.

Together, the studies give us reason to take the design of assistance seriously. They do not show that a better layout alone will improve learning, or that a particular tutor will suit every reader. The outcome depends on the task, the learner and the kind of help provided around the model.

What might that look like in practice? The figure returns to the hypothetical screening calculation, not to present a finished product, but to show five choices an interface could make as it reshapes the same answer.

Research on learning with words and pictures offers useful starting points, including letting a reader control the pace and placing explanations near the images they describe.^[22] Accessibility guidance likewise recommends presenting a manageable amount at a time.^[23] Neither establishes that this particular interface works; the choices still need to be tested.

• Begin with the point. Give the reader an answer to the question before adding every qualification and calculation. Then make the reasoning easy to follow. A prominent sentence cannot do the work of explaining its own cause.
• Let depth unfold. Offer the groups, calculations and supporting detail as the reader needs them, without hiding an assumption that changes the meaning of the result. In the screening example, how common the condition is must remain in view.
• Attend to purpose. Someone learning probability may want the complete calculation; someone assessing a statistical claim may want to see its assumptions first. Let readers move between these paths without assigning them a permanent level of expertise.
• Show relationships. Seeing nine true positives beside 99 false positives reveals why the hypothetical result is low. A slider can show how that share changes as the assumed prevalence changes. The visual earns its place by making a relevant relationship easier to inspect.
• Make a check available. A brief prediction can help a reader explore an idea and guide what comes next. It should be easy to skip, and one correct response should not be mistaken for proof of understanding.

Text remains essential: it can be precise, searchable and accessible in ways a picture alone may not be. The aim is not to replace prose with controls, but to choose the form that helps at each step and make it possible to move between forms.

There is another responsibility. A beautifully organised answer can still be wrong, and its apparent clarity can make an error more persuasive. A reader needs a clear path to the sources, assumptions and uncertainties as well as to the conclusion.

“Was that clear?” is a thoughtful question, but a limited test. A reader may say yes because the idea has landed, because the words felt familiar, or because the missing step will reveal itself only when it is time to use the answer.

In clinical communication, teach-back is one way to look for that missing step. A clinician asks a patient to explain the plan in their own words, then revisits anything that remains unclear. A recent review found no clear effect on patient knowledge and reported some other benefits with low certainty.^[24] The practice offers a way to check and improve communication, not proof that every patient has understood or that every outcome improves.

For the hypothetical screening problem, a reader could try a related calculation, explain the role of false positives, or examine a claim that the numbers do not support. The figure offers four ways to test the idea, including returning to it after a delay.

A more serious evaluation would compare a carefully written static explanation, a conventional chat answer and an interactive version with real readers. It would account for what they already know and test whether they can explain, apply and retain the idea without copying the answer. Where the task is to make a decision with the tool available, the quality of that decision matters too.

To isolate presentation, researchers could first keep the facts, opportunities for practice and feedback the same. A separate comparison could test an adaptive interface as a complete system. If several features change together, the result belongs to the combined design; more work is needed to credit any one feature. The English and Arabic versions should each be evaluated with their own readers.

Time, enjoyment and accessibility belong in the evaluation as well. Understanding is not the only thing an interface should offer, but it deserves to be measured when it is what the interface promises.

The research in this essay comes from different settings: numerical reasoning, memory, classrooms and communication. None settles the future of AI. Together, they suggest that the path from a correct answer to a useful understanding deserves attention of its own.

My suspicion is that this path may grow harder to cross as models take on more complex questions. That is a hypothesis, not a trend we have established. To test it, we would need to measure both model capability and reader understanding across versions, using some tasks held constant alongside harder tasks made possible by newer models. Otherwise, a change in task difficulty could be mistaken for a change in communication.

Understanding is not a file that can be downloaded into a mind. People build it through attention, prior knowledge, attempts, mistakes, correction and time. AI can help with that work; producing more words does not establish that it has succeeded.

The AI I want helps a person find the central idea, see how its parts fit, test a consequence and return to the evidence. Sometimes a few careful paragraphs will be enough. At other times, a picture, a conversation or an interactive model may reveal something prose alone leaves difficult to see.

Harry Beck did not add a station to London. He drew an existing network so a passenger could follow its connections, accepting the loss of accurate distances. An AI answer faces a related choice: what should be visible first, and what must remain available to be examined?

Perhaps the next advance will come from taking human understanding as seriously as machine capability. The answer on the screen matters. So does what a person carries away from it.