How to Think Clearly (3/11): Catch the Error While It Happens

Where this sits: stage 1 of 6, the Mind. The previous post described the machine and its limits. This one is about noticing the moment a limit is hit, because a failure that goes unnoticed cannot set off any safeguard.

Where this post sits in the series: stage 1 of 6, the Mind.

A safeguard is a decision made in advance, but it still needs a moment to fire. The difficulty is that the machine does not announce its own failures. In a series of twelve studies, Leonid Rozenblit and Frank Keil found that people feel they understand complex phenomena with far greater precision, coherence and depth than they really do, an illusion of explanatory depth. A thinker has no reliable gauge for what is missing. A gauge has to be built from the signals the mind does emit. Three are cheap to catch: confusion, the flinch, and the word “should”. The choice of these three is this post’s own selection, not a result from one study.

Confusion

Confusion is what a model feels like when it fails against the input. In a handbook chapter on the topic, Sidney D’Mello and Arthur Graesser list what triggers it: impasses, anomalies, contradictions, disrupted goals, and novelty too extreme to comprehend. In their experiments, planting contradictory opinions between two animated agents significantly raised learners’ confusion. The chapter also reports that learners who at least partly resolved their confusion scored significantly better on a later comprehension test than learners who stayed confused. The same chapter warns that unresolved, hopeless confusion goes with poor performance.

So the signal is only valuable if it is caught and then worked on, and catching comes first. It lasts a few seconds, and it is easy to let it pass: a word that reads oddly, a number that does not match the other number, a paragraph that cannot be retold. The practice is to name it and ask why: “I notice I am confused, because I assumed X.” Saying the assumption aloud turns a vague discomfort into a claim that can be checked, which is the map-versus-territory move that the Belief stage develops.

There is a small body of evidence for the naming step itself. In a 2007 fMRI study, Matthew Lieberman and colleagues found that putting feelings into words, compared with other ways of encoding the same images, reduced the response of the amygdala to negative emotional images and increased activity in the right ventrolateral prefrontal cortex. That study used emotional images, not confusion. Carrying the result over to confusion is this post’s argument, not a finding.

The flinch

The second signal is the small recoil on seeing an inbox, a number, a task. In a study of investor behaviour, Niklas Karlsson, George Loewenstein and Duane Seppi found, in two datasets of Scandinavian and American investors, that investors monitor their portfolios more frequently when markets are rising than when they are flat or falling. They named it the ostrich effect: attention to information follows how it will feel, not how useful it is.

The argument here is that a flinch is data. It points at what is least wanted and, often, at what matters most, since bad news is the news that can still change a decision. Looking away trains nothing useful. The practice is to hold still for a few seconds and ask which of three things is behind the recoil: the content, the person, or the effort. The ten-second length is a working rule, not a measured threshold.

The “should”

The third signal is the sentence that starts with “I should”. E. Tory Higgins’s 1987 paper on self-discrepancy defines the ought self as a representation of the attributes that someone, yourself or another, believes you should possess: a sense of duty, obligation, responsibility. Higgins proposed that a gap between who you are and who you ought to be goes with agitation-related emotions such as fear, threat and restlessness, while a gap from the ideal self goes with dejection, such as disappointment and sadness. In his data from 70 subjects, ought-discrepancies were associated with anxiety and paranoid symptoms, ideal-discrepancies most strongly with depression. The evidence is correlational.

What this gives the thinker is a question. A “should” has a source, and the source is often not a decision. The practice is to ask whose “should” it is. If it traces to nobody, drop it. If it traces to a real wish, turn it into the first concrete step. Either way an inherited sentence becomes a choice, which is the only thing a safeguard can be built on.

One label, three signals

The practice is the same for all three: say which one it is, in words. “Confused.” “Flinching.” “Should.” Nothing else is required at first, and nothing needs to be decided. The if-then plans of the next stage need a trigger that cannot be missed, and these three are the triggers the mind provides for free.

Figure 1. The three signals, what to say and what to ask.

A safeguard decided in advance needs a moment to fire. Catching the error costs seconds and decides nothing, but it keeps the failure from passing unseen. The next post turns from catching failures to rebuilding the setting, so there are fewer to catch.

Series: How to Think Clearly. 1. Where Thinking Breaks and What to Put There. 2. A Thought Has a Machine Behind It. 3. Catch the Error While It Happens. 4. Make the Right Thing the Easy Thing. 5. The Map Is Not the Territory. 6. Three Tools for Estimating the Unknown. 7. Averages Lie. 8. Price Your Own Hour. 9. Explore Early, Exploit Late, Imagine the Failure First. 10. The Hill You Are Standing On. 11. Let AI Think, Keep the Decision.