"I Knew It!" - The Retrospective Confirmation Bias

I should never have come to this movie set.

When standout events happen (we win the lottery, our camera is stolen, a flight is cancelled, we are diagnosed with a serious illness), many people report a sense that they somehow knew the event was about to occur.

“I sensed there was something wrong with that plane.”
“I should have listened to my gut when I bought that lemon of a house.”
“I knew I had the winning ticket before they called it out.”

Here’s the sequence:

1. Something happens that turns out to be much better or worse than the odds would suggest. 
2. We think back to our thoughts about the event/marriage/trip/purchase before we knew how it would turn out.
3. We find memories suggesting that we had advance knowledge of the outcome. These are often described as “gut feelings.”
4. We conclude from this that we had advance knowledge – either from some psychic ability or from unconsciously processing subtle cues at the time (“I must have sensed something wrong before the tire blew”).

Skeptical family and friends are generally unsupportive at these times. “Look, if people could tell winning numbers by predicting the future, there’d be more winners and the lottery corporation would go bankrupt.” 

Usually these skeptics refuse to believe that we really did have such premonitions prior to the event. But we did. We remember them.

So what’s going on here?

One possibility is that we really do have psychic abilities, and our only problem is that all too frequently we ignore our own predictions. Personally, I think this is unlikely (lotteries wouldn’t work, and so on).

More likely, we really do have such premonitions. But we have them across the board. When events work out as our premonitions suggest, we look back and remember the foreboding we experienced. When events don’t fit our predictions, we don’t look back. In other words, we remember our “hits” and forget our “misses.”

Personally, I can’t get on an aircraft (or see someone off at the airport) without at least a faint twinge of “what if.” Not surprising, given that aircraft are seldom on my mind except a) when I or someone I know are traveling, or b) one has just gone down in flames.

Should disaster strike, I would be able to cast my mind back, remember that twinge, and conclude “I knew this would happen.” To do so, I would have to neglect the hundreds of times when I also got that twinge and nothing happened.

In fact, in an alarmist society, the only way any of us survive is to disregard the dozens of little twinges we get each day. A tiny part of our consciousness, responsible for warning us away from danger, constantly alerts us to potential threats: earthquakes, car accidents, job firings, stock market plunges, lost cell phones, lethal ailments, asteroid strikes, you name it.

If we paid attention to all of these warnings we’d never leave the house – but then we’d have to disregard a different set: home invasions, stove fires, carbon monoxide leaks, falling space junk, termites – and, still, earthquakes, typhoons, and planes crashing on rooftops.

This, in short, is the Retrospective Confirmation Bias. The tendency to recollect predictions that are in accord with eventual reality, and to forget or neglect those that are not. The result is an erroneous sense that “I knew it!”

Is this phenomenon a tiny curiosity – an irrelevant peccadillo of the human mind with little consequence?

Probably not. For one thing, it contributes to the sense that the world is controllable and predictable, and results in us having greater faith in our beliefs and predictions than is probably merited.

It may also be one of the primary reasons that the gambling industry is so successful. When playing Blackjack, betting on horses, or choosing lottery numbers, we are constantly weighing our options. We find ourselves simultaneously attracted to and repelled by the various options.  “I could hold on 17 or ask for a card.” “Landslide has a good name for a racehorse, but has never won on this track…” “John’s birthday is the 23rd, but something about 43 is calling to me too…”

When we win, we see that our intuitions have borne fruit. When we lose, we can look for hints that our intuitions, properly sifted, would have moved us in the right direction. “Damn, I was tempted to go for Landslide. Why didn’t I?” Properly chastened, we can resolve to do better next time. And the only way to do that is to return to the betting window.

The Retrospective Confirmation Bias can also torture us with inappropriate guilt. “I had misgivings about the snow on the roads – why didn’t I warn my brother before he drove off?” “I wondered about that mole on Darlene’s arm for two years – how could I not have ordered her to get it checked?”

Clinical practices are filled with people mourning that "I should have known. And in fact, I did know - I just didn't do anything about it."

Much guilt, much self-reproach, much gambling addiction, and much of our misplaced faith in our abilities to tell the future - much of all of these things could be dispelled if we reminded ourselves of the legion of intuitions we had that were mistaken. If we conduct a full inventory, we will almost certainly find that our few "hits" were almost certainly random.

And once again we'll be flung into that unsatisfying state against which the human mind rebels. It's an unpredictable world. "I didn't know this would happen."