How We Know What Isn't So: The Fallibility of Human Reason in Everyday Life

When examining evidence relevant to a given belief, people are inclined to see what they expect to see, and conclude what they expect to conclude. Information that is consistent with our pre-existing beliefs is often accepted at face value, whereas evidence that contradicts them is critically scrutinized and discounted. Our beliefs may thus be less responsive than they should to the implications of new information

People will always prefer black-and-white over shades of grey, and so there will always be the temptation to hold overly-simplified beliefs and to hold them with excessive confidence

What we believe is heavily influenced by what we think others believe

We humans seem to be extremely good at generating ideas, theories, and explanations that have the ring of plausibility. We may be relatively deficient, however, in evaluating and testing our ideas once they are formed

it seems that once again people engage in a search for evidence that is biased toward confirmation. Asked to assess the similarity of two entities, people pay more attention to the ways in which they are similar than to the ways in which they differ. Asked to assess dissimilarity, they become more concerned with differences than with similarities. In other words, when testing a hypothesis of similarity, people look for evidence of similarity rather than dissimilarity, and when testing a hypothesis of dissimilarity, they do the opposite. The relationship one perceives between two entities, then, can vary with the precise form of the question that is asked

For desired conclusions, we ask ourselves, "Can I believe this?", but for unpalatable conclusions we ask, "Must I believe this?

When we do cross paths with people whose beliefs and attitudes conflict with our own, we are rarely challenged.

We hold many dubious beliefs, in other words, not because they satisfy some important psychological need, but because they seem to be the most sensible conclusions consistent with the available evidence. People hold such beliefs because they seem, in the words of Robert Merton, to be the “irresistible products of their own experience.”7 They are the products, not of irrationality, but of flawed rationality.

How do we distinguish between the legitimate skepticism of those who scoffed at cold fusion, and the stifling dogma of the seventeenthcentury clergymen who, doubting Galileo's claim that the earth was not the center of the solar system, put him under house arrest for the last eight years of his life? In part, the answer lies in the distinction between skepticism and closed-mindedness. Many scientists who were skeptical about cold fusion nevertheless tried to replicate the reported phenomenon in their own labs; Galileo's critics refused to look at the pertinent data.

A person's conclusions can only be as solid as the information on which they are based. Thus, a person who is exposed to almost nothing but inaccurate information on a given subject almost inevitably develops an erroneous belief, a belief that can seem to be "an irresistible product" of the individual's (secondhand) experience.

some evidence has accumulated that people who habitually fail to put the most favorable cast on their circumstances run the risk of depression.

Finally, it has been shown that the tendency for people to think of themselves as above average is reduced—even for ambiguous traits—when people are required to use specific definitions of each trait in their judgments.

People rate themselves more favorably on amorphous traits like sensitivity and idealism (at the 73rd percentile, on average) than on relatively straightforward traits like thriftiness and being well-read (48th percentile).

One of the simplest and yet most powerful ways we do so lies in how we frame the very question we ask of the evidence. When we prefer to believe something, we may approach the relevant evidence by asking ourselves, “what evidence is there to support this belief?

Our motivations thus influence our beliefs through the subtle ways we choose a comforting pattern from the fabric of evidence.

Psychologists have known for some time that rewarding desirable responses is generally more effective in shaping behavior than punishing undesirable responses.19 However, the average person tends to find this fact surprising, and punishment has been the preferred reinforcer for the majority of parents in both modern society19 and in earlier periods.

When people learn no tools of judgment and merely follow their hopes, the seeds of political manipulation are sown.”11 As individuals and as a society, we should be less accepting of superstition and sloppy thinking, and should strive to develop those “habits of mind” that promote a more accurate view of the world.

we believe certain things because they ought to be true.

It appears that the probabilistic sciences of psychology and medicine teach their students to apply statistical and methodological rules to both scientific and everyday-life problems, whereas the nonprobabilistic science of chemistry and the nonscientific discipline of the law do not affect their students in these respects (p. 438)…. the luxury of not being confronted with messy problems that contain substantial uncertainty and a tangled web of causes means that chemistry does not teach some rules that are relevant to everyday life

One can only agree with the general argument that generating more interest in the scientific enterprise would be helpful in these regards. There is an intriguing twist to this general contention, however. A set of recent studies suggests—albeit only tentatively at this point—that a particular kind of science education may be especially effective in developing the habits of mind necessary for thinking clearly about the evidence of everyday experience. The logic that motivated these studies was quite simple: Exposure to the “probabilistic” sciences may be more effective than experience with the “deterministic” sciences in teaching people how to evaluate adequately the kind of messy, probabilistic phenomena that are often encountered in everyday life. Probabilistic sciences are those such as psychology and economics that deal mainly with phenomena that are not perfectly predictable, and with causes that are generally neither necessary nor sufficient.

We therefore should be more skeptical than we seem to be about evidence presented to us secondhand. We should become accustomed to asking ourselves where the information originated, and how much distortion—deliberate or otherwise—is likely to have been introduced along the way.

develop the habit of employing one of several “consider the opposite” strategies. We can learn to ask ourselves, for example, “Suppose the exact opposite had occurred. Would I consider that outcome to be supportive of my belief as well?” Alternatively, we can ask, “How would someone who does not believe the way I do explain this result?”,

Perhaps the most general and most important mental habit to instill is an appreciation of the folly of trying to draw conclusions from incomplete and unrepresentative evidence. An essential corrollary of this appreciation should be an awareness of how often our everyday experience presents us with biased samples of information.

The most favorable attitudes toward ESP were found in a survey of Canadian college students, of whom 80% reported a belief in psi.21 National surveys of the U.S. population have found that roughly 50% of the population are believers, including 67% of those who are college educated.22 Perhaps

Successes, in other words, are generally seen as confirmations of one’s underlying strategy, whereas failures tend to be thought of only as failures of outcome, not as failures of strategy.

This tendency to focus too heavily on the occasional success is helped along by an asymmetry in the way we evaluate success and failure. A single success generally does more to confirm a strategy’s effectiveness than a single failure does to disconfirm it. Indeed, successes tend to be taken as prima facie evidence that the strategy is effective.

Consider the common belief among some segments of the population that “the only way to get anywhere with the opposite sex is to come on strong.” Someone who holds such a belief will consistently come on strong and, at some point, will succeed in meeting his or her objective. The occasional success, however rare, will then be attributed to the choice of tactics, and its effectiveness will seem to be an unassailable fact of the person’s own experience. Because no single failure serves to disconfirm the strategy’s effectiveness (after all, nothing works all the time), the only way it can be shown to be ineffective is by discovering that the rate of success is lower with this strategy than with others. Given that alternative techniques are rarely if ever employed, the person is in no danger of having his or her favorite theories disabused.

Flattery, then, is like the other strategies discussed above in that it should be ineffective; unlike the others, however, it usually works. As Milan Kundera points out, “How defenseless we are in the face of flattery!”4 An argument could even be made that flattery is a strategy that is actually underutilized.

The more ambiguous the criterion, the easier it is to detect evidence of success. It is for precisely this reason that many alternative health practices do not offer precise remedies for specific disabilities. They promise instead to bring about “wellness,” “higher functioning,” or “better integration”—ambiguous benefits that may be hard to refute. Faith healers take full advantage of ambiguous criteria and studiously avoid pinning themselves down to verifiable predictions.