A Field Guide to Critical Thinking

[Guest guest]

Although this was written by someone in the field of

paranormal study, I think the methodology applies to all fields.

Lynn

A Field Guide to

Critical Thinking

by James Lett

There are many reasons for the popularity of paranormal beliefs in the

United States today, including:

 

the irresponsibility of the mass media, who exploit the public taste

for nonsense, the irrationality of the American world-view, which supports such

unsupportable claims as life after death and the efficacy of the

polygraph, and the ineffectiveness of public education, which generally fails to

teach students the essential skills of critical thinking.

As a college professor, I am especially concerned with this third

problem. Most of the freshman and sophomore students in my classes simply

do not know how to draw reasonable conclusions from the evidence. At

most, they've been taught in high school what to think; few of them know

how to think.

In an attempt to remedy this problem at my college, I've developed an

elective course called " Anthropology and the Paranormal. " The

course examines the complete range of paranormal beliefs in contemporary

American culture, from precognition and psychokinesis to channeling and

cryptozoology and everything between and beyond, including astrology,

UFOs, and creationism. I teach the students very little about

anthropological theories and even less about anthropological terminology.

Instead, I try to communicate the essence of the anthropological

perspective, by teaching them, indirectly, what the scientific method is

all about. I do so by teaching them how to evaluate evidence. I give them

six simple rules to follow when considering any claim, and then show them

how to apply those six rules to the examination of any paranormal

claim.

The six rules of evidential reasoning are my own distillation and

simplification of the scientific method. To make it easier for students

to remember these half-dozen guidelines, I've coined an acronym for them:

Ignoring the vowels, the letters in the word " FiLCHeRS " stand

for the rules of Falsifiability, Logic, Comprehensiveness, Honesty,

Replicability, and Sufficiency. Apply these six rules to the evidence

offered for any claim, I tell my students, and no one will ever be able

to sneak up on you and steal your belief. You'll be filch-proof.

 

Falsifiability

It must be possible to conceive of evidence that would prove the claim

false. It may sound paradoxical, but in order for any claim to be true,

it must be falsifiable. The rule of falsifiability is a guarantee that if

the claim is false, the evidence will prove it false; and if the claim is

true, the evidence will not disprove it (in which case the claim can be

tentatively accepted as true until such time as evidence is brought forth

that does disprove it). The rule of falsifiability, in short, says that

the evidence must matter, and as such it is the first and most important

and most fundamental rule of evidential reasoning.

The rule of falsifiability is essential for this reason: If nothing

conceivable could ever disprove the claim, then the evidence that does

exist would not matter; it would be pointless to even examine the

evidence, because the conclusion is already known -- the claim is

invulnerable to any possible evidence. This would not mean, however, that

the claim is true; instead it would mean that the claim is meaningless.

This is so because it is impossible -- logically impossible -- for any

claim to be true no matter what. For every true claim, you can always

conceive of evidence that would make the claim untrue -- in other words,

again, every true claim is falsifiable.

For example, the true claim that the life span of human beings is less

than 200 years is falsifiable; it would be falsified if a single human

being were to live to be 200 years old. Similarly, the true claim that

water freezes at 32° F is falsifiable; it would be falsified if water

were to freeze at, say, 34° F. Each of these claims is firmly established

as scientific " fact, " and we do not expect either claim ever to

be falsified; however, the point is that either could be. Any claim that

could not be falsified would be devoid of any propositional content; that

is, it would not be making a factual assertion -- it would instead be

making an emotive statement, a declaration of the way the claimant feels

about the world. Nonfalsifiable claims do communicate information, but

what they describe is the claimant's value orientation. They communicate

nothing whatsoever of a factual nature, and hence are neither true nor

false. Nonfalsifiable statements are propositionally vacuous.

There are two principal ways in which the rule of falsifiability can be

violated -- two ways, in other words, of making nonfalsifiable claims.

The first variety of nonfalsifiable statements is the undeclared claim: a

statement that is so broad or vague that it lacks any propositional

content. The undeclared claim is basically unintelligible and

consequently meaningless. Consider, for example, the claim that crystal

therapists can use pieces of quartz to restore balance and harmony to a

person's spiritual energy. What does it mean to have unbalanced spiritual

energy? How is the condition recognized and diagnosed? What evidence

would prove that someone's unbalanced spiritual energy had been -- or had

not been -- balanced by the application of crystal therapy? Most New Age

wonders, in fact, consist of similarly undeclared claims that dissolve

completely when exposed to the solvent of rationality.

The undeclared claim has the advantage that virtually any evidence that

could be adduced could be interpreted as congruent with the claim, and

for that reason it is especially popular among paranormalists who claim

precognitive powers. Jeane Dixon, for example, predicted that 1987 would

be a year " filled with changes " for Caroline Kennedy. Dixon

also predicted that Jack Kemp would " face major disagreements with

the rest of his party " in 1987 and that " world-wide drug

terror " would be " unleashed by narcotics czars " in the

same year. She further revealed that Dan Rather " may [or may not] be

hospitalized " in 1988, and that Whitney Houston's " greatest

problem " in 1986 would be " balancing her personal life against

her career. " The undeclared claim boils down to a statement that can

be translated as " Whatever will be, will be. "

The second variety of nonfalsifiable statements, which is even more

popular among paranormalists, involves the use of the multiple out, that

is, an inexhaustible series of excuses intended to explain away the

evidence that would seem to falsify the claim. Creationists, for example,

claim that the universe is no more than 10,000 years old. They do so

despite the fact that we can observe stars that are billions of

light-years from the earth, which means that the light must have left

those stars billions of years ago, and which proves that the universe

must be billions of years old. How then do the creationists respond to

this falsification of their claim? By suggesting that God must have

created the light already on the way from those distant star at the

moment of creation 10,000 years ago. No conceivable piece of evidence, of

course, could disprove that claim.

Additional examples of multiple outs abound in the realm of the

paranormal. UFO proponents, faced with a lack of reliable physical or

photographic evidence to buttress the claims, point to a secret

" government conspiracy " that is allegedly preventing the

release of evidence that would support their case. Psychic healers say

they can heal you if you have enough faith in their psychic powers.

Psychokinetics say they can bend spoons with their minds if they are not

exposed to negative vibrations from skeptic observers. Tarot readers can

predict your fate if you're sincere in your desire for knowledge. The

multiple out means, in effect, " Heads I win, tails you lose. "

 

 

Logic

Any argument offered as evidence in support of any claim must be sound.

An argument is said to be " valid " if its conclusion follows

unavoidably from its premises; it is " sound " if it is valid and

if all the premises are true. The rule of logic thus governs the validity

of inference. Although philosophers have codified and named the various

forms of valid arguments, it is not necessary to master a course in form

logic in order to apply the rules of inference consistently and

correctly. An invalid argument can be recognize by the simple method of

counterexample: If you can conceive of a single imaginable instance

whereby the conclusion would not necessarily follow from the premises

even if the premises were true, then the argument is invalid. Consider

the following syllogism for example: All dogs have fleas; Xavier has

fleas; therefore Xavier is a dog. That argument is invalid because a

single flea-ridden feline named Xavier would provide an effective

counterexample. If an argument is invalid, then it is, by definition,

unsound. Not all valid arguments are sound, however. Consider this

example: All dogs have fleas; Xavier is a dog; therefore Xavier has

fleas. That argument is unsound, even though it is valid, because the

first premise is false: All dogs do not have fleas.

To determine whether a valid argument is sound is frequently problematic;

knowing whether a given premise is true or false often demands additional

knowledge about the claim that may require empirical investigation. If

the argument passes these two tests, however -- if it is both valid and

sound -- then the conclusion can be embraced with certainty.

The rule of logic is frequently violated by pseudoscientists. Erich von

Däniken, who singlehandedly popularized the ancient-astronaut mythology

in the 1970s, wrote many books in which he offered invalid and unsound

arguments with benumbing regularity (see Omohundro 1976). In Chariots of

the Gods? he was not above making arguments that were both logically

invalid and factually inaccurate -- in other words, arguments that were

doubly unsound. For example, von Däniken argues that the map of the world

made by the sixteenth-century Turkish admiral Piri Re'is is so

" astoundingly accurate " that it could only have been made from

satellite photographs. Not only is the argument invalid (any number of

imaginable techniques other than satellite photography could result in an

" astoundingly accurate " map), but the premise is simply wrong

-- the Piri Re'is map, in fact, contains many gross inaccuracies (see

Story 1981).

 

Comprehensiveness

The evidence offered in support of any claim must be exhaustive -- that

is all of the available evidence must be considered.

For obvious reasons, it is never reasonable to consider only the evidence

that supports a theory and to discard the evidence that contradicts it.

This rule is straightforward and self-apparent, and it requires little

explication or justification. Nevertheless, it is a rule that is

frequently broken by proponents of paranormal claims and by those who

adhere to paranormal beliefs.

For example, the proponents of biorhythm theory are fond of pointing to

airplane crashes that occurred on days when the pilot, copilot, anchor

navigator were experiencing critically low points in their intellectual,

emotional, and/or physical cycles. The evidence considered by the

biorhythm apologists, however, does not include the even larger number of

airplane crashes that occurred when the crews were experiencing high or

neutral points in their biorhythm cycles (Hines 1988:160). Similarly,

when people believe that Jeane Dixon has precognitive ability because she

predicted the 1988 election of George Bush (which she did, two months

before the election, when every social scientist, media maven, and

private citizen in the country was making the same prognostication), they

typically ignore the thousands of forecasts that Dixon has made that have

failed to come true (such as her predictions that John F. Kennedy would

not win the presidency in 1960, that World War III would begin in 1958,

and that Fidel Castro would die in 1969). If you are willing to be

selective in the evidence you consider, you could reasonably conclude

that the earth is flat.

 

Honesty

The evidence offered in support of any claim must be evaluated without

self-deception.

The rule of honesty is a corollary to the rule of comprehensiveness. When

you have examined all of the evidence, it is essential that you be honest

with yourself about the results of that examination. If the weight of the

evidence contradicts the claim, then you are required to abandon belief

in that claim. The obverse, of course, would hold as well.

The rule of honesty, like the rule of comprehensiveness, is frequently

violated by both proponents and adherents of paranormal beliefs.

Parapsychologists violate this rule when they conclude, after numerous

subsequent experiments have failed to replicate initially positive psi

results, that psi must be an elusive phenomenon. (Applying Occam's Razor,

the more honest conclusion would be that the original positive result

must have been a coincidence.) Believers in the paranormal violate this

rule when they conclude, after observing a " psychic "

surreptitiously bend a spoon with his hands, that he only cheats

sometimes.

In practice, the rule of honesty usually boils down to an injunction

against breaking the rule of falsifiability by taking a multiple out.

There is more to it than that, however: The rule of honesty means that

you must accept the obligation to come to a rational conclusion once you

have examined all the evidence. If the overwhelming weight of all the

evidence falsifies your belief, then you must conclude that the belief is

false, and you must face the implications of that conclusion

forthrightly. In the face of overwhelmingly negative evidence, neutrality

and agnosticism are no better than credulity and faith. Denial,

avoidance, rationalization, and all the other familiar mechanisms of

self-deception would constitute violations of the rule of

honesty.

In my view, this rule alone would all but invalidate the entire

discipline of parapsychology. After more than a century of systematic,

scholarly research, the psi hypothesis remains wholly unsubstantiated and

unsupportable; parapsychologists have failed, as Ray Hyman (1985:7)

observes, to produce " any consistent evidence for paranormality that

can withstand acceptable scientific scrutiny. " From all indications,

the number of parapsychologists who observe the rule of honesty pales in

comparison with the number who delude themselves. Veteran psychic

investigator Eric Dingwall (1985:162) summed up his extensive experience

in parapsychological research with this observation: " After sixty

years' experience and personal acquaintance with most of the leading

parapsychologists of that period I do not think I could name a half dozen

whom I could call objective students who honestly wished to discover the

truth. "

 

Replicability

If the evidence for any claim is based upon an experimental result, or if

the evidence offered in support of any claim could logically be explained

as coincidental, then it is necessary for the evidence to be repeated in

subsequent experiments or trials.

The rule of replicability provides a safeguard against the possibility of

error, fraud, or coincidence. A single experimental result is never

adequate in and of itself, whether the experiment concerns the production

of nuclear fusion or the existence of telepathic ability. Any experiment,

no matter how carefully designed and executed, is always subject to the

possibility of implicit bias or undetected error. The rule of

replicability, which requires independent observers to follow the same

procedures and to achieve the same results, is an effective way of

correcting bias or error, even if the bias or error remains permanently

unrecognized. If the experimental results are the product of deliberate

fraud, the rule of replicability will ensure that the experiment will

eventually be performed by honest researchers.

If the phenomenon in question could conceivably be the product of

coincidence, then the phenomenon must be replicated before the hypothesis

of coincidence can be rejected. If coincidence is in fact the explanation

for the phenomenon, then the phenomenon will not be duplicated in

subsequent trials, and the hypothesis of coincidence will be confirmed;

but if coincidence is not the explanation, then the phenomenon may be

duplicated, and an explanation other than coincidence will have to be

sought. If I correctly predict the next roll of the dice, you should

demand that I duplicate the feat before granting that my prediction was

anything but a coincidence.

The rule of replicability is regularly violated by parapsychologists, who

are especially fond of misinterpreting coincidences. The famous

" psychic sleuth " Gerard Croiset, for example, allegedly solved

numerous baffling crimes and located hundreds of missing persons in a

career that spanned five decades, from the 1940s until his death in 1980.

The truth is that the overwhelming majority of Croiset's predictions were

either vague and nonfalsifiable or simply wrong. Given the fact that

Croiset made thousands of predictions during his lifetime, it is hardly

surprising that he enjoyed one or two chance " hits. " The late

Dutch parapsychologist Wilhelm Tenhaeff, however, seized upon those

" very few prize cases " to argue that Croiset possessed

demonstrated psi powers (Hoebens 1986a:130). That was a clear violation

of the rule of replicability, and could not have been taken as evidence

of Croiset's psi abilities even if the " few prize cases " had

been true. (In fact, however, much of Tenhaeff's data was fraudulent --

see Hoebens 1986b. )

 

Sufficiency

The evidence offered in support of any claim must be adequate to

establish the truth of that claim, with these stipulations:

 

the burden of proof for any claim rests on the claimant, extraordinary claims demand extraordinary evidence, and evidence based upon authority and/or testimony is always inadequate

for any paranormal claim

The burden of proof always rests with the claimant for the simple reason

that the absence of disconfirming evidence is not the same as the

presence of confirming evidence. This rule is frequently violated by

proponents of paranormal claims, who argue that, because their claims

have not been disproved, they have therefore been proved. (UFO buffs, for

example, argue that because skeptics have not explained every UFO

sighting, some UFO sightings must be extraterrestrial spacecraft.)

Consider the implications of that kind of reasoning: If I claim that

Adolf Hitler is alive and well and living in Argentina, how could you

disprove my claim? Since the claim is logically possible, the best you

could do (in the absence of unambiguous forensic evidence) is to show

that the claim is highly improbable -- but that would not disprove it.

The fact that you cannot prove that Hitler is not living in Argentina,

however, does not mean that I have proved that he is. It only means that

I have proved that he could be -- but that would mean very little;

logical possibility is not the same as established reality. If the

absence of disconfirming evidence were sufficient proof of a claim, then

we could " prove " anything that we could imagine. Belief must be

based not simply on the absence of disconfirming evidence but on the

presence of confirming evidence. It is the claimant's obligation to

furnish that confirming evidence.

Extraordinary claims demand extraordinary evidence for the obvious reason

of balance. If I claim that it rained for ten minutes on my way to work

last Tuesday, you would be justified in accepting that claim as true on

the basis of my report. But if I claim that I was abducted by

extraterrestrial aliens who whisked me to the far side of the moon and

performed bizarre medical experiments on me, you would be justified in

demanding more substantial evidence. The ordinary evidence of my

testimony, while sufficient for ordinary claims, is not sufficient for

extraordinary ones.

In fact, testimony is always inadequate for any paranormal claim, whether

it is offered by an authority or a layperson, for the simple reason that

a human being can lie or make a mistake. No amount of expertise in any

field is a guarantee against human fallibility, and expertise does not

preclude the motivation to lie; therefore a person's credentials,

knowledge and experience cannot, in themselves be taken as sufficient

evidence to establish the truth of a claim. Moreover, a person's

sincerity lends nothing to the credibility of his or her testimony. Even

if people are telling what they sincerely believe to be the truth, it is

always possible that they could be mistaken. Perception is a selective

act, dependent upon belief context, expectation, emotional and

biochemical states, and a host of other variables. Memory is notoriously

problematic, prone to a range of distortions, deletions, substitutions

and amplifications. Therefore the testimony that people offer of what

they remember seeing or hearing should always be regarded as only

provisionally and approximately accurate; when people are speaking about

the paranormal, their testimony should never be regarded as reliable

evidence in and of itself. The possibility and even the likelihood of

error are far too extensive (see Connor 1986).

 

Conclusion

The first three rules of FiLCHeRS -- falsifiability, logic, and

comprehensiveness -- are all logically necessary rules of evidential

reasoning. If we are to have confidence in the veracity of any claim

whether normal or paranormal, the claim must be prepositionally

meaningful, and the evidence offered in support of the claim must be

rational and exhaustive.

The last three rules of FiLCHeRS -- honesty, replicability, and

sufficiency -- are all pragmatically necessary rules of evidential

reasoning. Because human beings are often motivated to rationalize and to

lie to themselves, because they are sometimes motivated to lie to others,

because they can make mistakes, and because perception and memory are

problematic, we must demand that the evidence for any factual claim be

evaluated without self-deception, that it be carefully screened for

error, fraud, and appropriateness, and that it be substantial and

unequivocal.

What I tell my students, then, is that you can and should use FiLCHeRS to

evaluate the evidence offered for any claim. If the claim fails any one

of these six tests, then it should be rejected; but if it passes all six

tests, then you are justified in placing considerable confidence in

it.

Passing all six tests, of course, does not guarantee that the claim is

true (just because you have examined all the evidence available today is

no guarantee that there will not be new and disconfirming evidence

available tomorrow), but it does guarantee that you have good reasons for

believing the claim. It guarantees that you have sold your belief for a

fair price, and that it has not been filched from you.

Being a responsible adult means accepting the fact that almost all

knowledge is tentative, and accepting it cheerfully. You may be required

to change your belief tomorrow, if the evidence warrants, and you should

be willing and able to do so. That, in essence, is what skepticism means:

to believe if and only if the evidence warrants.

 

References

Connor, John W. 1984. Misperception, folk belief, and the occult: A

cognitive guide to understanding. SKEPTICAL INQUIRER, 8:344-354, Summer.

 

Dingwall, E. J. 1985. The need for responsibility in parapsychology: My

sixty years in psychical research. In A Skeptic's Handbook of

Parapsychology, 161-174, ed. by Paul Kurtz. Buffalo, N Y. Prometheus

Books.

Hines, Terence. 1988. Pseudoscience and the Paranormal Buffalo, N.Y

Prometheus Books.

Hoebens, Piet Hein. 1981. Gerard Croiset: Investigation of the Mozart of

" psychic sleuths. " SKEPTICAL INQUIRER, 6(1):1728, Fall.

 

-- -- -- . 1981-82. Croiset and Professor Tenhaeff Discrepancies in

claims of clairvoyance. SKEPTICAL INQUIRER, (2):21-40, Winter.

Hyman, Ray. 1985. A critical historical overview of parapsychology. In A

Skeptic's Handbook of Parapsychology, 3-96, ed. by Paul Kurtz Buffalo,

N.Y. Prometheus Books.

Omohundro, John T. 1976. Von Däniken's chariots primer in the art of

cooked science. SKEPTICAL INQUIRER, 1(1):58-68, Fall.

Story, Ronald D. 1977 Von Däniken's golden gods, SKEPTICAL INQUIRER,

2(1):22-35, Fall/Winter.

 

About the Author

James Lett is a Professor of Anthropology, Department of Social Sciences,

Indian River Community College, 3209 Virginia Avenue, Ft. Pierce, FL

34981. He is author of The Human Enterprise: A Critical Introduction to

Anthropologcal Theory and Science, Reason, and Anthropology: The

Principles of Rational Inquiry (1997, Rowman and Littlefield Publishers).

 

 

 

Copied

from:

 

http://www.totse.com/en/ego/self_improvement/afieldguidetoc174154.html