Charles Rulon
Where people in general fail in their ability to think critically or scientifically is in the area of controls. Endless cause and effect errors and wrong conclusions have resulted because of our failure to consider the many variables in a situation. Testimonials, anecdotes and biased data gathering make up the evidential backbone of pseudoscience, paranormal beliefs, religions and quackery. But testimonials and anecdotes do not a science make.
Consider the following example:
“I’m convinced that vegetarians live longer. Many articles on the web agree with me and as proof both my mother and father were vegetarians and they lived into their 90’s. My friend, however, suggested that maybe my parents had good genes, and/or were very lucky, and/or had excellent health habits. Or maybe they did eat meat and lied. She even hinted that maybe I was exaggerating to win the argument. Picky! Picky!”
It is the failure of the public to both understand the rigor of the Scientific Method and to appreciate its power to discover empirical truths that has enabled creationism to be promoted over evolution. It has enabled “alternative medicine” charlatans to thrive, global warming deniers to pollute the media, and politicians to ignore its findings in shaping public policy.
Controls & subliminal learning
The belief in the effectiveness of subliminal messages and advertising used to be widespread. Some people believed they had become so good (or paranoid) at discovering subliminal messages that they saw such messages in everything from soap commercials to ancient works of art. Audiocassettes claiming to have subliminal messages that would produce everything from quick weight loss to peace of mind were hauling in $50 million annually. Testimonials of satisfied customers abounded. True believers were everywhere. But did these tapes really work? After all, if subliminal advertising really did work, wouldn’t there be numerous professional textbooks, conferences and workshops telling the pros how it’s best done? But there weren’t. When the variables were tightly controlled, subliminal messages didn’t work.
Consider one actual experiment: At the University of Washington 237 students listened to commercially available subliminal tapes aimed at improving one’s memory or self-esteem. The messages were hidden behind sounds of ocean waves. Each volunteer completed a series of both memory and self-esteem tests before and after using either a memory tape or a self-esteem tape for one month.
But unknown to the volunteers, the researchers had reversed the labels on half the tapes. So, 1/4th thought they got the memory tape and did get the memory tape; 1/4th thought they got the memory tape, but instead got the self-esteem tape; 1/4th thought they got the self-esteem tape and did get it; 1/4th thought they got the self-esteem tape, but instead got the memory tape.
At the end of one month these four groups were all tested again. Results: No group actually showed any more improvement in self-esteem or in memory than any other group. Yet, of interest, those who thought they had the memory tape were still convinced that their memories had improved and those who thought they were listening to a self-esteem tape (whether they were or not) remained convinced that their self-esteem had improved. (It would seem that we could all benefit from a course in the psychology of self-deception.)
After dozens of such experiments, the National Academy of Sciences concluded over 20 years ago (1991) that there was neither a theoretical foundation nor experimental evidence to support claims that subliminal self-help tapes enhanced performance.[i]
Controls: marijuana and auto accidents
Consider the following quote: “Persons arrested for use of marijuana have, on the average, 40% more auto accidents and 180% more traffic violations than do average drivers of the same age and sex. It’s obvious, therefore, that marijuana adversely affects driving.” Yet, are all these accidents and violations really due to the effects of marijuana, itself? Several variables need to be controlled if we wish a scientific answer. First, is the quote accurate? Second, maybe the same type of person who is willing to break the law to smoke marijuana also has less respect for our traffic laws in general. Third, maybe people high on marijuana are also more likely to be on other drugs which also decrease their driving ability. Fourth, maybe the paranoia and fear of being arrested on a drug charge contributes to careless driving.
Controls: the plague, God & christening
In Games, God and Gambling (1962), the author asserts that one purpose of keeping vital statistics several centuries ago was to understand the intentions of God. For example, the bubonic plague in the 1600s was recorded to be much more serious in those years when fewer children were christened. As a result, it was preached that the plague was God’s punishment for parents who didn’t christen their children. But assuming this data is even reliable, perhaps fewer children were christened during the plague years simply because more terrified parents fled to the hills with their children before they could be christened.
Controls: vitamin–C and colds
In one experiment 300 university students were given vitamin-C all winter. They had 65% fewer colds than the winter before. But, in addition, another randomly selected group of 300 students at the same college were each given what they thought was vitamin-C but, instead, was a placebo. Also, to make the experiment double-blinded, the dispenser of the real and the fake vitamin-C didn’t know which was which. Results: The placebo group also had about 65% fewer colds. Conclusion: in this experiment vitamin-C wasn’t directly responsible for the decrease in colds.
Controls: Join the navy, it’s safer
During World War II a poster read: “During wartime, for every 1000 people in the navy, fewer will die than for every 1000 people in New York City. Join the navy. It’s safer!” Q. What major variable was not considered?
Ans. The average physical characteristics of people selected for the navy (young, healthy) are markedly different from those who remained behind in New York City.
Controls: the Fremont Christian Clinic
The former Fremont Christian Clinic in Los Angeles would take an X-ray of any person who came in complaining of indigestion or constipation. The X-ray would show “a colon cancer which was still curable.” A barrage of expensive pills would follow for many months. Finally, a second X-ray would be taken. The cancer was gone! A miraculous cure! Grateful patients were only too happy to write glowing testimonials, or even to appear in court on behalf of the clinic. Q. What major variable was not controlled? Ans: The X-rays were phony. How many of us can recognize our own X-ray?
Controls: sexually abused children
When children who claimed to have been sexually abused were given anatomically correct dolls (having a rectum and a penis or vagina) to play with, they tended to concentrate on the genitals and would even stick things up the vagina and rectum. Such behavior was used as a test to determine if a child really had been molested, or was just making it up. Q. What control was missing for this to be a scientific test? Ans. If children who were not sexually abused were also given anatomically correct dolls, they would behave in the same way toward the dolls as did the molested children.
To summarize
In general, where people fail in their ability to think critically or scientifically is in their under-appreciation of the importance of controls. Whether we’re talking about subliminal learning, vitamin-C or whatever, tightly controlling all the variables is what separates science from non-science. This is one of the first things scientists must learn. Endless cause and effect errors and wrong conclusions have resulted from failure to control all the variables. Until people become aware of the critical necessity for rigorous controls, they will remain essentially unscientific, no matter how many scientific facts they may have memorized. They will remain like the rooster who crows, notices a moment later that the sun rises and then concludes that it’s because of his crowing.
Charles L. Rulon is an emeritus in the Life Sciences department at Long Beach City College
[i]In the Mind’s Eye, National Research Council, National Academy of Sciences (National Academy Press, Washington, 1991, p. 15-16).