Charles Rulon

 Where people in general fail in their ability to think critically or scientifically is in the area of con­trols.  Endless cause and ef­fect errors and wrong con­clusions have resulted because of our failure to con­sider the many variables in a situa­tion.  Testimonials, anecdotes and biased data gathering make up the evidential back­­bone of pseu­do­sci­ence, paranor­mal be­liefs, religions and quack­ery.  But testi­monials and anecdotes do not a science make.

Con­sider the following example:

“I’m convinced that vegetarians live longer.  Many articles on the web agree with me and as proof both my mother and father were vegetarians and they lived into their 90’s.  My friend, however, suggested that maybe my parents had good genes, and/or were very lucky, and/or had excellent health habits.  Or maybe they did eat meat and lied.  She even hinted that maybe I was exagger­ating to win the argument. Picky! Picky!”

It is the failure of the public to both understand the rigor of the Scientific Method and to appreciate its power to discover empirical truths that has enabled creationism to be promoted over evolution.  It has enabled “alternative medicine” charlatans to thrive, global warming deniers to pollute the media, and politicians to ignore its findings in shaping public policy.

Controls & subliminal learn­ing

The belief in the effectiveness of subliminal mes­sages and advertising used to be widespread.  Some people believed they had become so good (or paranoid) at dis­cov­ering subliminal messages that they saw such messages in every­thing from soap commercials to ancient works of art.  Audiocassettes claiming to have sub­liminal mes­sages that would pro­duce every­thing from quick weight loss to peace of mind were hauling in $50 million annually.  Testimonials of satisfied customers abounded.  True believ­ers were everywhere.  But did these tapes really work?  After all, if sublimi­nal advertising really did work, wouldn’t there be numerous profes­sional text­books, conferences and work­­shops telling the pros how it’s best done?  But there weren’t.  When the vari­ables were tightly controlled, subliminal mes­sages didn’t work.

Consider one actual exper­iment: At the Uni­versity of Wash­ing­ton 237 students listened to com­mer­cially available sub­limi­­nal tapes aimed at im­proving one’s mem­ory or self-es­teem.  The messages were hidden behind sounds of ocean waves.  Each vol­unteer com­­pleted a series of both memory and self-esteem tests before and after using either a memory tape or a self-es­teem tape for one month.

But unknown to the volunteers, the re­searchers had reversed the labels on half the tapes.  So, 1/4th thought they got the memory tape and did get the mem­ory tape; 1/4th thought they got the memory tape, but instead got the self-esteem tape; 1/4th thought they got the self-esteem tape and did get it; 1/4th thought they got the self-esteem tape, but in­stead got the memory tape.

At the end of one month these four groups were all tested again.  Results: No group actually showed any more improve­ment in self-esteem or in memory than any other group.  Yet, of interest, those who thought they had the mem­ory tape were still convinced that their memories had improved and those who thought they were listen­ing to a self-esteem tape (whether they were or not) remained con­vinced that their self-esteem had improved. (It would seem that we could all benefit from a course in the psy­chology of self-deception.)

After dozens of such experiments, the Nation­al Acade­my of Sciences concluded over 20 years ago (1991) that there was neither a theoreti­cal founda­tion nor experi­mental evidence to sup­port claims that subliminal self-help tapes enhanced performance.[i]

Controls: marijuana and auto accidents

Consider the following quote:  “Persons arrested for use of marijuana have, on the av­erage, 40% more auto accidents and 180% more traffic vio­lations than do average drivers of the same age and sex.  It’s obvi­ous, there­fore, that marijuana ad­versely affects driving.”  Yet, are all these accidents and violations really due to the effects of marijuana, itself? Several variables need to be controlled if we wish a scientific answer.  First, is the quote accurate?  Second, maybe the same type of per­son who is willing to break the law to smoke mari­juana also has less respect for our traffic laws in general.  Third, maybe people high on marijuana are also more like­ly to be on other drugs which also decrease their driving ability.  Fourth, maybe the paranoia and fear of being arrested on a drug charge con­tributes to careless driving.

Controls: the plague, God & christening

In Games, God and Gamb­ling (1962), the author asserts that one purpose of keeping vital statistics several centuries ago was to understand the intentions of God.  For example, the bubonic plague in the 1600s was recorded to be much more serious in those years when fewer children were christened.  As a result, it was preached that the plague was God’s punishment for par­ents who didn’t christen their chil­dren.  But assuming this data is even reliable, perhaps fewer children were christened during the plague years simply because more terrified parents fled to the hills with their children before they could be christened.

Controls: vitamin–C and colds

In one experiment 300 university students were given vitamin-C all winter.  They had 65% fewer colds than the winter before.  But, in addition, another randomly selected group of 300 students at the same college were each given what they thought was vitamin-C but, instead, was a placebo.  Also, to make the experiment double-blinded, the dispenser of the real and the fake vitamin-C didn’t know which was which.  Results: The placebo group also had about 65% fewer colds.  Conclusion: in this experiment vitamin-C wasn’t directly responsible for the decrease in colds.

Controls: Join the navy, it’s safer

During World War II a poster read: “During wartime, for every 1000 people in the navy, fewer will die than for every 1000 people in New York City.  Join the navy.  It’s safer!”   Q.  What major variable was not con­sidered?

Ans.  The average phy­sical characteristics of people selected for the navy (young, healthy) are markedly different from those who re­mained behind in New York City.

Controls: the Fremont Christian Clinic

The former Fremont Christian Clinic in Los Angeles would take an X-ray of any person who came in complaining of indigestion or constipation.  The X-ray would show “a colon cancer which was still curable.”  A barrage of expensive pills would follow for many months.  Finally, a second X-ray would be taken.  The can­cer was gone!  A miraculous cure!  Grateful patients were only too happy to write glowing testi­monials, or even to appear in court on behalf of the clinic.  Q.  What major variable was not con­trolled?  Ans: The X-rays were phony.  How many of us can recognize our own X-ray?

Controls: sexually abused children

When children who claimed to have been sexually abused were given anatomically correct dolls (having a rectum and a penis or vagina) to play with, they tended to con­centrate on the genitals and would even stick things up the vagina and rectum.  Such behavior was used as a test to determine if a child really had been molested, or was just making it up.  Q.  What control was missing for this to be a scientific test?  Ans. If children who were not sexually abused were also given anatomically correct dolls, they would behave in the same way toward the dolls as did the molested children.

To summarize

In general, where people fail in their ability to think critically or scientifically is in their under-appreciation of the importance of controls.  Whether we’re talking about subliminal learning, vitamin-C or whatever, tightly controlling all the variables is what separates science from non-science.  This is one of the first things scientists must learn.  Endless cause and ef­fect errors and wrong con­clu­sions have resulted from failure to control all the variables.  Until people become aware of the critical neces­sity for rigorous con­trols, they will remain essentially un­sci­en­tific, no matter how many scien­tific facts they may have memorized.  They will remain like the rooster who crows, notices a moment later that the sun rises and then con­cludes that it’s because of his crowing.

Charles L. Rulon is an emeritus in the Life Sciences department at Long Beach City College