Science versus Supposition
In 1989 on the Today Show Bryant Gumbel, a very intelligent
man, interviewed Dr. Charles Hennekens about a groundbreaking study that showed
participants in a Harvard study reduced their incidents of heart attacks by 47%
by taking one aspirin every other day.
Gumbel then asked – Could you reduce the risk of a heart attack by 94%
by taking one aspirin every day?
Like many intelligent people, Gumbel did not understand the
numerical results.
Gulf War What?
In 1996 the New England Journal of Medicine published a
compilations of studies. Involving tens of thousands of veterans who served in
the Persian Gulf compared to veterans of comparable age who did not serve
there. The study found no
different death rates, hospitalization rates or rates of any self-reported
symptoms. When I heard this
reported on the news the announcer read the items and then said “but that does
not explain why Gulf War veterans have so many unexplained problems.”
I was barely able to refrain from throwing my shoe at the
television. The study showed there were no differences between the groups of
veterans on symptoms that have been called “Gulf War Syndrome.” Note: this does not mean that any particular
person who describes himself/herself as having the syndrome is exaggerating or
is not really ill.
Which end of the telescope do we look though?
Alan Dershowitz during the OJ Simpson trial wrote to the LA
Times and noted that although Mr. Simpson might have abused his wife, in any
given year roughly 1,500 women are murdered by a current or former abusive
spouse while there are 2 to 4 million spousal assaults. The odds of an abuser being a killer
were roughly one in a thousand.
Problem? Nicole Simpson was murdered. Dr. Kevin Hays started from that fact and found that in 1992
of 890 murders of women who had been abused, 715 were killed by their
abusers. These odds were roughly
80 out of 100. How the question is
asked makes a gigantic difference.
Numbers have different meaning based on what they are used
for. One common use is for identification. Your address, phone number and social security number are
examples. Add together the social
security numbers of everyone in your family, divide by the number of people in
your family and you get …a meaningless number.
Absolute numbers are sums, e.g. there are 25 students
enrolled in a class.
Numbers can be used to find percentages. 45% of students in a class are
male.
Numbers can also be used for measurement. When a nurse or a
doctor asks —Are you in pain?
On a scale of 0 to 10 with 0 being no pain at all and 10 being the worst
pain imaginable how would you rate your pain? You give a number that can be compared with your answer to
that exact question in the past or in the future. It doesn’t matter that your 4 would be an 8 for me. (I’m a wimp.) There is no possible meaning to comparing different people
on that question but the scale is surprisingly helpful with a single person.
Wanna’ Bet?
Numbers can be used to calculate odds. If you flip an
unbiased coin chances are that on any one flip it will come up heads is 50% and
chances it will come up tails is 50%.
If I flip an unbiased coin nine times and it comes up heads nine
times. The odds of it coming up
head on the next flip is …50%; the odds of it coming up tails is also 50%.
“Gambler’s error” is thinking that any set of past random results will effect
the next random event in a sequence.
For scientific research to be used as evidence in federal
court it must meet Daubert Standards
1) the
testing and technique must be reliable (testing and retesting scores of the
same people over a short period of time should be similar)
2) must
be capable of showing the finding is wrong (which is why intelligent design is
not a scientific theory)
3) published
and peer reviewed (subject to criticism by knowledgeable people)
4) known
or potential error rate (no measurement is perfect)
5) generally
accepted by the scientific community
6) have
sufficient data; generally more is better (to calculate the odds of an unbiased
flipped coin coming up heads 10,000 flips is more likely to be accurate than 10
flips.)
7) relevant
to the court case in question
I could go on but I suspect the odds of you continuing to
find this interesting are decreasing.
Have you learned anything?
When have you noticed misunderstanding of numbers?
I have been known to yell at radio and TV when the announcer, commentator or (loudest reaction by me) supposed expert guest uses math and statistics improperly. Often it is a lack of understanding of the basic principals--sometimes, particularly with experts paid by industry, it is to be deceitful.
ReplyDeleteThe line goes: there are liars, damn liars and statisticians.
My father was a statistician and I was a math major. I have forgotten almost all I learned in college math, but this kernel I did retain: the answer depends on the assumptions.
A simple example will illustrate this. The sum of the angles of a triangle add to 180 degrees, right? Yes -- but if and only if the triangle resides on a plane. If the triangle is on a convex curved surface (like a globe) the three angles will sum to more than 180 degrees (and on a concave surface they will add to less than 180 degrees).
I could go on for several hundred pages of examples and explanations of errant math, but I think I'll stop for now.
Two interesting books on this subject are Innumeracy and Ruminations of a Numbers Man, both by John Allen Paulos.
~ Jim
I took statistics in grad school and enjoyed it, which was a great surprise to me. The professor stated the statistic principle in English, then wrote it mathematically on the board, then restated it using different words to convey the meaning. By the time she finished, everyone's light bulb shone. I was fortunate to have a professor whose talents included both English and Math.
ReplyDeleteWhen my daughter was in elementary school a misguided teacher picked her out to demonstrate probability to the class. She held up a deck of cards to prove that 50% of the time, she'd get a higher card than my daughter. She never proved her point. Little did she know my daughter was a card witch--beat her every time!
While some statistics illuminate, others conceal.
Thanks for the references, Jim.
ReplyDeleteI'm glad you had such an interesting professor and sorry your daughter had an elementary teacher with so little knowledge.
ReplyDeleteIt's easy to get probabilities wrong when you have a small sample.
Interesting blog, Warren. I get irritated by misleading poll numbers all the time. As for pain on a scale of 1 to 10, I'm a wimp, too. I doubt I'd last more than a few minutes or even seconds if I were being tortured to tell some secret information.
ReplyDeleteGloria, Luckily you and I are not spies.
ReplyDeleteI learned a valuable lesson from a former boss about numbers. I was stressing about being behind schedule and over budget on a project, but he was strangely relaxed. "It's not the number of beans you have," he told me. "It's the way you count them that matters."
ReplyDeleteSomehow he managed to prove to upper management that we were on track.
Our tendency is to believe quoted statistics as proof, not realizing they can be misused. One of the books I used to teach college freshman Expository Writing was called Asking the Right Questions. The book did a good job of demonstrating the misuse of statistics.
ReplyDeleteIn order to log on to a work computer from home, my husband has to type in six numbers that are displayed on a random number generator. About every 30 seconds the numbers switch. Even though I know it’s generating completely random numbers, if I watch it long enough I “see” a pattern.
ReplyDeleteKara,
ReplyDeleteWe humans are hard-wired to find patterns, even when they don't exist and ignore patterns when they do not fit our perceived gesalt.
When I was a student teacher, I had a high school class that was getting ready for their regents exam (state-wide exam in NY). I gave them a sample test where all of the answers were (c).
It drove them bonkers. They erased correct answers so as to spread their answers among a, b & d. Despite the mounting evidence, they always guessed something other than (c).
My point was to get them to guess based on the facts of the question, not the pattern of answers -- and to keep their calculated answers regardless of the pattern emerging.
Point made and all of them did pass the test. Several told me afterward that after the crap I had put them through, the actual test was not stressful at all.
~ Jim
Your sample test would have made me bonkers, too, Jim. But what a great lesson about answering questions based on the facts instead of the pattern of answers. My guess is that your students still remember this lesson.
ReplyDelete