Predictive Power of Scientific Polls
by Gordon Prather
wondered what if anything is scientific about a "scientific"
poll, which purports to show that if the election were held today,
50 percent would vote for TweedleDee and 50 percent would vote for
TweedleDum with a margin of "error" of plus or minus 5 percent.
is there is nothing scientific about it. In fact, political pollsters
turn statistical sampling/analysis which has no predictive
power in the ordinary sense of that term on its head.
How does a
real scientist go about getting a good estimate of the whole by
statistically analyzing samples of it?
example, suppose she wishes to know the temperature of a distant
object. She knows all objects radiate heat and light over a relatively
large range spectrum of energies and that the single-humped
shape of that energy spectrum is uniquely and directly related to
the absolute temperature of the radiating body.
believe it or not, the Black-Body Spectrum and it was first derived
by Max Planck, one of the founding fathers of quantum mechanics.)
So, all she
has to do to is to measure the amount of energy emitted per unit
time at a few hundred randomly selected places on the spectrum,
over and over again, until she has satisfied herself that she effectively
knows the shape of its Black-Body Spectrum. Once she knows the shape
of the spectrum, she can work backwards to "unfold" the
temperature of the distant object that produced it.
The more accurately
she wishes to know the temperature, the more sets of samples she
will have to take.
of error" quoted by a scientist for the black-body temperature has
the following meaning: if she took many, many samples of the whole
spectrum at a thousand randomly selected points, and from that determined
that the radiating body had an absolute temperature of 300 degrees
Kelvin, with a margin of error of 5 percent, that means statistically
that if she were to take three more independent sets of samples
of the same radiated spectrum, then the temperature determined from
two of those samplings would be somewhere between 285 degrees
and 315 degrees Kelvin.
the temperature determined in one of those statistical samplings
would be either greater than 315 or less than 285 degrees Kelvin.
The key point
in this scientific sampling/analysis is that if there had not existed
completely independent of what the scientist did or didn't do
a characteristic black-body spectrum being radiated by the hot
object, then there would have been nothing to sample.
can't sample anything unless there is something definitive for you
to sample that is not affected by your sampling.
The only way
to sample the opinion of 10 million voters on any subject is to
first obtain the opinions of those 10 million voters on a specific
subject, and, having gotten all those opinions call them "ballots"
piled on your desk, then sample the ballots, randomly selecting
a thousand or so of those 10 million ballots and tabulating the
sample of a thousand ballots, when analyzed, results in 500 anticipated
votes for TweedleDee and 500 for TweedleDum.
What does a
truly scientific pollster now know?
all she knows is that in that sample, its 50/50 for Dee/Dum.
What does that
one sample tell her about the other 9,999,000 ballots piled on her
she takes a hundred such samples, and unbelievably, in 50 of those
samples it comes out 52/48 Dee/Dum and in the other 50 samples it
comes out 48/52 Dee/Dum.
Now were cookin.
Now she can apply statistical analysis to the results of her samples
and make predictions with some degree of confidence about what
she would expect to find if she took additional samples of the nine-million
ballots piled on her desk.
Note that she
is still not in a position to make any prediction
with any "margin of error" of what she would
find if she tabulated all the ballots on her desk.
But, you say,
what about an exit poll? Surely that is semi-scientific sort of?
Suppose the pollster does wait till election day and asks as they
leave the voting booths 1,000 randomly selected voters who they
just voted for.
sampling, sort of? Aren't there effectively 10 million opinions
now stacked on the pollster's desk and isn't she essentially selecting,
randomly, a thousand of those?
In terms of
popular vote, most recent presidential elections have been fairly
close. The odds are that the votes will be about evenly split, nationwide,
between the candidate riding the elephant and the candidate riding
not in the District of Columbia. But the elephant/donkey split even
there in 2008 is likely to be about the same as it was in 2000,
the last time an incumbent President was not seeking re-election.
is that, even in DC, it is statistically possible that the first
1,000 voters exit-polled will tell the exit pollster that they just
voted for Ron Paul. Or, since it is not yet a federal crime to lie
to a pollster, every one of them may say they voted for Donald Duck.
Well you can
bet that, if that is what the exit pollster gets told, the main-stream-media
is not going to predict the election of Ron Paul (or Donald Duck)
by a landslide with a margin of error of plus/minus 5 percent.
Now, how scientific
is that, throwing out the results you don't like?
James Gordon Prather [send him mail]
has served as a policy-implementing official for national security-related
technical matters in the Federal Energy Agency, the Energy Research
and Development Administration, the Department of Energy, the Office
of the Secretary of Defense and the Department of the Army. Dr.
Prather also served as legislative assistant for national security
affairs to U.S. Sen. Henry Bellmon, R-Okla. ranking member
of the Senate Budget Committee and member of the Senate Energy Committee
and Appropriations Committee. Dr. Prather had earlier worked as
a nuclear weapons physicist at Lawrence Livermore National Laboratory
in California and Sandia National Laboratory in New Mexico.
© 2007 LewRockwell.com