A Field Guide to Lies: Critical Thinking in the Information Age (34 page)

sqrt [{(Observed proportion) × [1 – (Observed proportion)}/sample size]

The width of the 95 percent confidence interval then is ±2 × standard error.

For example, if you sampled fifty overpasses in a large city, you might have found that 20 percent of them needed repair. You calculate the standard error as:

sqrt [(.2 x .8)/50] = sqrt (.16/50) = .057.

So the width of your 95 percent confidence interval is ±2 × .057 = ±.11 or ±11%. Thus the 95 percent confidence interval is that 20 percent of the overpasses in this town need repair, plus or minus 11 percent. In a news report, the reporter might say that the survey showed 20 percent of overpasses need repair, with a margin of error of 11 percent. To increase the precision of your estimate, you need to sample more. If you go to 200 overpasses (assuming you obtain the same 20 percent figure), your margin of error reduces to about six percent.

this conventional explanation is wrong
:
Lusinchi, D. (2012). “President” Landon and the 1936
Literary Digest
poll: were automobile and telephone owners to blame?
Social Science History, 36
(1), 23–54.

An investigation uncovered serious flaws
:
Clement, S. (2013, June 4). Gallup explains what went wrong in 2012.
Washington Post
. https://www.washingtonpost.com/news/the-fix/wp/2013/06/04/gallup-explains-what-went-wrong-in-2012/.

http://www.gallup.com/poll/162887/gallup-2012-presidential-election-polling-review.aspx.

trying to figure out what proportion of jelly beans
:
Taken from http://www.ropercenter.uconn.edu/support/polling-fundamentals-total-survey-error/.

what magazines people read
:
Elaborated from an example in Huff, op. cit., p. 16.

Gleason scoring
:
This definition taken verbatim from http://www.cancer.gov/publications/dictionaries/cancer-terms?cdrid=45696. Accessed March 20, 2016.

they had made an error in measurement
:
Jordans, F. (2012, Feb. 23). CERN researchers find flaw in faster-than-light measurement.
Christian Science Monitor
. http://www.csmonitor.com/Science/2012/0223/CERN-researchers-find-flaw-in-faster-than-light-measurement.

1960 U.S. Census study recorded
:
This is from De Veaux, R. D., & Hand, D. J. (2005). How to lie with bad data.
Statistical Science, 20
(3), 231–238, p. 232. They cite Kruskal, W. (1981). Statistics in society: problems unsolved and unformulated.
Journal of the American Statistical Association, 76
(375), 505–515, and Coale, A. J., & Stephan, F. F. (1962). The case of the Indians and the teen-age widows.
Journal of the American Statistical Association, 57
, 338–347.

claimed measurement error as part of their defense
:
Kryk, J. Patriots strike back with compelling explanations to refute deflate-gate chargers.
Ottowa Sun
, May 15, 2015. http://www.ottawasun.com/2015/05/14/patriots-strike-back-with-compelling-explanations-to-refute-deflate-gate-chargers.

statistic you encounter may not have defined homelessness
:
This example from Spirer, H., Spirer, L., & Jaffe, A. J. (1998).
Misused Statistics,
2nd ed., revised and expanded. New York: Marcel Dekker, p. 16.

Imagine that you’ve been hired by a political candidate
:
This example based on one in Huff, op. cit., p. 80.

A newspaper reports the proportion of suicides
:
From Best (2005), op. cit.

I’m not going to wear my seat belt because
:
This example comes from Best, J. (2012), and my childhood friend Kevin.

the idea of symmetry and equal likelihood
:
The principle of symmetry can be broadly construed to include instances where outcomes are not equally likely but still prescribed, such as a trick coin that is weighted to come up heads two-thirds of the time, or a roulette wheel in which some of the troughs are wider than others.

If we run the experiment on a large number of people
:
We could also conduct the experiment with a small number of people many times, in which case we would expect to obtain different numbers. In this case, the true probability of the drug working is going to be somewhere close to the average (the mean) of the numbers obtained in all the experiments, but it’s an axiom of statistics that larger samples lead to more accurate results.

Both classic and frequentist probabilities deal with
:
Classic probability can be thought of in two different ways: empirical and theoretical. If you’re going to toss a coin or draw cards from a shuffled deck, each time you do this is like a trial in an experiment that could go on indefinitely. In theory, you could get thousands of people to toss coins and pick cards for several years and tally up the results to obtain the proportion of time that different outcomes occur, such as “getting heads” or “getting heads three times in a row.” This is an
empirically derived
probability. If you believe the coin is fair (that is, there’s no manufacturing defect that causes it to come up on one side more than the other), you don’t need to do the experiment, because it should come up heads half of the time (probability = .5) in the long run, and we arrive at this
theoretically
, based on the understanding that there are two equally likely outcomes. We could run a similar experiment with cards and determine empirically and theoretically that the chances of drawing a heart are one in four (probability = .25) and that the chances of drawing the four of clubs is one in fifty-two (probability ≅ .02).

When a court witness testifies about the probability
:
Aitken, C. G. G., & Taroni, F. (2004).
Statistics and the Evaluation of Evidence for Forensic Scientists,
2nd ed. Chicester, UK: John Wiley & Sons.

In Tversky and Kahneman’s experiments
:
Tversky, A., & Kahneman, D. (1974). Judgment under uncertainty: heuristics and biases.
Science
,
185
(4157), 1124–1131.

A telltale piece of evidence that this is subjective
:
For further discussion, and more formal treatment, see Iversen, G. R. (1984).
Bayesian Statistical Inference.
Thousand Oaks, CA: Sage, and references cited therein.

the case of Sally Clark
:
I thank my student Alexandra Ghelerter for this example. See also Nobles, R., & Schiff, D. (2007). Misleading statistics within criminal trials.
Medicine, Science and the Law, 47
(1), 7–10.

relative incidence of pneumonia
:
http://www.nytimes.com/health/guides/disease/pneumonia/prognosis.html.

Bayes’s rule to calculate a conditional probability
:
Bayes’s rule is:

The probability that a woman has breast cancer
:
This paragraph, and this discussion, quotes nearly verbatim from Krämer, W., & Gigerenzer, G. (2005). How to confuse with statistics or: the use and misuse of conditional probabilities.
Statistical Science, 20
(3), 223–230.

To make the numbers work out easily
:
How do you know what number to choose? Sometimes it takes trial and error. But it’s also possible to figure it out. Because the probability is .8 percent, or eight people per thousand, if you chose to build a table for 1,000 women you’d end up with eight in one of the squares, and that’s okay, but later on we’re going to be multiplying that by 90 percent, which will give us a decimal. There’s nothing wrong with that, it’s just less convenient for most people to work with decimals. Increasing our population by an order of magnitude to 100 gives us all whole numbers, but then we’re looking at larger numbers than we need. It doesn’t really matter because all we’re looking for is probabilities and we’ll be dividing one number by another anyway for the result.

If you read that more automobile accidents occur at seven p.m.
:
Still confused? If there were eight times as many cars on the road at seven p.m. than at seven a.m., the
raw
number of accidents could be higher at seven p.m., but that does not necessarily mean that the
proportion
of accidents to cars is greater. And
that
is the relevant statistic to you: not how many accidents happen at seven p.m., but how many accidents occur per thousand cars on the road. This latter formulation quantifies your risk. This example is modified from one in Huff, op. cit., p. 78, and discussed by Krämer & Gigerenzer (2005).

90 percent of doctors treated the two
:
Cited in Spirer, Spirer, & Jaffe, op. cit., p. 197: Thompson, W. C., & Schumann, E. L. (1987). Interpretation of statistical evidence in criminal trials,
Law and Human Behavior, 11
(167).

One surgeon persuaded ninety women
:
From Spirer, Spirer, & Jaffe, op. cit., first reported in Hastie, R., & Dawes, R. M. (1988).
Rational Choice in an Uncertain World.
New York: Harcourt Brace Jovanovich.

The original report of the surgeon’s work appeared in McGee, G. (1979, Feb. 6). Breast surgery before cancer.
Ann Arbor News,
p. B1 (reprinted from the
Bay City News
).

As sociologist Joel Best says
:
Best, op. cit., p. 184.

PART TWO: EVALUATING WORDS

Steve Jobs delayed treatment for his pancreatic cancer
:
Swaine, J. (2011, Oct. 21). Steve Jobs “regretted trying to beat cancer with alternative medicine for so long.” http://www.telegraph.co.uk/technology/apple/8841347/Steve-Jobs-regretted-trying-to-beat-cancer-with-alternative-medicine-for-so-long.html.

an article in
Forbes
that claims
:
Rees, N. (2009, Aug. 13). Policing word abuse.
Forbes
. http://www.forbes.com/2009/08/12/nigel-rees-misquotes-opinions-rees.html.

Respectfully Quoted
, a dictionary of quotations
:
Platt, S., ed. (1989).
Respectfully Quoted
. Washington, D.C.: Library of Congress. For sale by the Supt. of Docs., USGPO.

That book reports various formulations
:
Billings, J. (1874).
Everybody’s Friend, or Josh Billing’s Encyclopedia and Proverbial Philosophy of Wit and Humor.
Hartford, CT: American Publishing Company.

humans had twenty-four pairs of chromosomes instead of twenty-three
:
Gartler, S. M. (2006). The chromosome number in humans: a brief history.
Nature Reviews Genetics
,
7
, 655–660. http://www.nature.com/scitable/con tent/The-chromosome-number-in-humans-a-brief-15575. Glass, B. (1990).
Theophilus Shickel Painter
. Washington, D.C.: National Academy of Sciences. http://www.nasonline.org/publications/biographical-memoirs/memoir-pdfs/painter-theophilus-shickel.pdf. Retrieved November 6, 2015.

If people in the arts and humanities have won a prize
:
Paul Simon, Stevie Wonder, and Joni Mitchell can be considered experts in songwriting. Although they do not hold university positions, university scholars have written books and articles about them, and Mr. Simon and Mr. Wonder were recognized by the president of the United States with Kennedy Center Honors, reserved for individuals who have made great contributions to performing arts. Ms. Mitchell received an honorary doctorate of music and won the Polaris Music Prize.