Statistics for Dummies (29 page)
To report the results from this poll, you would say, "Based on my sample, 52% of all Americans approve of the president, plus or minus a margin of error of 3.1%." (Hey, you sound almost as good as Gallup!)
How does a polling organization report its results? Here is basically how it's done:
Based on the total sample of adults in (this) survey, we are 95% confident that the margin of error for our sampling procedure and its results is no more than ± 3.1 percentage points.
It sounds in a way like that long list of disclaimers that comes at the end of a car-leasing advertisement. But now you can understand the fine print!
| REMEMBER | Never accept the results of a survey or study without the margin of error for the study. The margin of error is the only way to measure how close the sample information is to the actual population you're interested in. Sample results vary, and if a different sample had been chosen, a different sample result may have been obtained; you need the margin of error to be able to say how close the sample results are to the actual population values. The next time you hear a media story about a survey or poll that was conducted, take a closer look to see if the margin of error is given. Some news outlets are getting better about reporting the margin of error for surveys, but what about other studies? |
When a research question asks people to give a numerical value (for example, "How many people live in your house?"), the statistic used to report the results is the average of all the responses provided by people in the sample, otherwise known as the
sample average
.
The general formula for margin of error for your sample average is
, where s is the sample standard deviation,
n
is the sample size, and Z is the appropriate Z-value for your desired level of confidence (from
Table 10-1
).
Here are the steps for calculating the margin of error for a sample average:
Find the sample standard deviation,
s
, and the sample size,
nFore more information on how to calculate the average and standarddeviation, see
Chapter 5
.Devide the sample standard deviation by the square root of the sample size.
You now have standard error.
Multiply by the appropriate Z-value (refer to
Table 10-1
).The Z-value is 1.96 if you want to be about 95% confident.
For example, suppose you're the manager of an ice cream shop, and you're training new employees to be able to fill the large-size cones with the proper amount of ice cream (10 ounces each). You want to estimate the average weight of the cones they make, including a margin of error. You ask each of your new employees to randomly spot check the weights of the large cones they make and record those weights on a notepad. For
n
= 50 cones sampled, the average was found to be 10.3 ounces, with a sample standard deviation of
s
= 0.6 ounces. What's the margin of error? (Assume you want a 95% level of confidence.) It would be calculated this way:
So, to report these results, you would say that based on the sample of 50 cones, you estimate that the average weight of all large cones made by the new employees to be 10.3 ounces, with a margin of error of plus or minus 0.17 ounces.
| HEADS UP | Notice in the ice-cream-cones example, the units are ounces, not percentages! When working with and reporting results about data, always remember what the units are. Also be sure that statistics are reported with their correct units of measure, and if they're not, ask what the units are. |
| Tip | To avoid round-off error in your calculations, keep at least two non-zero digits after the decimal point throughout each step of the calculations. Round-off errors tend to accumulate, and you can be way off by the time you're finished if you round off too soon. |
If you want to be more than 95% confident about your results, you need to add and subtract more than 2 standard errors. For example, to be 99% confident, you would add and subtract about 3 standard errors to obtain your margin of error. This makes the margin of error larger, which is generally not a good thing. Most people don't think adding and subtracting another whole standard error is worthwhile, just to be 4% more confident (99% versus 95%) in the results obtained. The only way to be 100% sure of your results is to make your margin of error so large (by adding or subtracting many, many standard errors) that it covers every single possibility. For example, you may end up saying something like "I'm 100% sure that the percentage
of people in the population who like ice cream is 50%, plus or minus 50%." In such a case, you would be 100% sure of your results, but what would they mean? Nothing.
| HEADS UP | You can never be completely certain that your sample results do reflect the population, even with the margin of error included (unless you do something crazy like include 100% of all the possibilities as in the preceding ice-cream example). Even if you're 95% confident in your results, that actually means that if you repeat the sampling process over and over, 5% of the time, the sample won't represent the population well, simply due to chance (not because of problems with the sampling process or anything else). So, all results need to be viewed with that in mind. After all, statistics means never having to say you're certain! |
The two most important ideas regarding sample size are the following:
All these formulas work well as long as the sample size is large enough. (So, how large is "large enough"? This is covered in this section.)
Sample size and margin of error have an inverse relationship.
This section illuminates both concepts.
Almost all surveys are done on hundreds or thousands of people, and that is generally a large enough sample of people to make the theory behind the statistical formulas all work out. However, statisticians have worked out several general rules to be sure that the sample sizes are large enough.
For sample proportions, you need to be sure that
n
×
is at least 5, and
n
× (1 –
) is at least 5. For exapmle, in a preceding example of a poll on the president,
n
= 1,000,
= 0.52 and 1
−
0.52 = 0.48. So,
n
×
= 1,000 × 0.52 = 520, and
n
× (1 –
) = 1,000 × 0.48 = 480. Both of these are safely above 5, so everything is okay.
For sample averages, you need only look at the sample size itself. In general, the sample size,
n
, should be above about 30 for the statistical theory to hold. Now, if it's 29, don't panic; 30 is not a magic number, it's just a general rule.
The relationship between margin of error and sample size is simple: As the sample size increases, the margin of error decreases. This is an inverse relationship because the two move in opposite directions. If you think about it, it makes sense that the more information you have, the more accurate your results are going to get. (That, of course, assumes that the data were collected and handled properly.)
| TECHNICAL STUFF | If you're interested in the math, I explain more about this inverse relationship in |
In the preceding example of the poll involving the approval rating of the president, the results of a sample of only 1,000 people from all 288,000,000 residents in the United States could get to within 3% of what the whole population would have said, if they had all been asked. How does that work?
Using the formula for margin of error for a sample proportion, you can look at how the margin of error changes dramatically for samples of different sizes.
Suppose in the presidential approval example that
n
= 500. (Recall that
= 0.52 for this example.) Therefore the margin of error for 95% confidence is
, which is equivalent to 4.38%. When
n
= 1,000 in the same example, the margin of error (for 95% confidence) is
, which is equal to 3.10%. If
n
were increased to 1,500, the margin of error (with the same level of confidence) becomes
, or 2.53%. Finally, when
n
= 2,000, the margin of error is
, or 2.19%.
Looking at these different results, you can see that after a certain point, you have a diminished return for larger and larger sample sizes. Each time you survey one more person, the cost of your survey increases, and going from a sample size of 1,500 to a sample size of 2,000 decreases your margin of error by only 0.34% (one third of one percent!). The extra cost and trouble to get that small decrease in the margin of error may not be worthwhile. Bigger isn't always that much better!
But what may really surprise you is that bigger isn't always even a little bit better; bigger can actually be worse! I explain this in the following section.
