The sample used in an inductive inference is relevantly different from the population as a whole. Sample size does not overcome sample bias.
Sampling is a technique used by pollsters. It is a device for gathering information about an entire population from a small subset — a sample. A representative sample is one in which whatever features in the overall population deemed relevant to the issue at hand are represented in roughly the same proportions as these features are found in the population.Johnson & Blair, Logical Self-Defense, p. 71.
[To Add: 2) Tautological Sampling 3) Tendentious Sampling]
A psychiatrist reported once that practically everybody is neurotic. Aside from the fact that such use destroys any meaning in the word “neurotic,” take a look at the man’s sample. That is, whom has the psychiatrist been observing? It turns out that he has reached this edifying conclusion from studying his patients, who are a long, long way from being a sample of the population. If a man were normal, our psychiatrist would never meet him.Darrell Huff, How to Lie with Statistics, p. 20.
When Puerto Rico was hit by a recent hurricane, there were 10,000 claims by residents for hurricane damage. The US Government decided to base its total grant aid on finding the total of claims in the first 100 applications and then multiplying by 100. A colleague was involved in the difficult task of persuading the US government that the first 100 applications need not necessarily constitute a representative sample! Small claims are likely to come in first as they need less preparation.Chatfield, Problem Solving, p. 16.
The Alfred Kinsey reports on male sexual habits are an example. Although he did not employ a sampling design, he interviewed only a small sample of Americans. Hence his results were a sample of the whole, and from them he reached some pretty striking conclusions. Kinsey found that one in ten men were homosexual, one in two had committed adultery, and one in six had been victimized by or had victimized another family sexually. Recent research into his data has discovered that his sample included prisoners and hospital patients in considerably larger numbers than their proportion of the actual population.Hoffer, The Historians’ Paradox, p. 83.
In 1936, the Literary Digest mailed out ten million ballots in a political poll to try to predict whether Franklin Roosevelt or Alfred Landon would win the upcoming election. According to the two million three hundred thousand ballots returned, it was predicted that Landon would win by a clear majority. The names for the poll were randomly selected from the telephone book, lists of the magazine’s own subscribers, and lists of automobile owners. In this case, when Roosevelt won with a 60 percent majority, it turned out that the poll was biased because it was selected from higher-income respondents (owners of cars and telephones, at that time), who tended to be Republicans. The poll was biased then, because of this association between income bracket and party preference.Walton, One-Sided Arguments, p. 225.
Show how the sample is relevantly different from the population as a whole, then show that because the sample is different, the conclusion is probably different.