
If a million students were to guess at a four answer multiple choice question, an average of 25% of them would get it correct. But if only 1,000 of them guessed at the question, the actual response could be off by 9.1% as it was in the following graph.
Another run of 1,000 students produces the following deviation from the 25%. The percentage below each column is how much the actual responses varied from 25%. For example, the third column to the right on the graph below represents answer 3) and the number below the column shows that the actual response was 2.5% lower than 25%, or 22.5%. Likewise, answer 1) was 3.1% higher or 28.1%.
A third run produces the following, with answer 4) getting only 13.4%.
The following four graphs show that increasing the number of students reduces the amount by which the responses vary from 25%. The first graph shows that answer 3) received 27.3% of the responses, whereas answer 4) received 26.9%.
Even with 10,000 students, the responses can be off by 1.1% as it was in the following graph where answer 4) received only 23.9% of the responses.
The variation from 25% tends to settle down to around +/ .3% with 70,000 students answering that question.
After a 20 runs at 100,000 students, the largest deviation from 25% observed was .6% for answer 2) below:
And after 20 runs at one million students, the largest variation from 25% was +/ 3% as observed below:
