Teaching Statistics

This provides an example of one of the three tests you can run with the information in this chapter — the paired-samples mean test. The structure follows a scientific method to emphasize the underlying structure to research. This example uses the techniques of Section 10.2.

The Research Question

Teacher training at Fisk University. Photo courtesy the Library of Congress.

The research question is a question that frames your interest in broad terms. It ends in a question mark and should be interesting to someone. For this example, the student was completing an assignment for me. The interest was as expected.

Is the teacher effective in teaching statistics?

This seems to be a settled question with a well-known answer. Thus, it may not be too interesting. However, Karen wanted to research it.

The Research Hypothesis

The research hypothesis is a proposed answer to the research question. From experience, Karen decided that the answer would be “yes, the teacher is effective at teaching statistics.” Measuring effectiveness is difficult, and there is a lot of educational research in that sub-sub-field alone. Karen just wanted to compare pre-test and post-test scores. She decided the teacher would be effective if student scores, on average, increased.

This decision is interesting because it means the researcher is making multiple measurements on the subjects. She used 30 students, but had 60 measurements. This is a clue that the measurements are not independent of each other. Students who started out with higher scores would naturally tend to end up with higher scores.

Actually, if you think about it, there really are only 30 measurements: the improvement of each student. Here, improvement is measured as the post-test score minus the pre-test score. Thus, the research hypothesis translated into symbols is

μ > 0

The population parameter is μ, the population mean of the student improvement. The (in)equality sign is “greater than” (improves). Note that this is now a one-sample means test.

Remember: The research hypothesis is what the scientist cares about… the only thing. However, because of probability and the randomness of life, statisticians need two other hypotheses: the null and the alternative. The null hypothesis here is

H₀ : μ ≤ 0

The alternative hypothesis is either the research hypothesis or its opposite. If there is an equals part to the research hypothesis, then the alternative hypothesis is the opposite of the research hypothesis. If there is no equals part, then the alternative hypothesis is the research hypothesis. Thus, for this example,

H₁ : μ > 0

Recall Table 1 in a previous mini-lecture. Again, feel free to learn that table.

Planning

Now that we have our null hypothesis and a better understanding of the processes involved in creating the data, we can explicitly write our plan, which allows others to replicate our work. This is not an ideal plan, but it is what the student did.

1. Give a pre-test on Chapter 8 to the students in the class.
2. Let the professor teach Chapter 8.
3. Give a post-test on Chapter 8 to the students in the class.
4. Record the improvements.

The second aspect of planning is planning the analysis. Here, it will be straight forward. We need to draw a conclusion about one sample: the improvement of the test scores. The correct test is the one-sample means test.

Execute the Plan

Now that we have a plan, we just need to execute it. The student collected all of the data and presented it here:

Analyze the Data

For the Excel people out there, you can download this Excel workbook. It provides the data and the analysis, giving you a chance to see the actual calculations in Excel.

Interpret the Results

And so, with the results of the analysis, we can write the following conclusion.

The null hypothesis was that there is no improvement in student understanding of statistics during the unit taught by the teacher. Because the p-value ($\approx 0.0000$) is less than our $\alpha=0.05$, we reject the null hypothesis. We conclude that students tended to improve between the pre-test and the post-test. In fact, we are 95 confident that the average improvement is between 71.07 and 78.93 points.

Note that I also included the 95% confidence interval in the conclusion. Why? That confidence interval gives us a lot of information about the magnitude of the improvement. An increase of 3 points is a lot different than an increase of 75 points, regardless of the calculated p-value.

This is very important: The p-value only tells us if we detected a difference, not how much of a difference. The confidence interval tells us how much of a difference. The two together tells a better story than either one separately.

And that is it. This example showed how to test comparisons between means of two dependent populations. Here, we were able to conclude that the (population) average pre-test score is significantly less than the (population) average post-test score. In other words, we concluded that student learning took place.