We just started discussing confidence intervals in probability and statistics. As expected, students had a difficult time with it.

As usual, they read the section, answered some questions online, and came to class. In class, we worked on clicker questions. The first was basically:

Q: The 95% confidence interval for the population mean is [x,y]. Based on this interval:

- There is a 95% chance that is in this interval.
- 95% of the observations are in this interval.
- This method of creating intervals works 95% of the time.

This is a tricky idea, but the third choice is the best answer of the three. In my second class, only 2 out 26 students got it correct. This was to be expected, though, since it is a tricky subject.

So I basically gave a 15-20 minute lecture as to why the third one was correct and the first two were wrong. Actually, it is more accurate to say that I repeated a six minute lecture three times about how to think about this.

We had two more clicker questions related to confidence intervals, and then I gave them the following question (perhaps you recognize it):

Q: The 95% confidence interval for the population mean is [x,y]. Based on this interval:

- There is a 95% chance that is in this interval.
- 95% of the observations are in this interval.
- This method of creating intervals works 95% of the time.

The class was completely split into thirds as to which of the three answers was correct (to be fair, the question was only isomorphic to the first question, not equal).

I re-gave my two more variations of my six minute lecture explaining how to think of each of the three choices.

Then I re-gave the question, only with the following choices:

- There is a 95% chance that is in the interval.
- The probability that is in the interval is 0.95.
- 95% of the observations are in this interval.
- Exactly two of these answers are correct.
- Each of the first three answers are correct.
- None of the above answers are correct.

The correct answer is “None of the above,” of course. Three of the 26 students got it correct, even though I had literally just told them why the first three choices were wrong *two minutes* prior to voting.

This means one of two things. Either

- Either learning is incredibly complex, and lecturing is not a good tool to help people understand, or
- I suck at lecturing.

To be fair, Peer Instruction was not working, either. But it is surprising to me that Peer Instruction works as well as it does, and it is surprising to me that lectures fails as miserably as it does. The confidence interval lesson is a good reminder of the latter.

The point is not that my students are dumb—they are not. Nor is it that they are bad students—they are not. The point is that learning is difficult (especially with tricky ideas like “confidence intervals”), and one must be sensitive to this fact.