The power of expectations

Bit of a dissertation, but well worth it in my opinion...

I don't know how the brain chooses the things that it focuses on, but at the end of the last school year my brain chose the following three observations as those that were among the most important to my teaching:

First, my students didn't seem to be doing any real learning. They were memorizing things from practice tests, but couldn't effectively apply the concepts on real tests. I confirmed this by analyzing their results on a few tests, relative to practice tests that they had been given before hand. I broke the questions from the real tests down into three categories:

1. Questions that were the same as those on the practice test, verbatim.
2. Questions that were the same as those on the practice test, with the exception that I had changed some value which would change the answer they got when they performed the calculation (I would change 10kg to 20kg, for example).
3. Questions that covered the same concepts as those on the practice test, but didn't explicitly appear on the practice test at all.

Not surprisingly, they did very well on type-1 questions (those that were taken verbatim from their practice tests). They did worse on type-2 questions (that were the same except for using different values), and worst of all on type-3 questions (those they hadn't seen before). The surprise came when I looked at the standard deviation of the number of students who answered each type of question correctly: statistically, there was a big difference in their performance on type 1 questions: they clearly performed better on the questions that they had been given before when nothing was changed. However, there was no statistical difference in their performance on type-2 and type-3 questions! In other words, if I simply changed a number in the problem from the practice test and put it on the real test, they may as well have never seen that problem before. My conclusion: they were simply memorizing, and not learning.

Second, they didn't really care about their own results. They seemed to have only a superficial interest in what they did wrong on any given assignment, quiz, lab or test. I had a box of graded assignments in my room so that students could help themselves when they wished, as opposed to me taking valuable class time to return graded assignments. The result was an overstuffed and completely ignored box of graded assignments that I would empty on the day of a test as I walked around passing them back. They would then proceed to dump all of those papers in the trash on their way out. And why not? At that point, anything I said in the corrections was useless since they had already taken the test. To me, this indicated a real lack of connection in the minds of the students between all the things we do in class and their own success on the tests.

Third, I was doing most of the work in my classroom. I was doing all the intellectual heavy lifting - bringing the subject to my students, as opposed to making them come to the subject. I would give answers, rather than provide Socratic guidance. I would lecture, rather than inquire. I would provide a practice test rather than remind them what to study on their own. 

I wondered if these observations weren't related in some way, and so my colleague and I set out to change the way we taught the course to address all of these issues. The solution we came up with was pretty simple:

1. We'll hand the students a packet at the beginning of every unit. It will have all the material in it for that unit including homeworks, labs, worksheets and notes.
2. They will be responsible for bringing this to class every day and keeping it up to date.
3. We will not collect/grade anything. We'll give points only for on-time completion and then go over the material so the students can correct their own errors if they choose.
4. We'll let them use their packets for all quizzes and tests.
5. Since the students could use their packets on tests and quizzes, the questions on the tests and quizzes would be harder than they were last year. Instead of asking students to simply retrieve information from their packets, many questions would be designed to require the students to form their answers using two or more bits of information from the packets. In other words, there would be no type-1 questions. There would be some type-2's but mostly type-3's.

We did this to encourage personal responsibility in the students - we made it very clear by doing this that the responsibility for their success or failure lay squarely on their shoulders. But we did this also to impress upon them the value of keeping good records and referring back to them often. The business of taking their own notes, having confidence in them, and building the habit of referring back to them often would (we hoped) reinforce the connections between the various things we learn in class. But it would also help build the soft skill of developing a store of information and then actually using it later. And that skill is something badly needed by these kids. It will serve them well whatever they choose to do in life.

Early results: We assumed that the first few quizzes would be extremely difficult for the students and that we would have a fairly high failure rate as they learned to take and then rely upon good notes. We've had 3 quizzes so far and the average across all three sections have been 78%, 74% and 76%. Obviously, I would love for those grades to be higher, but since I was expecting averages in the 50's, I'm delighted at the results. Several students even have a 100% average to date.

The results are so encouraging because the quizzes are so different from last year. None of the questions that we're asking appear explicitly in the packets and many of them require the students to use more than one bit of information to develop an answer to the problem. It's early in the year yet, but so far they seem to have responded to heightened expectations with a jump in performance.