“One idea people had was to check out Calibrated Peer Review. I have only scratched the surface at that site but I’m grateful for being pointed to it.”

That was a sentence from Andy Rundquist’s blog. As much as Andy ever has a throwaway line in a blog entry, this was it—this was his only mention of Calibrated Peer Review (CPR). I imagine that Andy simply put it in his weblog so he could find it later, on the off-chance that he ever thought about it again. But it changed my semester.

I have decided to use CPR in my real analysis courses this semester. Here is what CPR is in a nutshell:

  1. Students log on to CPR to get a writing assignment.
  2. Students complete the assignment and upload it to the CPR website.
  3. Students view three copies of the same assignment, all written by the instructor. These three copies are examples of differing quality.
  4. Students need to make judgements about the quality of each of the three instructor-written examples. The students answer specific questions about each article. If a student’s assessment of each of the three pieces agrees with instructor’s, the student moves to the next step. Otherwise, she must start the evaluation process again. This repeats until the student agrees with the instructor’s assessment.

    The purpose of this step is to “train” students to critically example these assignments; this is the “calibrated” part of “Calibrated Peer Review.”

  5. The student reads an anonymous article from a peer and rates it on the same criteria as the previous step. This happens a total of three times.
  6. The student evaluates his/her own article.
  7. The student sees the results from other people’s evaluation of his/her article.

By the end of this process, the student will have evaluated a total of seven different versions of the writing assignment, and will have thought about what makes a good piece of writing seven times.

I was planning on doing peer review, and I was planning on having students evaluate three different versions of the same proof. This combines the two in a nice way.

[Edit: A member of the CPR team emailed me to tell me that there is a pay version of CPR that supports a direct upload of PDF files (among other things). I don’t think that I can make it work this school year, but that would render the rest of the post irrelevant.]

[Edit: Also, here is a link to a screencast on the perhaps-unnecessary process below.]

The one catch: the CPR website only accepts text and html, which does not work well with mathematics. My workaround is this:

  1. The student writes up the solution offline in \LaTeX.
  2. The student uploads the resulting PDF to our Moodle site.
  3. The student copies the URL from the Moodle site, and simply creates a link to the Moodle site within the CPR website.

This is not the most elegant workaround, but it should work. If you have a better idea, I would love to hear it.

Tags: , , ,

6 Responses to “CPR”

  1. Andy "SuperFly" Rundquist Says:

    Hey, Bret, you just “outed” my process for keeping track of things so I don’t forget! I’m not sure how I feel about my inner workings being laid so bare for all to see . . .

    Seriously, though, how hard do you think it will be to write the three versions you have to do? Will you do that multiple times throughout the course?

    Thanks for doing some legwork for me 😉


    • bretbenesh Says:

      Hi Andy,

      Sorry! I didn’t mean to let the world know.

      I am planning on doing CPR 6-7 times throughout the course. There are seven chapters in my textbook, so I am planning on doing one for every chapter (except for maybe the first). This might be naive of me, but I don’t think that it will be too hard—I did this twice already last year for my elementary education students (one assignment was “justify why the standard addition algorithm gives the correct answer” and the other was “justify why the area of a right triangle is 1/2*a*b). I am hoping to write most of these today, so I will let you know soon if I am wrong. Bret

  2. Joss Ives Says:

    One thing you can do with CPR is that instead of writing your own instructor versions, you can use previous student assignments if you have that resource.

    And thanks for the workaround Bret. That will also help deal with the privacy laws that we have in Canada.

    • bretbenesh Says:

      Hi Joss,

      Unfortunately for me, I haven’t seen any math assignments yet. But I haven’t looked very hard, either.

      What privacy laws did my workaround take care of? I suppose Canada might not want students to give course assignments to UCLA. Bret

    • Joss Ives Says:

      We’re not allowed to use things in our courses that stores personal student information on out of country servers.

  3. Semester Reflection, Part I « Solvable by Radicals Says:

    […] also did two Calibrated Peer Review assignments. These failed due to errors on my part. First, I had students put their proofs on […]

Leave a Reply to Joss Ives Cancel reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: