Thursday, Robert Talbert and Theron Hitchman discussed the book Specifications Grading: Restoring Rigor, Motivating Students, and Saving Faculty Time by Linda Nilson on Google Plus (go watch the video of the discussion right now!)

First, I would like to say that using Google Hangouts like this is not done enough. Robert and Theron wanted to discuss the book, but live in different states. Using Skype or Google Hangouts is the obvious solution, but not enough people make the conversation public, as Robert and Theron did. I learned a lot from it, and I hope that people start doing it more (including me). Additionally, I think that two people having a conversation is about the right number. I found it more compelling than when I have watched panel-type discussions of 4–6 people on Google Hangouts.

As some of you know, I have pompously started referring to my grading system as Accumulation Grading. When Robert first introduced me to the Nilson’s book, I ordered it through Interlibrary Loan immediately. It has not arrived yet, so I probably should wait until I read it before I start comparing Specification Grading to Accumulation Grading.

But I am not going to wait. The people are interested in Specification Grading now, and so I am going to compare the two now. Just know that my knowledge of Specification Grading is based on 30 minutes of Googling and 52 minutes and 31 seconds of listening to two guys talk about it on the internet. I will read the book as soon as it arrives, but feel free to correct any misconceptions about Specification Grading that I have (there WILL be misconceptions).

Here is how to implement Specification Grading in a small, likely misconceived nutshell:

1. Create learning goals for the course.
2. Design assignments that give the students opportunities to demonstrate they have met the learning goals.
3. Create detailed “specifications” on what it means to adequately do an assignment. These specifications will be given to the students to help them create the assignment.
4. “Bundle” the assignments according to grade. That is, determine which assignments a B-level student should do, label them as such, and then communicate this to the students. This has the result that a student aiming for a B might entirely skip the A-level assignments.
5. Grade all assignments according to the specifications. If all of the specifications are met, then the student “passes” that particular assignment. If the student fails to meet at least one of the specifications, the student fails the assignment. There is no partial credit.
6. Give each student a number of “tokens” at the beginning of the semester that can be traded for second tries on any assignment. So if a student fails a particular assignment, the student can re-submit it for potentially full credit. You may give out extra tokens throughout the semester for students who “earn” them (according to your definition of “earn”).
7. Give the student the highest grade such that the student passed all of the assignments for that particular grade “bundle.”

Recall that Accumulation Grading essentially counts the number of times a student has successfully demonstrated that she has achieved a learning goal (students accumulate evidence that they are proficient at the learning goals). My sense is that Accumulation Grading is a type of Specifications Grading, only with two major differences: in Accumulation Grading, the specifications are at the learning goal level, rather than the assignment level, and also the token system is replaced with a policy of giving students a lot of chances to reasses.

Let’s compare the two point-by-point (the Specification Grading ideas are in bold):

1. Create learning goals for the course.
This is exactly the same as in Accumulation Grading.

2. Design assignments that give the students opportunities to demonstrate they have met the learning goals.
This is exactly the same as in Accumulation Grading. In Accumulation Grading, this mostly takes the form of regular quizzes.

3. Create detailed “specifications” on what it means to adequately do an assignment. These specifications will be given to the students to help them create the assignment.
This is slightly different. In Accumulation Grading, the assignment does not matter except to give the student an opportunity to demonstrate a learning goal. So whereas Specifications Grading is focused on the assignments, Accumulation Grading is focused on the learning goals.

To compare: in Specifications Grading, students might be assigned to write a paper on the history of calculus. One specification might be that the paper has to be at least six pages long.

In Accumulation Grading, this would not matter— a four-page paper that legitimately meets some of the learning goals would get credit for those learning goals. If you wanted students to write a six page paper, you would create a learning goal that says, “I can write a paper that is at least six pages long.”

4. “Bundle” the assignments according to grade. That is, determine which assignments a B-level student should do, label them as such, and then communicate this to the students. This has the result that a student aiming for a B might entirely skip the A-level assignments.

This is technically happens in Accumulation Grading, as you can see at the end of my syllabus:

However, something else is going on, too. The learning goals are really the things that are “bundled,” as you can see in the list of learning goals below:

I love this flexibility. Every student (at least those who wish to pass, anyway) need to know that a derivative tells you slopes of the tangent lines and/or an instantaneous rates of change, but only student who wish to get an A needs to figure out how to do $\delta-\epsilon$ proofs on quadratic functions.

5. Grade all assignments according to the specifications. If all of the specifications are met, then the student “passes” that particular assignment. If the student fails to meet at least one of the specifications, the student fails the assignment. There is no partial credit.

This is similar to Accumulation Grading, but not exactly the same. In both, there is no partial credit. The difference is that—since the main unit of Accumulation Grading is the learning goal, not the assignment—students will have multiple ‘assignments’ (really, quiz questions) that get at the same learning goal. Students can fail many of these ‘assignments’ as long as they demonstrate mastery of the learning goals eventually.

6. Give each student a number of “tokens” at the beginning of the semester that can be traded for second tries on any assignment. So if a student fails a particular assignment, the student can re-submit it for potentially full credit. You may give out extra tokens throughout the semester for students who “earn” them (according to your definition of “earn”).

There are no tokens in Accumulation Grading. Rather, students get many chances at demonstrating a particular learning goal.

7. Give the student the highest grade such that the student passed all of the assignments for that particular grade “bundle.”

This is exactly the same in both grading systems.

So the fundamental difference seems to be that Accumulation Grading focuses on how well students do at the learning goals, while Specifications Grading focuses on how well students do on the assignments. As long as the assignments are very carefully constructed and specified, I don’t really see one as being “better” than the other. However, it seems more natural to focus on learning goals rather than assignments, as the assignments are really just proxies for the learning goals; I would rather focus on the real thing than the proxy.

Another major difference is that Specification Grading uses a token system while Accumulation Grading automatically gives students many, many chances at demonstrating proficiency. One system’s advantage is the other’s disadvantage here:

• Accumulation Grading requires creating a lot of assignments (which have mostly been quiz questions for me), whereas Specification Grading requires fewer assignments. Moreover, Accumulation Grading requires that a lot of time be spent on reassessment—either in class or out (this is probably a positive in terms of learning, but definitely a negative with respect to me having a lot of class time available for non-reassessment activities and getting home for dinner on time).
• Accumulation Grading ideally requires some time for students to learn each learning goal between when it is introduced and when the semester ends. This is because the student needs to demonstrate proficiency multiple times (usually four times) during the semester. So either the last learning goal must be taught well before the end of the semester, or the Accumulation Grading format must be tweaked for some subset of the learning goals (you could use a traditional grading system just for the learning goals at the end of the semester). I do not think that this is an issue for Specifications Grading. On the other hand, I do not think that Specifications Grading would give the same level of confidence in a student’s grade, as it does not necessarily require multiple demonstrations of each learning goal.
• I am concerned that the token system could hurt the professor-student relationship, whereas freely giving reassessments helps it. Specifically, I am concerned that it might seem overly arbitrary and harsh to deny a tokenless student a chance to reassess—I could see being frustrated with the professor toward the end of the term for not allowing a reassessment. On the other hand, the professor in Accumulation Grading is the hero, since she allows students as many times as possible to reassess.

That last sentence is a half-truth, since there are limitations. For instance, I only allow reassessments in class now, so that immediately limits the number of possible reassessments (my life got really crazy when I allowed out-of-class reassessments). But that seems to me to be more reasonable than the token system, since class days are not arbitrarily set by the professor, but the tokens are.

The main thing working against Accumulation Grading is that one must figure out how to reassess in a reasonable way. I have been compressing my semester to fit more quizzes in at the end of the semester, and that has worked well for me. Other people may be fine doing reassessments outside of class.

Please correct me on where I am wrong on any detail of Specifications Grading. Right now, I am still leaning toward Accumulation Grading, although I hope that Specifications Grading blows me away—I am always looking for a better system, and I will gladly switch if I find it better.

2. How Specs Grading Is Influencing Me | Solvable by Radicals Says:

[…] hope I have not come off too negatively about specs grading. Reflecting on what I have written, it could seem like I am trying to discourage people […]

[…] have really enjoyed our discussions of Specifications Grading. I have learned a lot from it, and I have enjoyed the conversations (which I will continue to […]

4. Joss Ives Says:

Hey Bret. I have had these posts sitting in my queue for a while and am finally starting to read them. How do medium- or high-stakes assessments (quizzes and exams) fit into the specifications grading system?

• bretbenesh Says:

Joss,

I am not certain of the answer, but here is my best guess: you can include _any_ sort of assessment you want as long as the students know the specifications and the specifications are the only criteria used to grade the exam/quiz/whatever.

So I think that specs grading is agnostic toward higher-stakes assessments. If you wish to do exams, you just have to figured out a reasonable way to do it (Do I give the specifications prior to the exam? Do I give the specifications to the students at the same time as I give the exam question?)