Specifications Grading: Final(?) Thoughts

December 18, 2014

I have really enjoyed our discussions of Specifications Grading. I have learned a lot from it, and I have enjoyed the conversations (which I will continue to engage in). I particularly want to thank Theron Hitchman, Robert Talbert, and Andy Rundquist for helping me think through this. I feel like I kept asking the same questions, and everyone was very patient with me. In this post, I will post the answers I eventually came to to those questions.

My conclusion: my grading is going to become more like specifications grading, but I am not going to fully use it. I want to give my students specifications on how to do a good assignment; that is a great idea. But one of my specifications absolutely has to be “the mathematics is correct”—I cannot live with less than that.

But putting a correctness requirement in the specifications is problematic. Here is how Nilson introduces specifications (page 57, all emphasis is her’s):

For assignments and tests graded pass/fail, students have to understand the specs. They must know exactly what they need to do to make their work acceptable. Therefore, we have to take special care to be clear in laying out our expectations, and we cannot change or add to them afterward.

The problem is that the point of mathematics classes is arguably to teach the students when mathematics is correct and when it isn’t. This is obviously a huge simplification, but it would be ridiculous to expect students coming into a mathematics class to already know what is correct—it is our job to help the students learn this. As such, I think that it is not in the spirit of Nilson’s specifications grading to include a correctness specification (the same may be true of requiring that writing be clear).

Now I do not think that Nilson’s book was written on stone tablets, and Nilson herself has suggested that it may need to be modified for mathematics. I am happy to adapt specifications grading to make it work, but there is another issue: the tokens.

Viewed one way, the tokens are a way of allowing students a chance to reassess. I like that thought, but I can’t help but to view things the opposite way: tokens are a way of limiting reassessment chances. [Late edit: I think that specifications grading is a huge improvement over traditional grading, since it allows for reassessments. I just think that there are already better grading systems out there for mathematics courses. Thanks to Theron Hitchman for reminding me that I should say this.]

So we have a correctness specification that students do not understand, and they will not receive a passing grade if the work is not correct. Yet they only have limited chances to reassess due to the token system. So here is the situation I fear:

  1. A student comes to the course without knowing how to create correct mathematics.
  2. The student is given an assessment that says they are required to write correct mathematics to get a passing grade.
  3. The student, still in the process of learning, turns in an incorrect assignment and receives a failing grade on the assignment.
  4. The student uses a token to reassess; they may or may not get the mathematics correct on the reassessment because mathematics is hard. Maybe the students needs to use a second token to re-reassess.
  5. This process repeats 3–4 times until the student is out of tokens.
  6. The student never gets to reassess again, and therefore does not learn as much.

This is very similar to the reasons why Robert Talbert is considering moving from a PASS/FAIL specifications grading system to a PASS/PROGRESSING/FAIL system, where a grade of PROGRESSING is allowed to reassess without costing a token.

Here are a couple of other modifications that could avoid this:

  1. Give students a lot of feedback-only assignments prior to the graded assignments to help students learn what it means to be correct.
  2. Give students a lot of tokens so that they can get the feedback they need.

But if I give a lot of feedback-only assignments, why not give students credit it they demonstrate mastery? And if there are a lot of tokens, I think you may as well just allow an unlimited reassessments—you will probably come out ahead, time-wise, because you will not need to do the bookkeeping to keep track of the tokens (my opinion is that it is probably better to give unlimited reassessment opportunities over a PASS/PROGRESSING/FAIL system, too).

One clarification: when I say “unlimited” opportunities for reassessment, I do not literally mean “unlimited.” For one, the students will be limited by the calendar—there should not usually be reassessments once the semester ends. I am also fine limiting reassessments to class time, and not every class period needs to be for reassessment.

So I think that it is unfair to require a student’s mathematics be correct to pass an assignment, but then limit the number of reassessments. This is why I am not going to use specifications grading in my mathematics classes (I will just take some of the ideas of specifications grading and graft them onto accumulation grading).

That said, I like the general idea, and I would likely use it if and when I teach our First Year Seminar class. This is the class that Nilson mainly wrote about in the book, and I think that specifications grading could be fantastic for that class. But not for one of my mathematics classes.

Questions for you:

  1. Is there a way that we can break down the “correct” specification so that the student can know it is correct prior to handing it in? This is reasonable for computational questions (use wolframalpha!), but I don’t see how to do it any other type of question.
  2. Are there alternatives to the “lots of feedback-only assignments”/”lots of tokens”/”more than two possible grades” solutions to the issues above?

How Specs Grading Is Influencing Me

December 17, 2014

I hope I have not come off too negatively about specs grading. Reflecting on what I have written, it could seem like I am trying to discourage people from using it. I hope that is not the case. I am engaging in this conversation so much because I am very hopeful about it.

So when I say that the examples of specs given in the book are “shallow,” I do not intend this to say that specs grading is bad. Rather, what I mean (but say poorly) is that the examples of specs do not capture what I would want in a mathematics class. To put a word count requirement on a proof would be a very shallow way to grade, but I do not necessarily think that word counts are bad for other subjects (at the very least, I don’t know enough how to teach other subjects to make a judgment).

So this whole process is mainly to help me figure out how to make specifications grading work in my courses. I apologize if it sounds complainy.

So I am going to switch gears to describe the positive things I learned from the book.

  1. I should include specifications. I see no reason not to explicitly tell students what my expectations are; I just need to stop being lazy and do it.

    For instance, I collected Daily Homework in my linear algebra class last spring. It was graded only on completion, but some students did not know what to do when they got stuck or didn’t understand the question. If I had explicitly given them a set of specifications for Daily Homework that included something like, “If you cannot solve the problem, you should show me how the problem relates to \mathbb{R}^2” (we often worked in abstract vector spaces), I think that I would have been much happier with the results.

    Similarly, I gave my students templates (as Lawrence Leff does) for optimization and \delta-\epsilon proofs in calculus, but I could be doing more of that.

    The one catch is that I do not know how to specify for “quality” (thanks, Andy!). I think I have been annoying people on Google Plus trying to figure out how to solve this—sorry. But this is essential for my proofs-based courses. If I can’t figure out how to specify for quality in those courses, I will likely have to modify specs grading beyond recognition if I am going to use it in those courses.

  2. To get a higher grade in my course, I have been requiring students to master more learning goals. This is fine, but the book suggested that I could also consider having students meet the same learning goals, but have students try harder problems if they want a higher grade. Nilson’s metaphor is that the former is “more hurdles,” whereas the latter is “higher hurdles.”

    I really like this idea, and I can sort of imagine how that could work. In my non-tagging system, I could give three versions of the same problem: C-level, B-level, and A-level. For optimization in calculus, I could imagine that a C-level problem would give the function to be optimized, a B-level question wouldn’t, and an A-level would just be a trickeier version of a B-level question.

    This would require me to write more questions AND it would require me to be able to accurately judge the relative difficulty of problems. But I think that both are doable, and I like the idea.

  3. Specs grading requires that students spend tokens before being allowed to reassess. The thinking is that if reassessments are scarce, students will put forth more effort the first time. The drawback is that each assessment has higher-stakes.

    I definitely want to keep things low-stakes, but I am also finding that students aren’t working as hard as they should until the end of the semester. Using a token-like system could be a partial-solution to that.

  4. The book reminds me that I should be assigning things that are not directly related to course content; the book calls them meta-assignments. Here is a relevant quotation:

    Other fruitful activities to attach to standard assignments and tests are wrappers, also called meta-assignments, that help students develop metacognition and become self-regulated learners…Or to accompany a standard problem set, he might assign students some reflective writing on their confidence before and after solving each problem or have them do an error analysis of incorrect solutions after returning their homework (Zimmerman, Moylan, Hudesman, White, & Flugman, 2011).

    One such idea that I had to help the students start working earlier in the semester (see my previous item) is to have students develop a plan of action for the semester. Determine a study schedule, set goals for when to demonstrate learning goals, and (if they want to) determine penalities for missing those goals.

  5. I should consider including some “performance specs” (which simply measures the amount of work, not the quality of the work) in my grading. I don’t like this philosophically, but I think that it might help my students to practice more.

So even if I don’t convert to specifications grading, I have already learned a lot from it.

Specifications Grading, Revisited

December 16, 2014

The great specifications grading craze of 2014 continues, with Evelyn Lamb joining in and Robert Talbert going so far as to actually design a course using specs grading.

I have now actually read the book, so all of my misunderstandings have been updated to ‘informed misunderstandings.’ The book contained a lot of useful references to the literature on assessment, and I am planning on reading a couple of her other books soon.

I will write a second post soon about the ways the book is challenging me to improve my courses soon.

tl;dr Executive Summary

Most of the examples of specifications in the book are, in my opinion, very shallow. This makes me skeptical specifications grading is useful in a problem-solving classroom. The one example that Nilson gives from a computer science course seems to be isomorphic to accumulation grading (it seems like Leff gives 10 points for each demonstration, which is equivalent to simply counting the number of successes, as in accumulation grading, and then multiplying by 10), and seems like it is closer to my description of accumulation grading than Nilson’s description of specification grading (unless a problem template is equivalent to a set of specifications, which seems reasonable to me for some—but not all‐types of problems).

Barriers to Implementing in a Mathematics Classroom

The reason why this system is called “specifications grading” is because each assignment comes with a set of detailed specifications to guide the students in creating it. I think that this is a great idea, and I will say more about how this idea may influence my teaching in the next section.

My concern is almost all of the examples of specifications from the book are “mechanistic.” “Mechanistic” is actually Nilson’s word from page 63. She was only referring to one particular set of specs, although this set does not seem to me to be much different from the other examples. Here are all of the examples of specs from the “Setting Specs” section of Chapter 5 that I found from skimming:

  1. Do what the directions say.
  2. Be complete and provide answers to all of the questions.
  3. The assignment must contain at least n words.
  4. The assignment must be a good-faith effort.
  5. All of the problems must be set up and attempted.
  6. Focus on a couple ideas from the reading; explain how they relate to your everyday life.
  7. Briefly summarize the article.
  8. Describe in three or four sentences.
  9. Read and answer the following two questions.
  10. Read the article and summarize what you learned in five to eight sentences.
  11. Write an essay of the following length.
  12. Write an essay that is at least 1,250 words, answer the questions, include four resources (at most two can be from the internet), a personal reflection, and evidence of how the topic from the reading impacts society.
  13. Adhere to the following requirements on length, format, deadliens, and submission via turnitin.com, and also summarize the essential points of the article and “provide your reaction to those essential points, including a thorough and thoughtful assessment of the implications for doing business, particularly as related to concepts and discussions from class” (page 60).
  14. Write the specified number of pages (or words).
  15. Cite references correctly.
  16. Use recent references.
  17. Organize this literature review around this controversy (or problem, or question).
  18. The first paragraph should be about X. The second paragraph should be about Y. The paper should conclude with Z.
  19. Use the following logical conjugations to “highlight the relationships among the different works cited” (p 61).
  20. Write according to a certain length/for a certain purpose/for a certain audience.
  21. Have the following citations.
  22. Respond to the comments on the weblog.
  23. Include at least one image.
  24. Proofread.
  25. Address the task.
  26. Include 10 major concepts.
  27. It must be at least 1,200 words.
  28. The concept map must be at least four levels deep.
  29. The performance must be at least three minutes long.
  30. Research a topic and formulate a policy statement.
  31. Create a persuasive recommendation.
  32. Assess the accuracy of negative press and prepare a press release response.
  33. “Submit a 12-line biography that highlights your professional strengths while still conveying some sense of your personality” (page 63).
  34. Write 1,000 or 1,200 words.
  35. “Explain your solution (policy stance, recommendation, press release) in the first paragraph” (p 63).
  36. Make a three-point argument about why your idea is the best possible.
  37. Use at least n references, and the references must be of the following types.
  38. Write with at most n grammar/spelling/etc. errors.
  39. Spend at least four hours working on this assignment.

Nilson then writes, “Then these are the only features you look for in your students’ work and the only criteria on which you grade” (page 64). That sounds reasonable, since that is the point of specs grading. However, although Nilson at one point writes, “These critiera are not all low level” (page 61), I have to disagree. It seems to me that these examples help students to, say, write a particular type of paper; it does not seem to me that these promote any actual learning goals like critical thinking, taking other people’s perspectives, etc. I would have hoped for some specifications like, “Use the speculative method for analyzing this text”

Perhaps I am underestimating the power of simply doing the assignment properly (with respect to specs like page counts) in helping students learn—I definitely have no idea about how this would help students outside of mathematics learn. But within mathematics, I imagine that I would get a lot of proofs where the variables are properly defined, the proper symbols are used, students use “therefore/thus/etc.” correctly, but the student does not demonstrate much of any understanding of what the ideas of the proof are.

In short, I think that these specifications could be fine for, say, a humanities class (altough I do not know enough about how to effectively teach a humanities course to be sure), but I have little confidence that it would be useful in a problem solving class.

Now, Nilson did provide examples from Lawrence Leff’s and Steve Stevenson’s computer science classes. Here is a quote from page 113:

Leff uses a point system…He defines several “genres” of points in which each genre represents one of the education goals (content mastery or cognitive skills) or performance goals (amount of work)…In Leff’s area, one major performance goal is writing a minimal number of lines of code. So he defines a genre for each essential piece of content mastery or skill (e.g. bit-diddling and arrays) and another for lines of code. Each assessment is worth so many points toward meeting one or more educational goals and one or more performance goals, and he sets a minimum number of points in each genre that students must accumulate to earn a passing grade for the course. This minimum number ensures that all passing students have done an acceptable job on at least one assessment of every required educational and performance goal.

Here is my take (I will use the ‘education goals’ and ‘performance goals’ vocabulary for the next several paragraphs): if you allow for partial credit, this last bit is just traditional grading situation within a specifications grading wrapper. You get some—but not all—of the benefits of specs grading, and you might get most of the drawbacks of traditional grading. Worse yet, this is essentially traditional grading on the part of the course that I am most interesting in—the education goals.

If you do not allow for partial credit (which Leff doesn’t), then this system is isomorphic to accumulation grading. But I am not convinced that this is specifications grading, since I am not certain that actual specifications are provided. Leff does provide his students with templates for the C-level problems; B-level problems require some modification of the template; the A-level problems require independent reading (often of computer manuals) to complete, and I imagine they might deviate more from the template.

So perhaps the template is the best we can do for specifications grading for problem-solving courses. I am not sure if I like this, though, since one of my goals is usually for a student to evaluate which method to use. For example, a D-level goal I had for my calculus students was to identify problems that can be solved with an integral (they literally had to just say, “This problem can be solved with an integral” to get credit; actually, they just need to write “D8″). I do not see how a template could cover this learning goal—the template would be doing all of the work for them!

Also, I am frankly less concerned with the performance goals and, in many cases, I think that the performance goals might actually work against the education goals. For instance, there are many cases where 20 good lines of code can completely replace 100 crappy lines of code. Having such performance goals could actually discourage students from trying to find the 20 good lines. Similarly with word counts/page number requirements: my take is that it is more difficult to write a good short paper than a good long paper, yet every spec that I list above required longer papers for the higher grades.

My purpose is not to question the writing and computer science instructors’ judgment here—they definitely know more about teaching writing and computer science than I ever will. Moreover,I could solve this by reversing the specs (e.g. requiring short proofs to get the A).

But my main point is this: when it comes down to it, I just don’t think that I care a lot about performance goals. I would rather just measure the educational goals. If a student can demonstrate my education goals in a three-page paper, I don’t want to give them a grade of “fail” because she did not meet the performance goals. Worse yet, I don’t want the more conscientious students to take an excellent three-page paper, realize it does not meet my specs, and then include two pages of fluff so that it does meet my specs.

One quick comment: I fully understand that, to meet the education goals, one must put in a certain number of reps. One takeaway that I have is that I might not be supporting my students to put these reps in enough in my courses. I will definitely consider whether I should add performance goals to get students to help encourage my students to get the reps in so that they can do the education goals. But before I do this, I need to make completely sure that I am not going to be adding a bunch of busywork for many of my students.

Conclusion: My word count is already over 1700, so I have done enough for an A. So I am going to stop here and put my report on the “good” things about the textbook in a separate post.

Final questions:

  1. Am I underestimating how much students can learn by just adhering to the mechanistic specs?
  2. Am I wrong about equating Leff’s system with accumulation grading?
  3. Does a template constitute a set of specifications?
  4. How would one set up specifications for, say, a typical calculus assignment?

Specification Grading vs Accumulation Grading

December 8, 2014

Thursday, Robert Talbert and Theron Hitchman discussed the book Specifications Grading: Restoring Rigor, Motivating Students, and Saving Faculty Time by Linda Nilson on Google Plus (go watch the video of the discussion right now!)

First, I would like to say that using Google Hangouts like this is not done enough. Robert and Theron wanted to discuss the book, but live in different states. Using Skype or Google Hangouts is the obvious solution, but not enough people make the conversation public, as Robert and Theron did. I learned a lot from it, and I hope that people start doing it more (including me). Additionally, I think that two people having a conversation is about the right number. I found it more compelling than when I have watched panel-type discussions of 4–6 people on Google Hangouts.

As some of you know, I have pompously started referring to my grading system as Accumulation Grading. When Robert first introduced me to the Nilson’s book, I ordered it through Interlibrary Loan immediately. It has not arrived yet, so I probably should wait until I read it before I start comparing Specification Grading to Accumulation Grading.

But I am not going to wait. The people are interested in Specification Grading now, and so I am going to compare the two now. Just know that my knowledge of Specification Grading is based on 30 minutes of Googling and 52 minutes and 31 seconds of listening to two guys talk about it on the internet. I will read the book as soon as it arrives, but feel free to correct any misconceptions about Specification Grading that I have (there WILL be misconceptions).

Here is how to implement Specification Grading in a small, likely misconceived nutshell:

  1. Create learning goals for the course.
  2. Design assignments that give the students opportunities to demonstrate they have met the learning goals.
  3. Create detailed “specifications” on what it means to adequately do an assignment. These specifications will be given to the students to help them create the assignment.
  4. “Bundle” the assignments according to grade. That is, determine which assignments a B-level student should do, label them as such, and then communicate this to the students. This has the result that a student aiming for a B might entirely skip the A-level assignments.
  5. Grade all assignments according to the specifications. If all of the specifications are met, then the student “passes” that particular assignment. If the student fails to meet at least one of the specifications, the student fails the assignment. There is no partial credit.
  6. Give each student a number of “tokens” at the beginning of the semester that can be traded for second tries on any assignment. So if a student fails a particular assignment, the student can re-submit it for potentially full credit. You may give out extra tokens throughout the semester for students who “earn” them (according to your definition of “earn”).
  7. Give the student the highest grade such that the student passed all of the assignments for that particular grade “bundle.”

Recall that Accumulation Grading essentially counts the number of times a student has successfully demonstrated that she has achieved a learning goal (students accumulate evidence that they are proficient at the learning goals). My sense is that Accumulation Grading is a type of Specifications Grading, only with two major differences: in Accumulation Grading, the specifications are at the learning goal level, rather than the assignment level, and also the token system is replaced with a policy of giving students a lot of chances to reasses.

Let’s compare the two point-by-point (the Specification Grading ideas are in bold):

  1. Create learning goals for the course.
    This is exactly the same as in Accumulation Grading.

  2. Design assignments that give the students opportunities to demonstrate they have met the learning goals.
    This is exactly the same as in Accumulation Grading. In Accumulation Grading, this mostly takes the form of regular quizzes.

  3. Create detailed “specifications” on what it means to adequately do an assignment. These specifications will be given to the students to help them create the assignment.
    This is slightly different. In Accumulation Grading, the assignment does not matter except to give the student an opportunity to demonstrate a learning goal. So whereas Specifications Grading is focused on the assignments, Accumulation Grading is focused on the learning goals.

    To compare: in Specifications Grading, students might be assigned to write a paper on the history of calculus. One specification might be that the paper has to be at least six pages long.

    In Accumulation Grading, this would not matter— a four-page paper that legitimately meets some of the learning goals would get credit for those learning goals. If you wanted students to write a six page paper, you would create a learning goal that says, “I can write a paper that is at least six pages long.”

  4. “Bundle” the assignments according to grade. That is, determine which assignments a B-level student should do, label them as such, and then communicate this to the students. This has the result that a student aiming for a B might entirely skip the A-level assignments.

    This is technically happens in Accumulation Grading, as you can see at the end of my syllabus:

    However, something else is going on, too. The learning goals are really the things that are “bundled,” as you can see in the list of learning goals below:

    I love this flexibility. Every student (at least those who wish to pass, anyway) need to know that a derivative tells you slopes of the tangent lines and/or an instantaneous rates of change, but only student who wish to get an A needs to figure out how to do \delta-\epsilon proofs on quadratic functions.

  5. Grade all assignments according to the specifications. If all of the specifications are met, then the student “passes” that particular assignment. If the student fails to meet at least one of the specifications, the student fails the assignment. There is no partial credit.

    This is similar to Accumulation Grading, but not exactly the same. In both, there is no partial credit. The difference is that—since the main unit of Accumulation Grading is the learning goal, not the assignment—students will have multiple ‘assignments’ (really, quiz questions) that get at the same learning goal. Students can fail many of these ‘assignments’ as long as they demonstrate mastery of the learning goals eventually.

  6. Give each student a number of “tokens” at the beginning of the semester that can be traded for second tries on any assignment. So if a student fails a particular assignment, the student can re-submit it for potentially full credit. You may give out extra tokens throughout the semester for students who “earn” them (according to your definition of “earn”).

    There are no tokens in Accumulation Grading. Rather, students get many chances at demonstrating a particular learning goal.

  7. Give the student the highest grade such that the student passed all of the assignments for that particular grade “bundle.”

    This is exactly the same in both grading systems.

So the fundamental difference seems to be that Accumulation Grading focuses on how well students do at the learning goals, while Specifications Grading focuses on how well students do on the assignments. As long as the assignments are very carefully constructed and specified, I don’t really see one as being “better” than the other. However, it seems more natural to focus on learning goals rather than assignments, as the assignments are really just proxies for the learning goals; I would rather focus on the real thing than the proxy.

Another major difference is that Specification Grading uses a token system while Accumulation Grading automatically gives students many, many chances at demonstrating proficiency. One system’s advantage is the other’s disadvantage here:

  • Accumulation Grading requires creating a lot of assignments (which have mostly been quiz questions for me), whereas Specification Grading requires fewer assignments. Moreover, Accumulation Grading requires that a lot of time be spent on reassessment—either in class or out (this is probably a positive in terms of learning, but definitely a negative with respect to me having a lot of class time available for non-reassessment activities and getting home for dinner on time).
  • Accumulation Grading ideally requires some time for students to learn each learning goal between when it is introduced and when the semester ends. This is because the student needs to demonstrate proficiency multiple times (usually four times) during the semester. So either the last learning goal must be taught well before the end of the semester, or the Accumulation Grading format must be tweaked for some subset of the learning goals (you could use a traditional grading system just for the learning goals at the end of the semester). I do not think that this is an issue for Specifications Grading. On the other hand, I do not think that Specifications Grading would give the same level of confidence in a student’s grade, as it does not necessarily require multiple demonstrations of each learning goal.
  • I am concerned that the token system could hurt the professor-student relationship, whereas freely giving reassessments helps it. Specifically, I am concerned that it might seem overly arbitrary and harsh to deny a tokenless student a chance to reassess—I could see being frustrated with the professor toward the end of the term for not allowing a reassessment. On the other hand, the professor in Accumulation Grading is the hero, since she allows students as many times as possible to reassess.

That last sentence is a half-truth, since there are limitations. For instance, I only allow reassessments in class now, so that immediately limits the number of possible reassessments (my life got really crazy when I allowed out-of-class reassessments). But that seems to me to be more reasonable than the token system, since class days are not arbitrarily set by the professor, but the tokens are.

The main thing working against Accumulation Grading is that one must figure out how to reassess in a reasonable way. I have been compressing my semester to fit more quizzes in at the end of the semester, and that has worked well for me. Other people may be fine doing reassessments outside of class.

Please correct me on where I am wrong on any detail of Specifications Grading. Right now, I am still leaning toward Accumulation Grading, although I hope that Specifications Grading blows me away—I am always looking for a better system, and I will gladly switch if I find it better.

Quiz-Video Combination Instead of Lecture

November 14, 2014

Please help me understand if how much I am rationalizing here.

Here is a reminder of how I have been organizing my classes: I create learning goals for the course, and spend roughly two-thirds of the semester teaching them the content. The grading system is set up so that students have to demonstrate proficiency of each learning goal n times, where n \approx 4. The last third of the semester is spend 50/50 on quizzes and review.

I have felt a tiny bit guilty about this format for two reasons. First, I was concerned that I was depriving the students of 1/3 of the traditional instruction time. Second, I felt like a slacker because I don’t usually have to prep much for classes in the last third of the semester (also during the quizzes: I am writing this post during one of their quizzes, and I am slightly uncomfortable that they are working so hard on the class and I am not).

But I don’t feel all that bad about things now, because I realized a couple of things.

First, taking quizzes is about as active as learning gets (and maybe there are Testing Effect-type effects, especially since I purposefully spread out the learning goals on the quizzes). So students are very actively thinking about the material during the quizzes. So I am definitely giving them learning experiences, which goes a long way to alleviate my first source of guilt.

Also, I spent a lot of time creating solutions for every quiz problem. These are posted right after the quizzes so that students can get immediate feedback. This makes me feel better about my current lack of prep time—especially since I am still spending a decent amount of time writing the quizzes.

This also feels a bit better about my students’ learning experience in the last third of the semester. One of the ways I compress the material down to two-thirds of the semester is that I go lighter on the number of examples I give in the first part of the semester. However, my students probably have at least as many examples from the videos by this point in the semester than they would have gotten under a more usual course structure, and they have the added benefit of having had to attempt the problem first before viewing the solution (I am thinking about trying to make this the norm as much as possible. Ideally, things would go: try a problem on your own, try the problem with your team, see me do the problem, then try a similar problem on your own. This is a different blog post, though).

Finally, my overall impression is that the course is going well. I think that students are learning, and they are probably learning more than previous times I have taught the course.

So how much am I simply rationalizing here, and how much of my reasoning is sound?

Scheduling Large and Small

October 30, 2014

I have always known that I kind of like scheduling things. I will be department chair relatively soon, and I am looking forward to making making the teaching schedules for people. When I was a kid, I would schedule fake professional basketball and baseball seasons for fun. This is a sort of macro-scheduling, and I enjoy doing it.

What I have recently learned about myself, though, is that I hate micro-scheduling. I don’t like emailing back-and-forth to find a time that works for all people. I have embraced tools like Doodle to some extent, but I usually have to schedule one-on-one appointments, and Doodle seems like too much work for that.

I have a lot of appointments right now with my advisees to choose classes for next semester. I decided to use the Google Appointments replacement youcanbook.me, which I have written about before. This has worked ridiculously well, and it has saved me a lot of stress (Note: Neither Google nor youcanbook.me has contacted me, and I am not getting paid to write about this. I am just a simple user).

For the last several years, I have not scheduled office hours. I stopped doing this because it actually made it harder for me to meet with students. Regardless of when I schedule office hours, most of my students cannot attend. This means that I have to make individual appointments with them and still attend my regular office hours. Because I need to attend my office hours, I cannot schedule meetings during this time. So I have to schedule the meetings during times when I could have been meeting with students, which means I cannot meet with students during those times.

So I have just been scheduling “office hours” individually as students need them (which is getting to be less and less), and I hate schedule this stuff (although I like meeting with students).

So my plan is to schedule “open hours” when I plan my work day. I will use youcanbook.me, which I will post on my website and Moodle page. Students can sign up whenever they want, and I don’t need to do the scheduling, save for the random student who absolutely cannot meet with any of my preferred times. I think that this is going to greatly increase my quality of life.

Two things: Each morning, I am going to remove options to sign up for that day. This is because I don’t always re-check my calendar, and I don’t want any surprises. Also, I am going to start doing this after advising is done, since I don’t have any time between now and then anyway.

Does anyone have any experience with this? Any tips?

Three Benefits of “Accumulation Grading with Tagging”

October 15, 2014

So I decided to give my grading system a name: Accumulation Grading (or Accumulation Grading with Tagging). I just sick of writing “this grading system” or “how I am grading” all of the time.

Here are three benefits that I am seeing from this system. One has been mentioned before here (at least in the comments), one I anticipated, and one I only realized this week.

First, I suspect that there may be some sort of a metacognitive boost with this grading system. Students are forced to reflect on what they have done, and this may be helpful.

Second, grading is much easier when students use different approaches. In a very real way, I am just grading whether their “tags” are legitimate (the are correct, relevant to the problem, and point to a specific part of the solution where it is relevant). This means that students can have wildly different solutions with completely different tags, and they will both get appropriate credit. This hasn’t happened a lot yet, although I imagine it could.

Finally, my new realization is that this grading system may do away with a lot of fighting over grades. For example, a colleague recently complained that when students are asked to “graph functions” in Calculus I, many students were doing so simply by plotting points. My colleague did not want to give them credit, since he intended for them to find intervals of increasing/decreasing/concavity/etc. The students were not happy that they did not receive credit.

This is not an issue in Accumulation Grading with Tagging. Students are welcome to simply plot points to graph a question, but they run into an issue when they start to tag their work with the relevant learning goals (there are none). But nothing is marked wrong (because it isn’t wrong), so there is no real disagree to be had between student and teacher.

Update on Student-Claimed Learning Goals

October 8, 2014

I am halfway through the semester where I am using a new grading scheme for Calculus I. Here is a rough summary of the scheme:

  1. I give the students a list of learning goals. These are much finer than I have done in the past, which means that there are many more of them.
  2. I give students quizzes in class.
  3. For each quiz question, the student solves the problem as best as she can.
  4. Here is the important part: after solving the problem, the student reviews her work and determines which learning goals she has met.
  5. She indicates exactly where she met each learning goal. If she does not claim a learning goal, she does not get credit for the learning goal.

Basically, the students are forced to reflect on what they did in order to get credit for their work.

I just completed my midterm grades, and I would like to report on them. But I will first summarize where we are and describe my assessment of the course prior to seeing the grades.

We just finished off differential calculus. We will cover all of integral calculus in the next 2.5 weeks (I accelerate the schedule), and then we will move on to the review-and-quiz portion of the semester (we have quizzes for the entire class on Tuesdays and Fridays, and we review for the quizzes on Mondays and Wednesdays).

I have been simply thrilled with both sections of Calculus I. They discuss ideas, ask questions, and generally are willing to try whatever I throw at them. This has been a really fun semester. In contrast, I have heard that the other Calculus I classes have been struggling.

The good news is that my midterm grades reflect this. There are only three students who are presently in danger of getting below a C, assuming students continue on their current paces (one drawback to this grading system is that literally every student technically has an F right now, due to the fact that none of them have demonstrated any ability to work with integrals. But this is simply because they haven’t had a chance yet. But my original point for this parenthetical statement is that any student who starts slacking off is in danger of failing).

I am pleased and relieved about this. I certainly had considered that having the students claim credit for relevant learning goals could have been a disaster, but this not the case; the students have had minimal trouble with this.

One reason why they may not have had trouble is that I have been specifically referencing learning goals when they come up in class and then posting the slides to the CMS so that students can find where each learning goal is introduced. I also have been highlighting the relevant learning goals in the daily assignments (Example: “For Wednesday: We will discuss Learning Goals C4 and B9. Read 2.4 and 2.8. You should be able to do Preliminary Exercises 1 and 2 of 2.4, Exercises 63 and 65 of 2.4, and Preliminary Exercise 1 of 2.8.”).

So I am very happy and relieved at how the first half of the semester has gone. I really think that the focus on the learning goals has helped students learn how to talk about calculus. I will keep you all posted.

Mutt vs Gmail Revisited

October 2, 2014

I used to use Gmail; I thought about switching to Mutt last summer. I decided to try Mutt for the summer to see what happened.

Something surprising happened.

But first, I will briefly compare Gmail to Mutt: I like that they are both heavy with keyboard shortcuts. I find that I am get through my email really quickly with both applications.

Gmail (unsurprisingly) has a strong advantage when it comes to searching through old mail. I will eventually install something like Notmuch to make the searches faster.

Mutt has a big advantage in composing emails: vim is really awesome, and I prefer using it whenever I write any text.

Gmail has a slight advantage in convenience, since it is browser-based. However, I have an ssh app for my iPad and Chromebook, and I have a Mac at home, so Mutt is awfully easy for me to get to at home. If I am stranded someplace with only a Windows machine, I might be at a bit of a loss (I don’t know how to get the equivalent of a terminal in Windows without installing something like PuTTY), although I do have access to a website that acts as a terminal. So this is basically even.

Here is the surprise: Mutt indirectly makes me much more productive.

I was not expecting this. Here is the deal: I like staying on top of email, so I have my email open all day. But Gmail is in a browser, so checking email leads to checking Feedly…and Google Plus…and other distracting websites.

When I check my email with Mutt, I just look in the terminal, and when I am done with email, I go back to work.

This was not intentional at all. In fact, it took me a while to notice that I was spending a lot less time on the internet wasting time.

So Mutt is staying.

This has been enough of a positive over the past four months that I am going to try (and likely fail) at Cal Newport’s latest suggestion: don’t web surf during the work day.

I am a person who functions best when rules are black-and-white. I can be good at complete abstinence from things, but I am generally bad at moderation. I think this could work for me, and I am looking forward to the increase in productivity (especially since I keep to a strict work schedule).

The only things I need my browser open for are Google Calendar and Google Tasks. The latter I can take care of using a text file (I did this this summer already, and it worked fine). I can probably get by with looking at my calendar each morning and then immediately closing it. In other words, I think that I do not need to have my browser open at all during most of the work day.

This means that I will have to do all of my Feedly-checking and G+-checking (and, sadly, checking espn.com for NBA news) after my wife has gone to bed. I think this could work. But we will see.

Three Different Meanings of Mathematics

September 12, 2014

I “overheard” an exchange on social media that can be summarized like this:

Person A: I teach mathematics using an IBL-style.
Person B: I could never learn mathematics that way, even though I am good at mathematics.

I spent a lot of time thinking about this exchange, and I have found it helping me immediately in several ways. I guess this means that I might ramble a lot in this post. [Edit 20 minutes after first posting: this is probably not new to most people, and I have had similar thoughts before. But this was a bit of an epiphany for me for some reason I cannot explain.]

First, while Person A and Person B are both talking about “mathematics,” I think that they mean two different things. In fact, I think that there are (at least) three meanings for the term “mathematics” with respect to teaching.

The first meaning is what I call “application of existing mathematics.” [Edit 20 minutes after original post: mathematics that is often described as “procedural” belongs in here, although I suspect there might be more]. This comes in two flavors: the application to mathematics, and the application to outside fields. In a stereotypical “traditional” mathematics classroom, this is what is mostly meant by “mathematics.” For example: in a calculus class, finding the derivative of x^2+\sin x is an application of several existing bits of mathematics (the Power Rule, the Sum Rule, etc) to a mathematical problem to get the answer. And almost any sort of word problem fits this description.

The second meaning is what I call “understanding existing mathematics;” I think a lot of people would say this is about understanding concepts. In a Peer Instruction class (at least, in a PI class that operates in a similar way to how I do PI), this is what is mostly meant by “mathematics.” For example: in a calculus class, asking students how many tangent lines can be drawn at the point (0,0) of f(x)=|x| might be an example of that. To answer this, students need to understand the existing notion of tangent line to do this. Another example would be getting students to understand the \delta-\epsilon definition.

The third meaning is what I call “creating new mathematics,” or “doing mathematics” (when I say “new mathematics,” I mean that it is genuinely new to the student, not new to the entire community of mathematicians). I imagine that this is mostly meant by “mathematics” in a good IBL classroom. Students need to engage in the actual process of how mathematics is done by mathematicians, which includes dead ends and wrong answers (but also includes successes).

[Disclaimer: I am not trying to put a value judgment on these three meanings, although I am probably failing given that I am using the term “do mathematics” for one particular meaning. But I do happen to think that all three are extremely important. I also am probably talking in absolutes more than I should; please insert your own nuance.]

So it seems to me like that conversation actually was:

Person A: I teach students how to create new (to them) mathematics using an IBL-style.
Person B: I could never learn how to apply existing mathematics that way, even though I am good at applying existing mathematics.

I am guessing that Person A does teach students how to apply existing mathematics, but that it is secondary (or tertiary) to teaching students how to create/do mathematics.

Questions:

  1. Do any seasoned IBL instructors want to comment on the accuracy of my claims?
  2. Am I missing any other meanings?
  3. Anything else?

Follow

Get every new post delivered to your Inbox.

Join 593 other followers