Posts Tagged ‘SBF’

Students Figure Out Which Standards They Meet

April 16, 2014

I am starting to think about planning for Calculus I for next year, and there is an idea I would like to try: I want to stop labelling problems according to the corresponding standard, and put the burden on the student to determine which standards they met. I have tried this before (as have other people), but I would implement it different from how I did it last time.

So each quiz would go like this: I give them several (unlabelled) quiz problems. The students do what they can. When they are done, they submit their work. However, when they submit, we make some sort of a copy (perhaps a paper copy, perhaps just take a picture with the smart phone), and then the student takes one copy home.

At home, the student tries to figure out which standards she met on the quiz. For each standard, she writes up an argument as to why she met that standard. Specificity is key—the student would need to explicitly say where and how she met the standard. She submits this at the next class period, and this is graded as I usually do.

Here are the things I like about this idea:

  1. Students have to reflect on their work in order to get credit. This could lead to higher quality writing.
  2. Students would have to take ownership of their learning. They need to be aware of the standards they are missing, and make a concerted attempt to learn it well enough to be able to apply it on a quiz (including recognizing where it makes sense to apply it).
  3. Students can solve problems any way they like. As long as they can solve the problem using a standard, it counts. For instance, a linear algebra student might get “eigenvalue” and “determinant” credit for finding the eigenvalues of a matrix.
  4. Students are forced to really think about what the standards are and mean. There could be metacognitive benefits.
  5. I can ask more synthesis questions on quizzes; I do not need to isolate ideas for each question.
  6. Students no longer get the hint that the label provides (if the quiz question is labelled as corresponding to the “Tangent line” standard, then the student has a pretty good idea that he should find a tangent line at some point).
  7. It might give me room to have more standards (and more specific standards of the “I can do this” variety, rather than standards that are really topics, as in “Tangent lines.” David Clark encouraged me to make this transition last weekend).

Here are some potential problems:

  1. If the problems are too synthesis-y, then students won’t be able to do very many on each quiz. This might be fine, but it would be bad for a student who gets stuck and does not know where to start (on the other hand, maybe it would help teach students to start with something?).
  2. Students may try to shoehorn standards where they do not belong. This is what I would do if I were missing a small subset of standards.
  3. I am not certain I can write quiz problems that will give everyone the opportunities they need at the end of the semester. Students need different things, so I would have to have a lot of questions (note: this actually doesn’t need to be any different than how it is now; I can just provide straightforward, say, “Tangent lines” problems to quizzes if I need to. So this actually isn’t much of a problem).
  4. It forces students to be aware of what they have not yet demonstrated; this might be asking too much of some first-years.

I am on the fence about this, although I would really like to try it. Perhaps I could do both: keep the old way (with the labels) and do the new way. I could make that work.

What am I missing? What other advantages, disadvantages, and difficulties would this have?

Peter Elbow is awesome

January 12, 2013

I am busy preparing for classes, but I want to post something here so that I can find it later: Peter Elbow writes about “minimal grading,” which is essentially the wheel that I am reinventing. Enjoy the article.

(hat tip to Angela Vierling-Claassen, who tweeted the article)

Assessing with Student-Generated Videos

January 17, 2012

I regularly teach a course for future elementary education majors. The point of the class is for the students to be able to do things like explain why you “invert and multiply” when you want to divide fractions. This involves defining division (which, itself, requires two definitions—measurement division and partitive division are conceptually different), determining the answer using the definition, and justifying why the “invert and multiply” algorithm is guaranteed to give the same answer. At this stage, I simply tweak the course from semester to semester. This semester, though, I am making a major change in how I will assess the students.

Since this class is for future teachers, it makes sense to assess them teaching ideas. So there are three main ways of assessing the students this semester:

  1. The students will have two examinations. Part of each examination will be standard (a take-home portion and an in-class portion), but there will also be an oral part of the examination. The oral portion will require students to explain why portions of the standard arithmetic algorithms work the way they do.

    I only have 31 students in this class (I have two sections), so hopefully this will be doable. Moreover, I am going to distribute the in-class portion of the exams over a period of weeks: many classes will have a 5 minute quiz that will actually be a portion of the midterm.

  2. The students will regularly be presenting on the standard algorithms in class. This is only for feedback, and not for a grade. I am hoping that the audience will listen more skeptically to another student than they listen to me.
  3. The students will be creating short screencasts explaining each of the standard algorithms (Thanks to Andy Rundquist for this idea). Students will be given feedback throughout the semester on how to improve their screencasts, but they will create a final portfolio blog that contains all of their (hopefully improved) screencasts for the semester. This portfolio blog will be graded.

I will keep you posted. I welcome any ideas on how to improve this.

Semester Reflection, Part I

December 12, 2011

I am back to blogging after a semester of figuring out how to be the parent of two kids.  We are slowly figuring it out.


Anyway, below is a summary of what I did for the semester followed by how I would change in future semesters.  Recall that I am teaching real analysis.


  1. Students read a section of the text and watch some screencasts before class.  Students had to answer some questions online before class; if students did not answer the questions, they got a nagging email asking why.  This led to a very high completion rate.

  2. Students could request screencasts, thereby giving them a customized lecture (of sorts).

  3. For the first 60% of the semester, students spent about 75% of the time answering clicker questions (individually and in teams of three).  The remaining 25% of the time was spent starting homework problems.

  4. For the last 40% of the semester, we reviewed.  Students had to re-read a chapter before class.  In class, I gave the students four proofs to do in teams of two on a whiteboard.  Two proofs were very basic, and two were more complex.  I went around and gave feedback to each of the teams individually.  The idea was to run through the proofs of these four problems by the end of class (I put the proofs on slides), but we rarely got to all four questions.  I would also present the proof of a major theorem from the chapter about halfway through class.

  5.   Students were graded according to a midterm, a final, a portfolio, and a “practice portfolio.”  The exams are fairly standard.  The portfolio is a collection of each student’s best proofs throughout the semester, and the student has to provide evidence that he/she understands each of the course topics.  These are yet to be graded.  The practice portfolio was the same idea mid-way through the semester; this was graded on completion only, since the purpose of the practice portfolio was to get the students used to this different way of grading.

  6. Students who wanted to get an A for the semester had to do a project.  This means that they had to create screencasts on a section of the textbook that we had not covered during the semester (I used Abbott’s textbook, and he has them designated as “project sections”).


What went well:


  1. The clickers/peer instruction.  Analysis is full of ideas that are difficult to understand; if you do not understand them, it is even more difficult to prove anything about them.  The clickers really gave everyone—with virtually no exception—a solid idea of what was going on.

  2. The last 40% of the class was terrific.  We essentially went through the textbook twice, and the students made huuuuuuuge improvements the second time.


What I would improve next time:


  1. During the “clicker” portion of the semester, the class time spent starting the homework was not effective (in part because I did not give it enough time, but I don’t think that it would have been great with ample time, either).  I would recommend giving them the “basic” proofs that I did in the review portion of the semester each class period instead.  Perhaps do 50% clickers each class and 50% “basic proofs” (two would probably suffice, and most teams would probably only get to one).

  2. Do the practice portfolio much earlier.  I did it right after midterm, and that did not give students enough time to digest it.  Also, I recommended whether someone should do a project based on this, and students would have more time for projects if the practice portfolio went earlier.

  3. I also did two Calibrated Peer Review assignments. These failed due to errors on my part. First, I had students put their proofs on Moodle, which they linked to on the CPR site. This was a problem because students do not actually have access to the files on Moodle (it worked when I tested it because I have more permissions). Second, I told the students the wrong deadline for the second assignment. I think that this tool has a ton of potential, but I need to eliminate the user error first.
  4. I screwed up the standards a bit. For example, I was missing “Cauchy sequences” and “Limits” (Limits!). I was able to come up with fair workarounds for the students, but I think that I will only release standards to the students as we reach them in later semesters. This should force me to think through the standards an nth time, and I likely won’t miss anything major by doing this.


The jury is still out on portfolios.  We will see.


August 22, 2011

“One idea people had was to check out Calibrated Peer Review. I have only scratched the surface at that site but I’m grateful for being pointed to it.”

That was a sentence from Andy Rundquist’s blog. As much as Andy ever has a throwaway line in a blog entry, this was it—this was his only mention of Calibrated Peer Review (CPR). I imagine that Andy simply put it in his weblog so he could find it later, on the off-chance that he ever thought about it again. But it changed my semester.

I have decided to use CPR in my real analysis courses this semester. Here is what CPR is in a nutshell:

  1. Students log on to CPR to get a writing assignment.
  2. Students complete the assignment and upload it to the CPR website.
  3. Students view three copies of the same assignment, all written by the instructor. These three copies are examples of differing quality.
  4. Students need to make judgements about the quality of each of the three instructor-written examples. The students answer specific questions about each article. If a student’s assessment of each of the three pieces agrees with instructor’s, the student moves to the next step. Otherwise, she must start the evaluation process again. This repeats until the student agrees with the instructor’s assessment.

    The purpose of this step is to “train” students to critically example these assignments; this is the “calibrated” part of “Calibrated Peer Review.”

  5. The student reads an anonymous article from a peer and rates it on the same criteria as the previous step. This happens a total of three times.
  6. The student evaluates his/her own article.
  7. The student sees the results from other people’s evaluation of his/her article.

By the end of this process, the student will have evaluated a total of seven different versions of the writing assignment, and will have thought about what makes a good piece of writing seven times.

I was planning on doing peer review, and I was planning on having students evaluate three different versions of the same proof. This combines the two in a nice way.

[Edit: A member of the CPR team emailed me to tell me that there is a pay version of CPR that supports a direct upload of PDF files (among other things). I don’t think that I can make it work this school year, but that would render the rest of the post irrelevant.]

[Edit: Also, here is a link to a screencast on the perhaps-unnecessary process below.]

The one catch: the CPR website only accepts text and html, which does not work well with mathematics. My workaround is this:

  1. The student writes up the solution offline in \LaTeX.
  2. The student uploads the resulting PDF to our Moodle site.
  3. The student copies the URL from the Moodle site, and simply creates a link to the Moodle site within the CPR website.

This is not the most elegant workaround, but it should work. If you have a better idea, I would love to hear it.

SBF Grading Policy (Draft)

July 1, 2011

I previously wrote about transitioning from Standards Based Grading (SBG) to Standards Based Feedback (SBF). Here is a first pass at the policies. These will hopefully address Andy Rundquist’s question about grading.

In a nutshell, a student’s grade is determined by (roughly) the number of standards met. Slightly more detail is given in the list below, and an excerpt from the first draft of my syllabus provides even more detail below it.

I would appreciate feedback, ideas, and critiques. This is a first draft, and there must be many improvements that can be made.

  1. Homework (in the form of proofs) will be assigned regularly, but it will only be submitted for written feedback—no grades.
  2. Students will also have frequent (weekly?) opportunities for peer feedback on their proofs.
  3. Since I need to assign a grade at the end of the semester, the students will need to reflect on how their homework has demonstrated understanding of the course standards. They will assemble well-written, correct homework in a portfolio that summarizes how they met the standards for the semester.
  4. The portfolio will also contain the student’s favorite three proofs for the semester. These should be correct and well-written. Ideally, the students will also have other reasons for including them—perhaps they worked really hard on the particular proofs, found them surprising, or found them particularly interesting.
  5. Also in the portfolio will be a cover sheet cataloging the homework assignments that correspond to each standard.
  6. The portfolio will contain a self-evaluation. We will take class time in the beginning of the semester to discuss what constitutes a good proof, and the syllabus (see below) details how the portfolio will be graded. The student will have to do an honest self-evaluation of the portfolio.
  7. At midsemester, students will need to submit a trial portfolio (thanks, Joss). This will be done for credit—students either get 100% on this assignment or 0%. The purpose for this is to give students a practice run at this unusual form of grading—I don’t want their first experience with it to be high-stakes.
  8. There will also be at least one traditional graded midterm (the students will decide how many) and a final.
  9. There will be at least one ungraded, feedback-only midterm.

Syllabus Excerpt


You will be given a selection of homework problems to do each night. You are encouraged to work with other people, but you must write up your own solutions.

There are three levels to handing in homework.

  1. Once per cycle, you can hand in three proofs for me to look at; these proofs should be considered drafts, not final papers. I will give you comments on what you did well and what you need to improve upon in your next draft. I will give you only feedback on how to improve; I will not give you a grade.
  2. There will frequently be an opportunity for peer feedback of the proofs in class. Your classmates will give you feedback on the quality of your proof, and you will do the same to their proofs.
  3. At two points in the semester, you will hand in proofs to be graded. See the grading section below.

Basically, I want you to have very good proofs by the time they are assigned a grade, and I am going to help you improve your homework (without any penalty) until then.

This homework should be mostly done in \LaTeX, if only for the very practical reason that you will be re-submitting drafts; instead of re-writing each draft by hand, you will be able to simply edit a computer file. You will put more time into creating the file at the beginning, but you will save time with each draft after that.


At the end of the semester, you should have a collection of completed homework problems. At the end of the semester, you will reflect on the problems you have done, organize your homework, and submit a selection of your completed homework assignments (called your “portfolio”) for a grade. At the end of the semester, you will literally create a physical portfolio of your best work.

Here is how you will select your portfolio:

  1. You will select all bits of homework that show evidence of the Course Topics (see the section above) and place them in the portfolio. You should have multiple proofs for those labelled “Core Topics;” you only need one proof to demonstrate evidence for the “Supporting Topics.”
  2. You will select your three Favorite Proofs and put them in the portfolio. These will be well-written according to the criteria discussed in class. Also, these may be proofs that you are particularly proud of.

There is a balancing act when deciding whether a proof goes into your portfolio. On one hand, you want to provide as much evidence for the Core Topics as possible (and some evidence for the Supporting Topics). Other the other hand, an incorrect or poorly-written proof is not counted as evidence and will weaken your portfolio. Part of your goal for the semester is to learn to determine what is a good proof and what is not, and use your judgment accordingly.

Here is how your portfolio will be graded.

A: All of your Favorite Proofs are well-written, complete, and concise. Well-written, complete, concise proofs are provided for all topics; many proofs demonstrate understanding of each core topics. There are no wrong or poorly-written proofs in the portfolio.

B: All of your Favorite Proofs are well-written, complete, and concise. Many well-written, complete, concise proofs are provided for all Core Topics. Most of the Supporting Topics are supported by well-written, complete, concise proofs. There is at most one wrong proof in the portfolio.

C: All of your Favorite Proofs are well-written, complete, and concise. At least a couple of well-written, complete, concise proofs are provided for all Core Topics. Many of the Supporting Topics are supported by well-written, complete, concise proofs. There are at most two wrong proofs in the portfolio.

I will use my judgement to decide for the grades AB, BC, CD, D, and F.

Finally, you will evaluate your portfolio and determine what grade you think you deserve according to the criteria above. Be honest and be specific in your justification.

Here is how you will organize your portfolio. The first page(s) will be a cover sheet with your name, your self-assigned grade (but no discussion of it), and a list of the topics for the course. You will see that you are going to number the proofs; you should write the number of each proof that provides evidence for each topic (a single proof might provide evidence for more than one topic).

After the cover page, include your three Favorite Proofs. Start numbering these with “1.”

Next, include proofs that demonstrate each of the Core Topics for the first Core Topic in the list in the syllabus. Continue numbering these proofs as needed. If one of your Favorite Proofs provides evidence for the first Core Topic, you do not need to include a second copy of it—your cover page will indicate that the proof is evidence for both. Then, do the same with the second Core Topic. Note that if a proof from the first Core Topic also demonstrates evidence for the second Core Topic, you do not need to include a second copy of it—your cover page will indicate that the proof is evidence for both.

Continue with the other Core Topics in the same manner. Then do the same for the Supporting Topics (in the order they are listed).

Finally, include your detailed self-assessment of the portfolio; be sure to include your self-grade on this sheet, too.

Analysis Standards Draft

June 29, 2011

I spent most of the day on Monday designing the course standards for my real analysis class in the fall. I ended up with a rough draft and went home.

Later that night, I got this mysteriously clairvoyant tweet from Joss Ives: “writing learning goals is a task well-suited to collaboration.” He is right, of course, but I was astounded that—seemingly out of nowhere—he decided to tell me exactly what I needed to hear at exactly the time when I needed to hear it. He was some sort of JIT Jedi mentor.

(What actually happened was this: Joss was tweeted last week that he was writing learning goals, I replied, and he was just replying back. I like the Jedi version better, though).

Anyway, here is the first draft of the standards. I welcome feedback in the comments (what big ideas are missing, which of these aren’t so important, etc):

Core Topics: repeatedly demonstrate

Apply the Completeness Axiom
Determine convergence/divergence of sequences
Determine if a function is differentiable at a point
Apply the Mean Value Theorem to solve problems.
Use sequences when working with functions to show divergence
Determine if functions are continuous
Determine if functions are uniformly continuous
Determine if a function is integrable

Supporting Topics: demonstrate at least once

Determine limits of sequences
Determine limits of functions
Determine convergence/divergence of series
Apply Bolzano-Weierstrass
Know three equivalences for compactness on R (compact, closed bounded, finite subcover)
Determine if a sequence of functions converges pointwise; if so, determine the limit
Determine if a sequence of functions converges uniformly; if so, determine the limit
Apply Abel’s Theorem
Determine radii of convergence for power series
Apply the Fundamental Theorem of Calculus