Assessment Idea for Calculus I: Feedback desperately wanted!

June 25, 2014

I am planning an overhaul of Calculus I for the fall. I used a combination of Peer Instruction and student presentations in Fall 2012, and I was not completely happy with it.

So I am starting from scratch. I am following the backwards design approach, and I feel like I am close to being done with my list of goals for the students. Here is my draft of learning goals, sorted by the letter grades they are associated with:

View this document on Scribd

I previously had lists of “topics” (essentially “Problem Types”). These lists had 10–20 items, and tended to be broad (e.g. “Limits,” “Symbolic derivatives,” “Finding and classifying extrema”). This list will give me (and, I hope, the students) more detailed feedback on what they know.

This differs from how I did things in the past, in that I used to list “learning goals” as very broad topics (so they weren’t learning goals at all, but rather “topics” or “types of problem”). Students would then need to demonstrate their ability to do these goals on label-less quizzes.

The process would be this:

1. A student does a homework problem or quiz problem.
2. The student then “tags” every instance of where she provided evidence of a learning goal.
3. The student hands in the problem.
4. The grader grades it in the following way: the grader scans for the tags. If the tags correspond to correct, relevant work AND if the tag points to the specific relevant part of the solution, the students gets credit for demonstrating that she understands that learning goal. Otherwise, no.
5. Repeat for each tag.
6. Students need to demonstrate understanding/mastery/whatever for every learning goal $n$ times throughout the semester.

Below are three examples of how this might be done on a quiz. The first example is work by an exemplary student: the student would get credit for every tag here (In all three of the examples, the blue ink represents the student work and the red ink indicates the tag).

View this document on Scribd

The second example has the same work and the same tags, but the student would not get credit due to lack of specificity; the student should have pointed out exactly where each learning goal was demonstrated.

View this document on Scribd

The third example (like the first) was tagged correctly. However, there are mistakes and omissions. In the third example, the student failed to claim credit for the “FToCI” and the “Sum/Difference Rule for Integrals.” Because of this, the student would not get credit for these two goals (even though the student did them; the point is to get students reflecting on what they did).

Additionally, the student incorrectly took the “antiderivative of the polynomial,” which caused the entire solution to the “problem of motion” to be wrong. Again, the student would not get credit for these two goals.

However, the student does correctly indicate that they know “when to use an integral,” could apply the “Constant Multiple Rule for integrals,” and “wrote in complete sentences.” The student would get credit for these three.

View this document on Scribd

I like this method over my previous method because (1) I can have finer grained standards and (2) students will not only “do,” but also reflect on what they did. I do not like this method because it is more cumbersome than other grading schemes.

My current idea (after talking a lot to my wife and Robert Campbell, and then stealing an idea from David Clark) is to require that each student show that he/she can do each learning goal six times, but up to three of them can be done on homework (so at least three have to be done on quizzes). I usually have not assigned any homework, save for the practice that students need to do to do well on the quizzes. This is a change in policy that (1) frees up some class time, (2) helps train the students on how to think about what the learning goals mean, (3) force some extra review of the material, (4) provide an additional opportunity to collaborate with other students, and (5) provide an opportunity for students to practice quiz-type problems.

My basic idea is that I will ask harder questions on the homework, but grade it more leniently (which implies that I will ask easier questions on the quizzes, but grade it more strictly).

I have been relying solely on quizzes for the past several years, so grading homework will be something that I haven’t done for a while. I initially planned on only allowing quizzes for this system, too, but it seemed like things would be overwhelming for everyone: we would likely have daily quizzes (rather than maybe twice per week); I would likely not give class time to “tag” quizzes, so students would do this at home (creating a logical nightmare); I would probably have to spend a lot more time coaching students on how to tag (whereas they now get to practice it on the homework with other people).

Let’s end this post, Rundquist-style, with some starters for you.

1. This is an awesome idea because …
2. This is a terrible idea because …
3. This is a good idea, but not worth the effort because …
4. This is not workable as it is, but it would be if you changed …
5. Homework is a terrible idea because …
6. You are missing this learning goal …
7. My name is TJ, and you are missing this process goal …

Summer Plan

June 11, 2014

My family and I agree that things work best when I work pretty strict hours—I work 7:45 am to 5 pm during the school year. I do very little work at home. However, I need to do a lot of prep work during the summer to make this possible. Because of this, I work a lot in the summer (we allow for 6 weeks of vacation for the year, so the default mode for the summer is “work”), although my hours are now 8:15 am to 5 pm.

Here is my plan for the summer:

1. Take care of all of the annoying paperwork-type-stuff that needs to be done. This includes some work that I do every summer: updating my CV, updating websites, and reading and summarizing course evaluations. I also have some jobs that are particular to this summer, such as determining which mathematics courses should be considered for transfer credit at some neighboring colleges. (I am happy that I have already done this entire item).
2. Do some reading about redesigning general education requirements. My college is considering restructuring these requirements, and my main goal for the summer is to try to determine (along with my other committee members) some sort of reasonable process for this. Fortunately, this is paid work (mostly).
3. Plan my geometry (and prob/stats/graph theory) course for elementary education majors for the fall. This is also done, largely because I taught this course in the spring. I kept detailed notes (I am grateful I did this), and I mainly updated this course by building in more feedback. In particular, I wrote all of my quizzes for the semester, created solution videos for each quiz, and updated my examinations.
4. Plan my calculus course. I am planning on using Team-Based Learning, which I learned about from Eric Mazur in this video. Again, planning includes (in chronological order) creating all learning goals, creating all assessments, and creating all class activities. When the semester comes, my main task will be briefly reviewing the plan, adapting that plan based on the students’ needs, recording what actually happened (and how I might improve things next time), meeting with students, and grading.
5. Do research. I have 3–4 papers that I need to write up, and I hope to re-start work on two projects that have been on hold for too long.

Finally, one benefit of working during the summer is you can be amazingly productive. I am often the only person here, and I can be very productive in such an environment.

Mutt revisited

June 4, 2014

This is a short story about why it is nice to blog; your comments helped me realize what I actually wanted to do. I wrote last week about how I was unhappy with Mutt. Summary: I tried to run GMail through the email program Mutt, and the result was a really slow email program.

Because of several comments by different people, I realized that

1. I have a slight concern about how Google respects my personal privacy; this is not a huge concern for me, though, and it would not be enough to make me switch.
2. I have a huge concern about my students’ privacy, and I have been concerned about GMail for a while. Your comments helped me realize that Mutt could be a solution.

Because of a conference relating to my biological children’s educations that I attended last weekend, I realized that I really want to learn more about Linux. So Mutt gives me a chance to do this.

So now I have three reasons to change, and a colleague (Michael Gass) gave me the most elegant solution: use Mutt without GMail. That is, I now use Mutt and POPMail to get mail directly from my school’s servers. Now, Mutt is as fast as it should be.

Today is the first day that I have Mutt up and running, although I have been reasonably happy with it so far. But if I run into problems, I might switch back. One potential problem is that I check my email mostly on my iPad at home, so I need to figure out something to do there (although it could be that my ssh app will work just fine with Mutt on the iPad). A possibly related problem is that getmail is not deleting the emails once they are fetched from the school’s servers (even though I have ‘delete=true’ in my .getmailrc file). This drives me crazy because I crave empty inboxes, although this may be a solution to my iPad problem; I can access Outlook via the school’s webpage or some other app, and the messages will all be there.

I suppose there is a good chance that I will change my mind again next week.

Mutt vs Gmail

May 29, 2014

I really wanted it to work. I really did.

I experimented for a couple days with using Mutt for email. I love the ability to compose emails with Vim, and I had heard that it was lightning fast. If I am honest with myself, my interest in Mutt was also related to the fact that I aspire to be a Linux geek (which I cannot claim to be right now).

So I configured Mutt to get mail from my Gmail account. That way, I could still get all of the benefits of Mutt at work while still enjoying the ease of access to Gmail everywhere else (I usually use the iPad to check email at home).

I wanted Mutt to work. I really wanted Mutt to work. But it didn’t. The problem was that it was muuuuuuuch slower than Gmail. When sending or archiving messages, Mutt took probably 5 to 10 times longer than Gmail did (Mutt takes maybe 2 seconds, whereas Gmail is near instantaneous. I recognize that 1998 Bret would be ashamed of 2014 Bret for caring about this small of a difference). Since I can mostly use Gmail without a mouse, I have come to the conclusion that Mutt is not worth it for me.

Of course, there is always the possibility that I screwed something up while configuring Mutt; let me know if you think that I did something wrong to cause this.

The Importance of Feedback

May 22, 2014

My semester is ended, and now is the time to write some post-mortem entries into this weblog. The first idea is something that is probably obvious, but I over-thought it. I have been been putting more of the course’s assessment at the end of the semester lately, thinking that that is when students are most prepared to do well.

And I am correct, but I took it too far. I did not give my students enough regular feedback during the first part of the semester this spring. My education students actually pointed this out to me—I realized that they were correct as soon as they said it (it also reinforced that they are pretty on top of education issues). Fortunately, I get to teach that course for education majors again this fall; I will make things right this time.

Additionally, I am working on ways of getting students immediate feedback. Clickers are one way of doing this, but I also might have students start grading their own quizzes (I would provide a couple of solution keys and a marker for them) and doing more computer-graded stuff.

Speeding Up Videos

May 14, 2014

I had my 20 elementary education majors produce video projects. These were all due at the end of the semester, which means I have to grade them all now. Each student produced roughly seven 4-minute videos, which means that I have roughly 10 hours of video watching to do as part of my grading.

Or do I? I downloaded this app for my Chromebook, and I have been watching the videos at twice or triple the usual speed (depending how quickly the student naturally talks), cutting my time watching videos to 3–5 hours with no loss of grading quality.

I am very grateful for this app today.

Undergraduate Reseach: Jump Before Looking

May 7, 2014

I talked about my plan for undergraduate research last week. This week, I invited my linear algebra class to join a research team I am forming.

The class is roughly half sophomores and half first-years. They have had calculus and linear algebra. My plan is to come up with a research question based around either finite fields or group actions on cyclic groups. I feel like I have some questions at the appropriate level that have come up in my own research, although I cannot explicitly state them right now. I had better be able to by next fall if any students decide to join the research team.

Draft of an Undergraduate Research Philosophy

April 30, 2014

I am working on establishing a sustainable undergraduate research program. I want to record some ideas that I have here.

First, I think that this might mean a shift toward searching for problems that undergraduates can understand the research question. This is actually pretty close to what I have been doing anyway, although I hope to more consciously seek out easily-understood problems. I had lunch last weekend with Andy Rundquist (#brag), and he told me that he changed his research focus so that it would be easier for undergraduates to work with him. Fortunately for me, he did this in part because of non-academic considerations (lasers are expensive), but I understand that his main focus was to allow students to work with him immediately after their freshman year. I still will study group theory, although I will see if there are questions that students could quickly understand after just having had one or two semesters of calculus. In particular, I might start learning something about finite fields, which I think could be accessible (it is just like the real numbers, only there are only a finite number of points!).

If I can find good questions (that is part of my goal for my sabbatical next spring), then I would like to form a research group. I hope to work with several students at once. The model I have in mind is that each student will work on solve the research question for a specific case—likely a specific family of groups. The students will be able to talk to each other, since each knows the question being asked, although not every student will know the structure of each, say, particular group.

Essentially, these students would be working on the examples that I would do myself if I were trying to solve the problem on my own. After the students complete their work, they perhaps write a thesis on the problem and I see if I can use their work to solve the entire problem.

This gives students a chance to do undergraduate research, gives them a chance to do it in a more collaborative manner (they get to work with a research team), and it gives me a chance to kill two birds with one stone—the undergraduate research is actually supporting my own research, so time spent with undergraduates is really time spent on my own research.

Do you have anything thoughts on this model? Do you have any alternative models for undergraduate research that work?

Firefox Shortcuts

April 27, 2014

I have been so happy with the GMail keyboard shortcuts that I have decided to learn similar shortcuts for Firefox.

I mainly just want to be able to quickly change tabs, move to the search bar, and move to the address bar. If I could do those three things, I will be in good shape.

[Edit 6:29 am on April 28: I am already thrilled just from knowing that control-l puts you in the address bar, control-k puts you in the search bar, control-[ goes back a page, and control-] goes forward. This is in addition to already using control-t to open a new tab. I find that control-1 through control-8 is also helpful to go to the 1st through 8th tab, although I use this less than the other commands.]

I haven’t practiced these yet, but I will starting tomorrow.

(Again, I am probably really late on this).

Students Figure Out Which Standards They Meet

April 16, 2014

I am starting to think about planning for Calculus I for next year, and there is an idea I would like to try: I want to stop labelling problems according to the corresponding standard, and put the burden on the student to determine which standards they met. I have tried this before (as have other people), but I would implement it different from how I did it last time.

So each quiz would go like this: I give them several (unlabelled) quiz problems. The students do what they can. When they are done, they submit their work. However, when they submit, we make some sort of a copy (perhaps a paper copy, perhaps just take a picture with the smart phone), and then the student takes one copy home.

At home, the student tries to figure out which standards she met on the quiz. For each standard, she writes up an argument as to why she met that standard. Specificity is key—the student would need to explicitly say where and how she met the standard. She submits this at the next class period, and this is graded as I usually do.

Here are the things I like about this idea:

1. Students have to reflect on their work in order to get credit. This could lead to higher quality writing.
2. Students would have to take ownership of their learning. They need to be aware of the standards they are missing, and make a concerted attempt to learn it well enough to be able to apply it on a quiz (including recognizing where it makes sense to apply it).
3. Students can solve problems any way they like. As long as they can solve the problem using a standard, it counts. For instance, a linear algebra student might get “eigenvalue” and “determinant” credit for finding the eigenvalues of a matrix.
4. Students are forced to really think about what the standards are and mean. There could be metacognitive benefits.
5. I can ask more synthesis questions on quizzes; I do not need to isolate ideas for each question.
6. Students no longer get the hint that the label provides (if the quiz question is labelled as corresponding to the “Tangent line” standard, then the student has a pretty good idea that he should find a tangent line at some point).
7. It might give me room to have more standards (and more specific standards of the “I can do this” variety, rather than standards that are really topics, as in “Tangent lines.” David Clark encouraged me to make this transition last weekend).

Here are some potential problems:

1. If the problems are too synthesis-y, then students won’t be able to do very many on each quiz. This might be fine, but it would be bad for a student who gets stuck and does not know where to start (on the other hand, maybe it would help teach students to start with something?).
2. Students may try to shoehorn standards where they do not belong. This is what I would do if I were missing a small subset of standards.
3. I am not certain I can write quiz problems that will give everyone the opportunities they need at the end of the semester. Students need different things, so I would have to have a lot of questions (note: this actually doesn’t need to be any different than how it is now; I can just provide straightforward, say, “Tangent lines” problems to quizzes if I need to. So this actually isn’t much of a problem).
4. It forces students to be aware of what they have not yet demonstrated; this might be asking too much of some first-years.

I am on the fence about this, although I would really like to try it. Perhaps I could do both: keep the old way (with the labels) and do the new way. I could make that work.

What am I missing? What other advantages, disadvantages, and difficulties would this have?