Posts Tagged ‘Evaluation’

Grading for Probability and Statistics

January 23, 2013

Here is what I came up with for grading my probability and statistics course. First, I came up with standards my students should know:

“Interpreting” standards (these correspond to expectations for a student who will earn a C for the course.

  1. Means, Medians, and Such
  2. Standard Deviation
  3. z-scores
  4. Correlation vs. Causation and Study Types
  5. Linear Regression and Correlation
  6. Simple Probability
  7. Confidence Intervals
  8. p-values
  9. Statistical Significance

“Creating” standards (these correspond to a “B” grade):

  1. Means, Medians, and Standard Deviations
  2. Probability
  3. Probability
  4. Probability
  5. Confidence Intervals
  6. z-scores, t-scores, and p-values
  7. z-scores, t-scores, and p-values

(I repeat some standards to give them higher weight).

Finally, I have “Advanced” standards (these correspond to an “A” grade):

  1. Sign Test
  2. Chi-Square Test

Here is how the grading works: students take quizzes. Each quiz question is tied to a standard. Here are examples of some quiz questions:

(Interpreting: Means, Medians, and Such) Suppose the mean salary at a company is $50,000 with a standard deviation of $8,000, and the median salary is $42,000. Suppose everyone gets a raise of $3,000. What is the best answer to the following question: what is the new mean salary at the company?

(Interpreting: Standard Deviation) Pick four whole numbers from 1, . . . , 9 such that the standard deviation is as large as possible (you are allowed to repeat numbers).

(Creating: Means, Medians, and Standard Deviations) Find the mean, median, and standard
deviation of the data set below. It must be clear how you arrived at the answer (i.e. reading the answer off of the calculator is not sufficient). Here are the numbers: 48, 51, 37, 23, 49.

Advanced standard questions will look similar to Creating questions.

At the end of the semester, for each standard, I count how many questions the students gets completely correct in each standard. If the number is at least 3 (for Creating and Advanced) or at least 4 (for Interpreting), the student is said to have “completed” that standard (the student may opt to stop doing those quiz questions once the student has “completed” the standard).

If a student has “completed” every standard within the Interpreting standards, we say the student has “completed” the Interpreting standards. Similarly with Creating and Advanced.

Here are the grading guidelines (an “AB” is our grade that is between an A and a B):

-A student gets at least a C for a semester grade if and only if the student “completes” the Interpreting standards and gets at least a CD on the final exam.
-A student gets at least a B for the semester grade if and only if the student “completes” the Interpreting and Creating standards and gets at least a BC on the final exam.
-A student gets an A for the semester grade if and only if the student “completes” all of the standards, gets at least an AB on the final exam, and completes a project.

The project will be to do some experiment or observational study that uses a z-test, t-test, chi-square test, or sign test. It can be on any topic they want, and they can choose to collect data or use existing data. The students will have a poster presentation at my school’s Scholarship and Creativity Day.

I would appreciate any feedback that you have, although we are 1.5 weeks into the semester, so I am unlikely to incorporate it.

Advertisements

Grading Epiphany (or, Bret was completely wrong about everything)

January 8, 2013

LED Light Bulb

I am teaching probability and statistics, a course for first-year students, for the first time this spring. I have been struggling with how to grade the students.

This course is unusual in that there is only a little mathematics in the course (we throw in all of the probability that we can, but it is still mainly statistics). This requires that I think like a statistician, which is new and somewhat painful.

It also makes designing a course more difficult. I have the basics of what I want to do, but—as mentioned above—how to grade the students is the most difficult part. I want to stick with an SBG approach, but I was not sure how to set up the standards.

In calculus last semester (and other courses in previous semesters), I had a general format of: “if you can do the basic skills from the course, you will get a C. To get a grade higher than C, you must demonstrated some conceptual understanding.”

I realized as I was brushing my teeth last night that this is completely and utterly backwards.

I want my C students to understand the concepts of the course, but not necessarily be able to do the computations and symbolic manipulations. My B students should, in addition to understanding the concepts, be able to do many of the computations and symbolic manipulations. My A students should, in addition to understanding the concepts, be able to do all of the computations AND demonstrate that they can do some self-guided work.

Here is my rationale for requiring understanding the concepts to get a C:

  1. I am convinced that the concepts are easier in most college-level mathematics courses—students are better at drawing tangent lines on graphs of functions than they are at finding the equations of tangent lines.
  2. The students who need the calculation and symbolic manipulation skills are the ones who are going to continue taking more mathematics (and related) courses. I am guessing that C students are less likely to continue taking these courses.
  3. Computers can now do much of the calculation and symbolic manipulation, although the user has to understand the concepts to correctly enter the information.
  4. Most importantly, the concepts are the most important part of the course! I want to explicitly encourage students to focus on the concepts—I don’t want, say, a calculus student to be able to get an A in the course by only having good algebra skills (a colleague yesterday complained to me about such students; I view this as a flaw in the grading system).

I am not happy about having this completely backwards, and I feel bad for my previous students. But I am happy that I now understand what I want.

The tough part is designing assessments that isolate concepts. But that is part of my job, and I find it fun to come up with such questions.

How can this be improved? Am I wrong about this?

(image is “LED Light Bulb” by flickr user Wade Brooks, Creative Commons License)

Assessing with Student-Generated Videos

January 17, 2012

I regularly teach a course for future elementary education majors. The point of the class is for the students to be able to do things like explain why you “invert and multiply” when you want to divide fractions. This involves defining division (which, itself, requires two definitions—measurement division and partitive division are conceptually different), determining the answer using the definition, and justifying why the “invert and multiply” algorithm is guaranteed to give the same answer. At this stage, I simply tweak the course from semester to semester. This semester, though, I am making a major change in how I will assess the students.

Since this class is for future teachers, it makes sense to assess them teaching ideas. So there are three main ways of assessing the students this semester:

  1. The students will have two examinations. Part of each examination will be standard (a take-home portion and an in-class portion), but there will also be an oral part of the examination. The oral portion will require students to explain why portions of the standard arithmetic algorithms work the way they do.

    I only have 31 students in this class (I have two sections), so hopefully this will be doable. Moreover, I am going to distribute the in-class portion of the exams over a period of weeks: many classes will have a 5 minute quiz that will actually be a portion of the midterm.

  2. The students will regularly be presenting on the standard algorithms in class. This is only for feedback, and not for a grade. I am hoping that the audience will listen more skeptically to another student than they listen to me.
  3. The students will be creating short screencasts explaining each of the standard algorithms (Thanks to Andy Rundquist for this idea). Students will be given feedback throughout the semester on how to improve their screencasts, but they will create a final portfolio blog that contains all of their (hopefully improved) screencasts for the semester. This portfolio blog will be graded.

I will keep you posted. I welcome any ideas on how to improve this.

Gaming the Classroom

February 23, 2011

I had previously heard about Lee Shelton‘s effort to turn a classroom into a video game-type experience from several sources, although I did not know that he had created a website for it until Professor Hacker pointed me there.

Before viewing his website, I was simultaneously intrigued and concerned about his approach. I was intrigued because games clearly have a quality that gets people to devote large quantities of time and effort to them for very little outside purpose. It would be great to create an educational situation which causes students to have sustained, near-obsessional interest.

I was concerned it seems like such a set-up could easily devolve into a series of external motivators that ultimately decrease the student’s intrinsic motivation for the academic subject. Since I believe that we should be fostering an interest in learning for its own sake—not just to earn points—I was concerned about how the class was set up.

I have now looked through the site. I have only skimmed it, and I am far from an expert on gaming. Considering this, please take this next sentence with a grain of salt: I now no longer feel intrigued nor concerned; I just feel a little disappointed.

Largely from looking at the syllabus, it seems to me that the main difference in this class is that everything has a cute, gaming-type name. There are no “quizzes,” but rather “monsters to fight.” There are no “points,” but rather “XP” (“eXperience Points,” for those non-gamers out there). “Groups” of students are “guilds,” and “doing well on a midterm” is “defeating the level Boss.”

One feature I like is that it appears to have some sort of 0-1 grading scheme (you either get the XPs or you don’t), although I cannot tell if there is a mechanism for re-doing work that students attempt but do not succeed on. My opinion is that this is an essential component of a 0-1 grading system.

Largely, I feel that I am missing something. In the words of a mathematician, this seems “isomorphic” to any other classroom. Please let me know why this class structure deserves all of the buzz it is receiving—I want to go back to being intrigued and/or concerned about it.

Midterm Evaluations

March 11, 2010

I am pleased to say that I have been in the habit of offering midterm evaluations to my students for the past couple of years. I have always meant to hand them out, but I sometimes got lazy. No longer.

I have found that there are two advantages to these evaluations. First, I learn more about the class. I can learned how effective things have been, and I get a sense of how the students feel about the class. Second, the students have said they feel better about the course by my offering a chance to evaluate it. This is not surprising—everyone likes being listened to, and few people are listened to less than a college student.

One nice thing is that I can customize my evaluations to my course (as opposed to the evaluations that many schools require, which usually involves a bubble sheet and generic comments like “Bret rocks!” or “Bret sucks!”).

Here are the questions I asked this time:

  1. How helpful was the introduction of our 10 “toy” groups (S_3, D_4, quaternions, etc) to your learning? Should we have spent more time on these, less time, or did we spend the correct amount of time getting familiar with these groups.
  2. How useful is it when we go over proofs that people submitted for individual homework? How much do you learn from comparing these proofs?
  3. Has Bret provided enough support on \LaTeX for you to use it effectively?
  4. What are the benefits and drawbacks of our in-class exam format of “no surprises?” Would it be better to add a problem that you have not yet seen? Would it be better to add more “cooperative” questions? Should we leave the format the same?
  5. How useful has the feedback on the individual homework been?
  6. How could the in-class lecture time be improved? Should we be spending our in-class time differently?
  7. How effective have the cooperative groups been in helping you learn the material? Would you guess that you have learned more, less, or the same amount that you would have if you did all of the homework on your own?
  8. I am planning on following the textbook (Gallian) more closely from now on. How likely would you be to pre-read if I told you which section of the text would be covered in the next lecture?
  9. Overall, how much do you feel like you are learning in this class?
  10. What other suggestions do you have?

Here is a brief summary of student responses for these questions:

  1. Somewhere between “helpful” and “very helpful.” We spent roughly the right amount of time on them.
  2. Somewhere between “useful” and “very useful.” One student suggested that I have the students read through the proofs at home to save on class time. This was a brilliant suggestion, and I am going to change my course accordingly.
  3. Yes. Google is also very helpful.
  4. Most people liked the exam format, although some wanted more “surprise” computational questions. We will discuss this before the next midterm.
  5. The feedback has been helpful.
  6. Sloooooooooow doooooooown. I apparently go through proofs quickly. This response played a large role in my decision to start using Beamer for my classes. So far, it has been working well—a straw poll of my students suggests that we are now moving at an appropriate pace.
  7. “Very helpful” to “extremely helpful,” with perhaps five exceptions, who said that they learned an equal amount to if they had been working individually. But of those five, three said that they really did not meet much with their cooperative team. It seems like those who work with teams almost always get a lot out of it.
  8. Some said they would read ahead, some said they would. This information is embedded in my Beamer slides, so it is there for the taking.
  9. “The usual amount” to “an unbelievable amount.” No one suggested that they are not learning much.
  10. Sloooooooooow doooooooown.

One thing that I did not do this time was to ask questions that make students reflect on how they are responsible. I usually ask a question like, “What can Bret do to help you learn more?”. I follow this with “What could you be doing to help you learn more?”. I omitted it this time due to space considerations; also, I ask them a similar question every cycle (roughly a week) on their homework (so I guess I have mini-evaluations once per cycle).

Finally, I feel like I have the responsibility to report back to the class what the students said in their evaluations. This took the form of a three minute class presentation.