Oh, boy: our twice-a-year ritual, in which we hand out forms in our classes and let our students grade the faculty. And then, in another yearly ritual every fall, the faculty will gather and peer intensely at the numbers, presented with at least three significant digits, and we will see graphs and charts and over-interpreted analyses of these gnomic parameters.
Unfortunately, they probably aren’t as useful as administrators would like to imagine.
Michele Pellizzari, an economics professor at the University of Geneva in Switzerland, has a more serious claim: that course evaluations may in fact measure, and thus motivate, the opposite of good teaching.
His experiment took place with students at the Bocconi University Department of Economics in Milan, Italy. There, students are given a cognitive test on entry, which establishes their basic aptitude, and they are randomly assigned to professors.
The paper compared the student evaluations of a particular professor to another measure of teacher quality: how those students performed in a subsequent course. In other words, if I have Dr. Muccio in Microeconomics I, what’s my grade next year in Macroeconomics II?
Here’s what he found. The better the professors were, as measured by their students’ grades in later classes, the lower their ratings from students.
“If you make your students do well in their academic career, you get worse evaluations from your students,” Pellizzari said. Students, by and large, don’t enjoy learning from a taskmaster, even if it does them some good.
I also have some reservations about this study, though. What if the Macroeconomics II professor simply shares some biases with the Macroeconomics I professor, and is an easy grader? I wouldn’t want my teaching to be evaluated by how well students do in another professor’s course. That’s as scary as the arbitrary roller-coaster of student evaluations. I’ve had a few students openly downgrade me, for instance, because they know I’m an atheist, and they love Jesus so much.
But otherwise, yes, this jibes well with our general assumptions about the process: grade leniently, give light amounts of work, and students will tend to rate you highly. (They’ll also rate you highly if you’re inspiring and enthusiastic and entertaining, too, so it’s not all a drive to slackerdom).
If you must know, my student evaluations are fine — not the highest at my university, but not grounds for concern (oh, yeah, another thing about faculty assessment of these things: apparently, we’re all supposed to be above average, which simply doesn’t work). I generally ignore the numeric scores, which are mostly pointless noise, but the written comments are often actually informative and let me know what aspects of the course I should change next time I teach it.
Also, I had my students evaluate me on Monday, so I’m saying all this after they’ve had an opportunity to hack at me a bit.