I love student evaluations. I hate student evaluations.
Every semester, at the end, I’m required to go through this rigamarole where we give students an opportunity to evaluate our teaching, by handing out a standardized form with a Likert scale for telling us how wonderful or awful we are. It’s useless. They get to color in little dots that put us on a scale of quality, and most students don’t seem to enjoy it, and I’ve also noticed that the way they score the teacher is more reflective of how well they’re doing in the class than how well they were taught. I could easily boost my score by giving out more A grades.
And, unfortunately, they’re taken way too seriously by our review committees. I’ve seen committees split hairs over a hundredth of a point, or compare faculty on the basis of sample sizes of less than 10 students. Worst of all, I’ve been in meetings where faculty seriously insist that every instructor ought to be getting above average scores on student evaluations. And you can’t speak out against them, because then they’ll get revenge by carefully scrutinizing your scores.
In their defense, though, people have argued for years that student evaluation scores are positively correlated with academic effectiveness. Only that turns out to be not necessarily true.
A new study suggests that past analyses linking student achievement to high student teaching evaluation ratings are flawed, a mere “artifact of small sample sized studies and publication bias.”
“Whereas the small sample sized studies showed large and moderate correlation, the large sample sized studies showed no or only minimal correlation between [student evaluations of teaching, or SET] ratings and learning,” reads the study, in press with Studies in Educational Evaluation. “Our up-to-date meta-analysis of all multisection studies revealed no significant correlations between [evaluation] ratings and learning.”
These findings “suggest that institutions focused on student learning and career success may want to abandon SET ratings as a measure of faculty’s teaching effectiveness,” the study says.
Oh, please, yes, make it so. Kill these things. Not only would it stop wasting our time, but it would end pointlessly innumerate conversations in faculty meetings.
But wait, I also said I love student evaluations. I do! But not the numbers. Our forms also have an open space for free-form student comments, and those are often very useful. They’re also abused (one year a group of students colluded to write the same thing on every form: “This class taught me to love Jesus even more”, because of my reputation as an atheist. I hadn’t mentioned anything, pro or con, about Christianity in the course — it was a cell biology class, but I had brought up evolution quite a bit), but they also tell me what students found memorable or problematic. That’s good to know, and I try to reduce the problems and use the memorable strategies more in subsequent classes.
Also, believe it or not, grades aren’t just a way of punishing and rewarding students. I have goals for my courses, and they also tell me if I’m getting essential concepts across. So, for instance, the first exam in my cell bio course this term was intended to evaluate whether students had a good grasp of basic general chemistry; if they didn’t, I would have to go over redox reactions yet again before I plunged into oxidative phosphorylation. There’s no point in pushing on into more complex topics if they don’t have a good grip on the basics. (I’m relieved to say they did surprisingly well on the first exam, so our general chemistry course has clearly prepared them well.)
There are better ways of assessing whether a course is accomplishing its goals than handing students who don’t see the big picture a Likert scale and asking them to state whether the course and teacher are good or not. And do I need to even go into the superficial biases that color SETs? It matters whether you are good-looking or not, and students are nests of gender biases. I know that a benefit from being male — I’m not judged on appearance as much — but suffer a bit from being older and less attractive. But those are things that shouldn’t matter at all in judging teaching effectiveness.
I was amazed and appalled once in a college promo/tenure meeting when the English prof claimed student evals were ‘far and away the best indicator of teaching effectiveness’. This was evidently because they could be calculated to three decimal places. The math prof agreed with me (I’m physics), but as we got farther from the quantitative side, into bio and the arts, agreement with his sentiments increased dramatically.
Then the statistics prof made the claim that the standard deviation we should use is of the mean, divided by sqrt(N). But this is wrong – it is the standard deviation of the FACULTY scores, not the students in the class, that should look at the range.
But any scientist knows that there ISN’T one way of measuring that is far and away better than any other – there is absolutely no way to CHECK such a statement!
With regard to students rating the instructor correlating with how they did in the course…
When I was a student at Berkeley in the late 1960s, more for fun than anything else, I took a two quarter sequence on Navigation (one quarter of terrestrial navigation and one quarter of celestial navigation) offered by the Department of Naval Science, which was actually NROTC program. But they were fully accredited university course and anyone could take them. Things got occasionally amusing because I was the only “civilian” in the classes.
On the first day, the instructor–a Naval Air Lieutenant–announced that the class would *not* be graded on a curve because everyone there (well…all the NROTC cadets) might some day be in charge of navigating a Navy ship and they were *required* to pass the course in order to be commissioned. The idea being–actually *learn* or wind up in the enlisted ranks. I really doubt he had the least concern about any adverse student evaluations.
On the other hand, the TA (who I got along with wonderfully) was a real character. He was a Navy Chief Petty Officer who was on his “retirement posting” after 30 years in the service. Wonderful guy, very laid back, and knew the required material forwards and backwards. He could probably have taken sights and done the sight reductions in his sleep.
I’m sorry to say that I’ve completely given up reading my student evaluations. Not just the numerical scores, which of course don’t mean much, but also the student comments.
Yes, there are more than 600 of them every semester to review, but I would if there were comments like PZ describes. Unfortunately, because I’m female, close to 400 or so of them will consist mostly of a sentence either praising or critiquing how I look and/or dress. It’s just too much work to wade through all that dross to get to the gold of the minority of students who have something to say about the class. So, I’ve just given up on them.
Thanks, PZ, I also really try to get redox reactions into their heads, glad to hear your chem department does also. Redox is everything! Especially in biology.
I had a psychology statistics course that was so stupefyingly cryptic that I left the student evaluation blank. I just couldn’t think of anything to say. The whole semester was like a memory blackout.
Years later I wish I had written “The only think I learned in this course was that I am wasting my time trying to get a degree”.
That at least would have been the complete truth.
And of course there’s the comments in the survey of your lecture style that complain about the way the tutorials were structured, or the way the unit was laid out, or whether there was too much or too little assessment and basically *nothing* to do with the thing they were being asked to evaluate.
Just be glad that the Shrub apparently skipped class the day SETs were handed out, or they’d’ve been made the primary determinant in whether teachers got fired and schools shut down across the country.
Thank goodness for professors like you, PZ, that actually care about whether or not the students understand the material.
I’ve noticed a trend in my campus of professors intentionally gunning for high amounts of failing grades, presumably to seem more elite and accomplished.
Like, when you flat out say that you’re aiming to fail >50% of the class, I think you’re no longer trying for the best outcome for the students and instead just aiming for that succulent ~hard class~ prestige.
It is extremely depressing to see, that high positioned faculty members of american universities can be just as stupid as american corporate managers with whom I have the ocasional displeasure to deal with (albeit indirectly, because had I have to deal with such an idiot directly, I would call them an idiot and be fired for that truth). How can anyone be this stupid and be employed at a university as anything else than a janitor? Even highschool students should know this. The hell, even highschool dropouts should have at least vague idea how average works, it is not as if that is some obscure and unknown statistical concept that is not used in news when talking about this and that.
Re: Vivee @ #7…
For a good many years, our family moved every year or two because my father was working as a field service engineer (aka tech rep) with the US military. The older of my two sisters (both older than I am), for instance, went to a different school for each year of high school. Because of the way different districts did things, when the younger of my sisters took algebra, it was a year later than the the “best” students in the district. The teacher got up the first day and said that he would not be giving out any A grades because all the *good* students had already been through the class. Then my sister won the district math competition. She–and another student–got As in the class.
…that’s ’cause you hide the tentacles.
Students love tentacles.
i laughed at the faculty insisting everyone be above average. then i thought of this George Carlin quote and laughed more.
“Think of how stupid the average person is, and realize half of them are stupider than that.”