The Happy Scientist took a look at the test questions for Florida’s FCAT exam, used to assess whether or not fifth graders have achieved expected levels of scientific literacy for their age group, and found some problems.
I expected the Test Item Specifications to be a tremendous help in writing simulated FCAT questions. What I found was a collection of poorly written examples, multiple-choice questions where one or more of the wrong responses were actually scientifically correct answers, and definitions that ranged from misleading to totally wrong.
Click on the link to see some specific examples (the predatory cows are my favorite). But you know what’s even worse? The response he got when he pointed out the problems.
He wrote to FLDOE’s Test Development Center, and received a reply that enumerated each error and declared the material to be “deemed appropriate.” Upon pressing for a more personal reply, he received this response.
“we need to keep in mind what level of understanding 5th graders are expected to know according to the benchmarks. We cannot assume they would receive instruction beyond what the benchmark states… We cannot assume that student saw a TV show or read an article.”
This argument is being used to defend the practice of marking correct answers as wrong, if they’re not the answer the test developers picked as the correct one. I’ll grant you, students should not be expected to know more than was presented in the classroom, but if it happens that they are curious, well-read, and well-informed, that’s no reason to penalize them for being better at science than their peers—and/or, apparently, the people who wrote this test.
Absolutely terrifying.
My thought upon seeing the multi-choice question with 3 correct answers: a student who is good at learning the curriculum and trusts the test would learn through the test that certain things aren’t true, when they actually are. Way to penalize logical and critical thinking. This would then confuse the student in later years.
The best lecturer I had at uni was adamant that tests/exams were also learning exercises, and should be treated as such. Meaning that material related to but not covered in the curriculum would be seen on the exam – forcing students to think critically about the curriculum and what it means on a wider range of subjects, rather than just regurgitate facts and dot points. Although I joined in the chorus of groans whenever he said this, I wholeheartedly agreed with him.
In the example where the test glossary reads; “Germination—The process by which plants begin to grow from seed to spore or from seed to bud.”
Was probably meant to read “seed to sprout” but the author was stoned. Which is fine, but if the editor is also stoned — then it’s a problem.
from the linked article:
This smells of bureaucratic CYA, and probably (we’re talking about Florida, after all) shields some test-making corporation with mediocre staffing and excellent political connections.
Even uglier, from a comment following the linked article:
IOW, it may be that other political influences beyond cronyism in the state Dept of Ed help to sabotage the kids in our schools.
I was intrigued that these “results” are used to provide extra funding to reward “A” schools when clearly it is the “C” and “D” schools that need extra funding…..
The whole thing put me in mind of Peter Ustinov, in a BBC interview, who said that at school on a Music test one of the questions was “Name a Russian composer”. He answered Rimsky-Korsakov and was told to stop showing off – the answer was, of course, “Tchaikowsky”. Another question was “Name the greatest composer the world has ever known.” He answered “Mozart”. This too was wrong as the answer, obviously, was “Beethoven”. Plus ca change, plus c’est la meme shit!
I had a college professor try this on me. I answered a question correctly and he marked it wrong. Confused, I went to him and he told me his desired answer. I explained his answer was only mostly correct and pointed out examples that would make it wrong. He considered this and stated that since my answer was obtained outside of his lecture, it was invalid. I took this reply to the head of the biology department and unfortunately ended the man’s career at our university (I would rather he had been instructed to be a better teacher, but apparently he had words for any university that bred such a lack of respect for his knowledge and dug himself deeper).
I wish I could remember the question, but it had something to do with kin selection and his insistence that a genetic impulse to protect another could only apply to one’s own young. I remember staring blankly and uttering just one word at first: “ants?”
The whole marking the correct answer ‘wrong’ because it doesn’t fit the dumbed down version you saw in class is just another demonstration to be that education in the US isn’t about knowledge, but about training kids to be obedient robots who would give all the wrong answers to all the questions if the robot overlord told them to. How else can kids be taught to believe in supply-side economics and that corporations are people and that global warming is just a myth?