‘Evidence-based medicine’ is a term that is now very much in vogue. It suggests that medical practitioners move away from basing their practice on traditions and folklore and instead look to the results of carefully controlled clinical studies for guidance. This is of course good advice. But the problem is determining what makes something ‘evidence-based’. After all, even anecdotes and single events can be considered as evidence since they do provide empirical data. The key question is how to determine when there is a preponderance of evidence that gives confidence that the practice being adopted is the best among all the alternatives. The gold standard consists of carefully controlled, double-blind, reproducible tests with large sample sizes but that is not always feasible and demanding that this be the measure for determining whether a conclusion is evidence-based can be used to delay the adoption of some beneficial measure. It was demands for such unreasonably high standards of evidence that enabled tobacco companies to fight for so long the medical consensus that smoking was highly harmful to human health.

In the article that I linked to yesterday where the Trump administration was attempting to derail a global consensus of encouraging breastfeeding, I saw that the Trump administration is trying to weaponize the idea of ‘evidence-based’ by narrowly defining it and demanding its adoption when ethical considerations prevent it being used. For example, take the case of the resolution encouraging the breastfeeding of infants that the Trump administration tried to scuttle.

The United States also insisted that the words “evidence-based” accompany references to long-established initiatives that promote breast-feeding, which critics described as a ploy that could be used to undermine programs that provide parents with feeding advice and support.

Elisabeth Sterken, director of the Infant Feeding Action Coalition in Canada, said four decades of research have established the importance of breast milk, which provides essential nutrients as well as hormones and antibodies that protect newborns against infectious disease.

A 2016 Lancet study found that universal breast-feeding would prevent 800,000 child deaths a year across the globe and yield $300 billion in savings from reduced health care costs and improved economic outcomes for those reared on breast milk.

Scientists are loath to carry out double-blind studies that would provide one group with breast milk and another with breast milk substitutes. “This kind of ‘evidence-based’ research would be ethically and morally unacceptable,” Ms. Sterken said.

There are things called ‘natural experiments’ where circumstances produce experimental conditions that allow for comparative studies. For example, in analyzing the health benefits of expanding Medicaid access, it would be unethical to deliberately withhold benefits to a control group just to create a particular kind of experimental protocol where other variables were kept the same or similar. Some people may be harmed because they were in the control group and thus did not receive treatment. But given that the expansion was not required of all states, some states did expand Medicaid access while others did not so one could compare health outcomes in those two populations. This experiment was not as clean as a controlled one where one could, as far as possible, match the control and experiment groups in terms of other variables. Hence this required more complicated statistical analyses since the two populations may have been quite different but it did enable the researchers to circumvent the ethical dilemma.

In my teaching career I kept experimenting with various techniques as I sought to improve my teaching. I would adopt new practices that I thought would result in better outcomes and learning experiences for students. Some of these, such as using cooperative groups, were based on many studies done by others that had demonstrated a range of cognitive and social benefits. But other practices were not as well-grounded in research but had shown signs of promise. These mostly involved the scope and sequence of the curriculum, ways of presenting the material, and practices that aimed at improving the classroom climate so that students felt comfortable engaging with others and with me. Furthermore, they had no downsides as far as I could see.

When I adopted those practices, I adopted them for all students, even in those years when I taught two parallel sections that made them ideal for experimental testing, where I could use one group as the control and the other as the experimental testing ground for the innovation. I just couldn’t bring myself to do it because it seemed to me to be unethical to withhold from the control group a practice that I believed would benefit them. I could not justify using them as guinea pigs. So my experimentation was of the less rigorous form, where I compared the performance of the current group of students with performances of students in previous years before I learned of and adopted the innovation. This is not as rigorous but deliberately withholding a benefit to one group of students just for the sake of research seemed wrong to me. I wanted every student I taught to get what I believed to be my best effort.