Woo Woo and the Scientific Method


Note: post from the old blog, with an added post-script to make the point I am trying to make with this post as clear as possible.

The other day I posted about an example of bad science, in which I explained that a common misconception that people have about scientists is that they dismiss anything that they can’t explain. The idea that everything has to have an explanation and until that explanation is found it is completely rejected is fundamentally untrue. The problem here lies in confusing the word “explanation” with the word “evidence”.

I think that an excellent example of this is this video that I saw a while ago on TheYoungTurks about a study on the effectiveness of prayer in curing AIDS.

This kind of study is one that left many non-science minded people scratching their heads. Prayer? But that’s bullshit! How can mumbling some words about someone help their sickness in any way? What a waste of time and money! Well yes, it is bullshit, but how do we know it’s bullshit? By doing a study.

Our logic tells us this doesn’t work, but what if it does? If the study were to demonstrate that people who were prayed for were statistically more likely to get better we would have evidence that prayer might work, even if we wouldn’t necessarily have an explanation as to why it works. The topic would be pursued and an explanation would be searched for, but many scientists would stop their scoffing and take a look at what might be going on. To do this, prayer would have to pass the same test that any other drug would: a double-blind random placebo-controlled study. Placebo-controlled meaning that people would not know what group they are in, therefore you control for the placebo effect. The double-blind part ensures that the researchers/doctors that collect the data for the study also do not know which patients are in which group, minimizing their own (conscious or unconsious) bias in collecting the data. Random so as to ensure that any patient has an equal chance of being placed in either group, to avoid (consciously or unconsciously) placing people that are doing slightly better in one or the other. If prayer passes this first hurdle then there is evidence that prayer might help, and the subject is pursued further. Problem: it hasn’t.

It is not a useless waste of money*. It is a valuable demonstration that prayer does not work. When scientists say that prayer doesn’t work it is because there is no scientific evidence that it works, not because they simply can’t figure out how it could therefore dismiss it offhand. Although there will always be people that refute evidence and believe whatever they want, studies like these keep logically minded people informed and hopefully further from the hands of scheming faith healers and con artists.

Don’t be afraid to demand the same evidence of “spiritual” or “supernatural” phenomena that you would anything that comes out of a lab. At the same time, don’t spit on studies like these that thoroughly debunk these phenomena as “too obvious to waste money on”. Believe it or not scientists do have a very open mind, they are open to the possibility that these things could work, the only problem is that there hasn’t been a single supernatural phenomenon to pass the first basic scientific tests. That is why scientists (most, of course) do not believe in this crap – not because they don’t want to, but because they have no good reason to.

 

*I would like to note that I understand why many would object to what money was used to pay for this study. There are limited resources for scientific research, and funneling money away from drug trials and towards this could certainly be considered a poor use of these resources. I am not trying to advocate for taxpayer or NIH money to go towards the testing of supernatural phenomena, not by any stretch of the imagination. I am rather trying to make the point that these kinds of claims can, and should be tested, in the interest of educating the public in what kind of evidence holds up to scientific scrutiny, and what to look for when deciding what to believe. I also want to emphasize the fact that these debates are not, like the alternative medicine/new agey groups would like you to believe, cold-hearted narrow minded scientists vs. open minded spiritually in touch people. It is a debate of evidence vs. a lack of evidence.

I am aware that many of you are well aware of the lack of evidence regarding supernatural claims. However, please keep in mind that many of those who have been taken in by the alternative medicine racket are not, and these kinds of studies play their own, important part in the defense of science-based medicine and logical reasoning.

Comments

  1. sonofrojblake says

    they are open to the possibility that these things could work, the only problem is that there hasn’t been a single supernatural phenomenon to pass the first basic scientific tests.

    In fairness, there’s a problem with that statement. If a supernatural phenomenon did pass basic scientific tests… it’d stop being supernatural. Arguably it used to be witchcraft/”supernatural” that chewing willow bark made your headache go away, or applying a mould poultice fixed your infection. Now you just buy aspirin or penicillin.

    Tim Minchin summed it up:

    Alternative Medicine”, I continue
    “Has either not been proved to work,
    Or been proved not to work.
    You know what they call “alternative medicine”
    That’s been proved to work?
    Medicine.”

    I recommend the entire thing:

    • thoughtsofcrys says

      I am well aware of Tim Minchin’s Storm, he also happens to be one of my favorite people of all time!
      I am not saying, by any stretch of the imagination, that supernatural anything can be proven scientifically. Supernatural, by definition, is something that is outside of nature and thus untestable. However, the second you claim that whatever supernatural entity you claim to believe in can influence the natural world, that influence on the natural world can be tested scientifically. Like prayer. If it passed a randomized double-blind placebo-controlled study, it would be an indicator that something was giving an effect, although the nature of that something would still remain elusive.

      • sonofrojblake says

        One of the interesting things about the now-defunct (I think) James Randi Million Dollar Challenge was the difficulty they often had getting potential claimants to state clearly what it was they claimed they could do. A lot of their claimants were legitimately mentally ill. Many of the rest were merely confused. It was extremely difficult for the organisation to pin people down and say, in simple terms, what their “superpower” actually amounted to, in terms that could be tested. As a result, quite a lot of the ones who actually made it to preliminary testing were dowsers, because at least that’s something you can test pretty clearly. Spoilers: they all failed, obvs, and many had excellent excuses, after the fact, explaining away why, despite having been required as a condition of the test to agree in advance that there were no adverse circumstances present. I mostly feel sorry for dowsers, as they’re not usually bilking vulnerable people out of money like snake-oil salesmen and necromancers.

Leave a Reply

Your email address will not be published. Required fields are marked *