Aug 22 2005

Science and trust

My first scientific paper involved correcting an error made by others in an earlier paper published on the same topic. The error was a very simple one (a plus sign had been replaced by a minus sign) but had been buried in a complicated calculation that made it hard to detect. However, the consequences of the error were quite significant and had caused some puzzlement amongst the physicists in that subfield.

Ironically, some years later I too made a sign error in a published paper and my error was pointed out by someone else.

This kind of mistake and correction happens in science. Scientists are generally cautious and careful (otherwise they cease to be taken seriously by their peers) but are not infallible. And when they make a mistake, they are corrected by their peers, either in print or in private, and they move on. It is almost invariably assumed that the error was an honest mistake, not an attempt to cheat. Scientists trust each other.

In fact, the whole enterprise of science is based on trust and could not function otherwise. This does not mean that there are no checks in the process but those checks are not designed to catch fraud.

The process of peer review is one such measure. In this process, once the editors of a journal receive a submission, they send it out to (usually) two or more scientists who work in the same field to review the paper and recommend one of three actions to the editors – accept, reject, or make revisions.

I have had my papers reviewed by anonymous peers and have reviewed the papers of others. The point of the review is to check for clarity and completeness and proper methodology. The reviewer does not usually try to reproduce the paper’s results but instead tries to get a feel for whether the paper’s conclusions make sense and are consistent with other information. The reviewer assumes that the authors are honest, that the data given is correct, and that the calculations the authors say they made using the data have been done with due care.

So how do errors and fraud get caught? The way this usually happens is when another scientist wants to build on the previous published work and extend it or take it in a new direction. Then that scientist usually begins by trying to reproduce the results of the earlier work, and it is because of this that errors usually get detected. This is why reviewers try to make sure that all the information necessary to reproduce the results is present in a paper, even if they do not actually check the results themselves, so that future work can be built on it. (This is how the two errors that I was personally involved in got detected.) Clearly the chances of errors being detected become greater if the original work has major significance since then many people want to take advantage of that work and try to reproduce the results.

An example of this process at work occurred just this month with the important issue of global warming. While there is an emerging scientific consensus that it is occurring, there are disagreements over details. As the website What’s New reports: “One detail was records that were interpreted by a group at the U. Alabama in Huntsville as showing that the troposphere had not warmed in two decades and the tropics had cooled. However, three papers in Science this week report errors in the Alabama-Huntsville calculations. It seems that warming of the troposphere agrees with surface measurements and recent computer predictions. The group at Alabama-Huntsville concedes the error, but says the effect is not that large. That’s the way it’s supposed to work.”

If no one else cares about the work or is unaware of it, errors can remain undetected. Since trust is assumed, it is possible for an unscrupulous author to abuse that trust and to falsify and fabricate data and results and get their work published. But to remain undetected over an extended period of time usually means that the work was not considered of much use to begin with and was ignored by the scientific community.

Another way in which trust manifests itself in science is that unless there is some reason to suspect otherwise, scientists assume that whatever gets published in a journal (especially one that is peer-reviewed) is correct, even if they do not know the authors personally or even know the field. So scientists quote each other’s work freely, and often base their own papers on the work of others without knowing for sure whether that work is correct or not.

This might seem to be a risky thing to do but it is this very interconnected nature of science that keeps the system functioning. If at some point a result shows up that is plainly wrong or does not make sense, people can sometimes trace through the network of connections and find the original error that triggered the problem. Thus even errors that have remained undetected for a long time can suddenly surface because of research done in a seemingly distant area.

Given this feeling of openness and trust, it is possible to manipulate the system and get fraudulent results published. This can be for bad reasons such a deliberate fraud for personal gain (say because the authors are trying to pad their resumes or are trying for fame and hoping not to get caught). These are clearly wrong. But there are reasons for faking that, at least on the surface, may be good and these raise ethical issues that I will examine in a the next few postings.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite="" class=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>