The Wikipedia problem


“Experts are scum.” That’s the amusing interpretation of Lore Sjöberg, and even better, Kieran Healy finds an entertaining example. I use Wikipedia fairly often, but it is not for anything with much depth or controversy—even fake controversy, like issues in evolution. And one serious difficulty (which is going to be endemic to any human endeavor) is that some people are jerks.

Comments

  1. says

    You need to sign up with them (it’s free) so you can go ahead and edit articles that you know have been vandalized. Better yet, make some of your grad students do it — and give them extra course credit for doing so.

  2. Caledonian says

    And some experts are idiots. I wouldn’t let Chalmers touch an article discussing his “work” with a ten-foot pole, no more than I would let a designer of perpetual-motion machines edit an article on physics.

  3. says

    Experts are not authorities. The advantages they have over the rest of us are that they’ve been exposed to advanced material in a rigorous setting, that they know which books are best, and that they’ve probably explained the same thing before. They’re not necessarily any better at conflict resolution (see academia, passim). What’s more, their time is just as limited as any amateur, if not more so.

    The Wikipedia editing process has problems, or perhaps “lifestyle issues”, which apply equally to almost all editors regardless of expertise. Remember, even if you’re a tenured professor in field X, you don’t have an infinite amount of time, so you will make the easiest contributions which strike your fancy. The majority of edits will by nature be minor ones, affecting and being affected by only their immediate surroundings. Typically, a small group of editors works very hard to improve an article in which they have an interest to a local maximum. After they move on, other people come along, adding bits of this or that — everyone mentioning their favorite Calvin and Hobbes strip, say, or tossing in the latest breathless pop-science article about the String Wars. These edits, each one of which is made in good faith, collectively bury the article’s organization under geological strata of cruft. Then someone notices the problem and puts in the effort to clean it up, and the cycle continues. Overall, the motion may be forward, but it can sure be a Brownian path getting there.

    (This problem might not matter so much if the intent were to produce collaborative course notes or co-author a textbook. In that case, each page might be intensively “workshopped” for a month, and then the whole thing is exported to a PDF. Instead of continual revision, with its inherent hazard of continual degradation, the system would operate more like a process of successive editions.)

    No matter how smart and well-read the contributors are, the organization of material will suffer. Even if an article has all the relevant facts, they won’t be arranged in a pedagogical way, and the relationships between articles will be even worse. Imagine what it would take to enforce a consistent notation across all of Wikipedia’s math pages!

    I’m going out on a limb here, but I predict that Larry Sanger’s “Citizendium” will not solve this problem. Not unless a professor will get tenure for working on it.

    A better approach might be the following: set up your own server, install MediaWiki and fork all the articles in a given subject area. The GFDL allows you to do this as long as you meet certain attribution requirements. Then you set a deadline: “In one year, we will have a compendium of biology (or physics, etc.) useful for the general public as well as graduate students which we will be proud to place on a DVD.” You look at university curricula, find all the topics which must be covered, and fill in the gaps where they exist.

    Without a “ticking clock” like this, and without the promise of enshrining one’s words in print, I don’t think humdrum but essential introductory concepts can get the coverage they need. Who’s going to write the page on kinetic energy, say, or the parts of a eukaryotic cell?

    (For a while, it looked like I actually had a chance at putting this plan in motion, but then other tasks like “earning a living” came to the fore. I still have some server space, if you’d like to try. . . .)

  4. Chris says

    Experts are not scum. But there are a lot of fake experts.

    If a real expert shows up armed with citations from verifiable sources that are not himself, and sets out to seriously prove his point, he would very likely succeed.

    But simply saying “I’m an expert so believe what I say” – with or without namedropping – doesn’t fly on WP. And shouldn’t.

  5. says

    Oh, and in the “some people are jerks” category, the most amusing example might be the odd case of Christopher Michael Langan. He’s a fellow of the ISCID and inventor of the “Cognitive-Theoretic Model of the Universe”.

    (You might remember him: he’s that guy from TV. You know, the bar bouncer with the genius IQ, like the real-life version of Good Will Hunting? Yeah, that guy.) He contributed an essay on “causality and teleologic evolution” to Uncommon Dissent: Intellectuals who find Darwinism Unconvincing, and the “CTMU” was published in — wait for it — Progress in Complexity, Information and Design (September 2002), the world-renowned journal of ISCID itself.

    Insofar as mere mortals can comprehend, the CTMU resembles poorly digested Platonism; by beginning with “logical tautologies” — or what he calls tautologies — Langan proceeds to derive the whole structure of the Cosmos, subsuming Hubble and Goedel alike in a mishmash of pseudophysics and pseudophilosophy. It reads like a Sokal parody of cranky mutterings — fractured ceramics, if you will — really a barrel of laughs, and a fine illustration of Mark Chu-Carroll’s maxim that the worst math is no math at all.

    The CTMU sank without a ripple, of course. What else would you expect from a “theory” even less scientific than specified complexity? (It got mentioned in a bit of the press coverage which Langan received when he was the real-world Will Hunting, but never in a more respectable journal than Popular Science — and never in detail.)

    A Wikipedia article was created on the CTMU, which soon enough ended up filed under “Bad Jokes and Other Deleted Nonsense”. The text was as follows:

    The CTMU is the “Cognitive Theoretic Model of the Universe”. It is also known as the theory of everything or unified field theory. In mathematical terms, it is meant to bring the reader into isomorphism with the universe. Not only does the CTMU explain the universe, but also explains itself as it too is part of the universe.

    The uniqueness of the CTMU amongst other theories of the sort is that it postulates a solution to what are termed “false dichotomies” or paradoxes. In so describing such paradoxes it covers the scientifically vexing determinancy/indeterminancy paradox, which is common in things like wave-particle duality and quantum physics. It implies that the determinancy and indeterminancy in systems is no paradox at all, but together compliment one of the CTMU’s most important causal/acausal principals: Self-determinancy.

    Amongst the self-determinancy idea there are new and equally important constructs of self-containment (the universe is self-contained), self-reference (the universe references itself), and other examples sufficient to make the argument clear and to a point.

    Since the CTMU cannot be proven under the traditional scientific method (involving an observer and that which is observed because in the CTMU observer = observed) it proposes to prove itself. Hence the CTMU can be thought of as supertautological; an axiom of choice; or self-proving.

    The CTMU defines the universe as the set of all sets.

    The universe can also be thought of as the “set of all information”. The CTMU brings together all the fields of science in the sense that it blurs all the lines between all the fields to a point where they become indistinguishable. Hence the reason it can be called the “unified field theory”.

    Some time later, a new article on the CTMU was created (with a hyphen in the title, oh my gosh). It was also nominated for deletion. An acrimonious and time-wasting debate followed, marked by a considerable amount of abusive language, trolling and sock puppetry. One-shot accounts rose out of the quantum foam to say, in essence, “Keep this article because Langan is a genius!”

    The article was deleted.

    Most participants in the debate formed the opinion that the two most obnoxious individuals arguing for the article’s retention were Langan himself and his wife. If they weren’t, so the logic went, they must have been his #1 and #2 fans from across the street (based on IP-address geolocation).

    The dispute continued in fits and spurts through late summer and into autumn, until in a fit of pique, the user believed to be Christopher Michael Langan took his case to the Arbitration Committee, which is a judicial body formed by the authority of Jimbo Wales. Unfortunately, the ArbCom ruling he had hoped for went instead against him. The proletariat had risen and deposed the philosopher-king.

    This took quite a few months to die, and it wasted more person-hours than pointless debate should. Having witnessed a few affairs of this sort play themselves out, I’m sympathetic to concerns that “real intellectuals should be given more weight”, but I’m also pretty sure that such fixes only address one side of the problem. It may be the most colorful side — it’s the one which produces entertaining stories about Langan, the Bogdanov brothers and their ilk — but it’s not the only thing which must be addressed if we want useful, encyclopedic prose.

  6. dzd says

    The problem with Wikipedia in a nutshell is that we are now in a situation where the reputation of the site is such that any wrongheaded bullshit is lent authority by its presence, rather than the quality of the articles improving the reputation of the site.

    Well, that and the users who appear to treat it as some kind of MMORPG, only with numbers of edits replacing experience points.

  7. says

    It’s worth pointing out that Lore Sjöberg’s referenced article was satire.  It generated so much feedback that he gave his more serious opinions on Wikipedia (and the response to his piece) on his blog here.

    [comment cross-posted at Crooked Timber]

  8. says

    Wikipedia is an awesome resource, and like any sort of reference material its users need to know how to use it. I was recently dismayed at its article on the Vietnam War which seems to be edited by the Swiftboat Vetrans for Truth. On the other hand many of the articles are excellent, and where else are you going to find an encyclopedia that’s not only free, but free from advertising?

  9. says

    Wikipedia is like the local school board: Gotta volunteer to be on it or it’ll get overrun by Fundies and other well-organized nutballs trained in how to game it.

  10. Kim says

    Wikipedia is mediocre. Some stuff is good, but it is interspersed with nonsense. Just look up Drosophila, it says that Drosophila (Musca) funebris is the type species. Yeah right. I am not correcting any more, it is not worth the time because before you know, you have to spend hours of teaching just to get some basic stuff around to them. I am waiting till the subgenus Musca shows up on more serious websites…..

  11. TAW says

    Wikipedia rocks. There was a study that found wikipedia to be very close in accuracy to encyclopedia britanica (in science anyway) http://www.nature.com/news/2005/051212/full/438900a.html . I also love the very concept of it. It’s hundreds of people volunteering their free time to accumulate knowledge in a single, easy to use source that includes external links and sources. It’s really amazing how accurate it really is. Yes there’s vandalism and wrong information, but it’s usually fixed pretty soon… not to mention that most vandalism is blatantly obvious. The amount of information in one place as well as the links/sources and the amount of CORRECT information more than makes up for the imperfections.

    as for the link/example, I have to point out that it was only ONE person who was arguing against the edits. You can’t just take one person and say that wikipedia as a whole things experts are scum.

  12. says

    . Just look up Drosophila, it says that Drosophila (Musca) funebris is the type species. Yeah right. I am not correcting any more, it is not worth the time because before you know, you have to spend hours of teaching just to get some basic stuff around to them. I am waiting till the subgenus Musca shows up on more serious websites…..

    That’s not a very good example of the problems of Wikipedia. *Most* systematists would say that the type species is funebris and that’s what standard works like Yeates and Wiegmann’s “The Evolutionary Biology of Flies” (2005) say it is. Yes, there is current debate as to whether Drosophila is monophyletic, but can you really fault people for wanting the article to follow standard references and not the personal preferences of whatever systematist is editing the article?

  13. Dustin Locke says

    Michael Kremer,

    I couldn’t agree more. This isn’t the first time Caledonian has pretended to be the expert on consciousness that he/she claims Chalmers isn’t:

    http://scienceblogs.com/pharyngula/2006/09/cool_a_new_argument_for_dualis.php#comment-211916

    http://scienceblogs.com/pharyngula/2006/09/cool_a_new_argument_for_dualis.php#comment-213014

    One day, I believe, Caledonian which actually take the time to explain WHY he/she thinks Chalmers is such a fool. Until then, we’ll just have to draw our own conclusions about who the fool is.

  14. says

    The point of wikipedia _isn’t_ to be a reliable, definitive source, it’s merely to point people vaguely in the sort-of correct direction, to week out the complete bullshit (leaving only that which is good fertilizer), and to gradually get people involved in their own continuing educations and the intellectual life of the great emerging Noosphere that is the Inernet. Or something like that, all I know is that I take sadistic glee in editing screwed up articles and forcing the idiots that fucked them up to admit that _I’M_ right.

  15. says

    This is a very interesting discussion. I am moved to write a post on my blog on how Wikipedia should or should not fit into “research” (look for it about 6:00 AM Central Time).

    I think it’s a great resource and I use it all the time for stuff I don’t know too much about. I never really used encyclopedias before (well, not since second grade) but Wikipedia is handy precisely because its entries DO come up so often in a Google search. (Google is the REAL research engine… along with Amazon.com as the world’s greates card catalog)

    The key here is that scholarship is exactly the act of understanding the nature of sources and how to use them. I think Wikipedia adds to the available sources at certain levels for certain kinds of learning and research. I also love the fact that it annoys the heck out of so many academics. Anything that academics find annoying (and I’m saying this as an academic) has some good in it.

    And if you don’t like Wikipedia (you = whomever, no one in particular) then you’ve gotta love the Uncyclopedia.

  16. MTran says

    Wikipedia is like the local school board: Gotta volunteer to be on it or it’ll get overrun by Fundies and other well-organized nutballs trained in how to game it.

    Wikipedia is already over-run with nutballs. There are definitely some extremely knowledgeable contributors who put up with the most tiresome tripe but most of the good ones quit in short order.

    Telling someone to “volunteer” for this type of activity is just another way of blaming the victim. It is also the most frequent, and useless, retort to those who complain about WP.

    As for the study that “showed” WP to be somehow “equivalent” to Britannica, have you ever read the details of that study? My take was that the Britannica errors were along the lines of typos and minor mistakes while WP made about the same number of errors but those errors were substantive.

    The major problem that I see with WP is that it has institutionalized the “concern troll” as the ultimate arbiter of what content is accepted.

    Doubtless, it is a useful tool, especially when it is well referenced with links to legitimate resources. It is also useful for those who teach writing courses as a site that tracks editorial changes. Writing students seldom get a chance to see a work in progress and its successive edits with the quantity, variety and detail that WP presents.

  17. says

    Greg Laden:

    I also love the fact that it annoys the heck out of so many academics. Anything that academics find annoying (and I’m saying this as an academic) has some good in it.

    Ah, that’s what I was trying to say! It reminds me a little of Michael Moorcock:

    This is not to deny that courageous characters are found in The Lord of the Rings, or a willingness to fight Evil (never really defined), but somehow those courageous characters take on the aspect of retired colonels at last driven to write a letter to The Times and we are not sure — because Tolkien cannot really bring himself to get close to his proles and their satanic leaders — if Sauron and Co. are quite as evil as we’re told. After all, anyone who hates hobbits can’t be all bad.

  18. TAW says

    “As for the study that “showed” WP to be somehow “equivalent” to Britannica, have you ever read the details of that study? My take was that the Britannica errors were along the lines of typos and minor mistakes while WP made about the same number of errors but those errors were substantive.”

    First off, learn how to use quotation marks. Nobody ever said the study showed WP and EB to be equivalent.

    Secondly, yes I did read them. EB’s errors were not just typos:

    “The reviewers, who were not told which article was which, were asked to look for three types of inaccuracy: factual errors, critical omissions and misleading statements.”

  19. says

    I’m a little curious about why Wikipedia is any worse than the Wobosphere at large. I mean, browse Crank dot Net sometime and check out the broken reasoning which flourishes outside the much-maligned encyclopedia project. . . the nonsense which nobody has the ability to stem!

  20. MTran says

    First off, learn how to use quotation marks.

    Yes, and learn how to use single quotes within double quotes, too. Or stick with the blockquote method. There is more than one reason to use quotation marks, sorry you took it as some sort of insult.

    I was happy to see the headlines for the press release about the strength of WP vs Brittanica. So I read the article as soon as it came out, and concluded that the errors in WP were more substantive than those in Brittanica, although there were certainly some weird errors in EB, as well. But the hype about WP’s error rate was just that.

    Don’t get me wrong, I like the WP. But you need to use it for what it is, with a wary eye for pranksters and with the understanding that the content can reflect propaganda wrapped in objective sounding language.

  21. Carlie says

    No one has yet mentioned Stephen Colbert’s ability to change Wikipedia on a whim? The great elephant coup was probably a bit of an eye-opener.

    I agree that in many (maybe most) cases Wikipedia is as good as any other secondary source out there, but I have to beat it into my students’ heads that neither it nor any other encyclopedic source is good enough for a resource in college level papers. Take the extra 5 minutes to follow the source links, at least!!! Why, back in my day we had to walk up two flights of stairs in the snow to find the reference books with primary source information and it was in teeny text in the heavy index books to start with… (mutter mutter mutter)

  22. David Marjanović says

    No, you don’t need to register just for editing. You only need to register if you want to do grander deeds like creating a new article.

  23. David Marjanović says

    No, you don’t need to register just for editing. You only need to register if you want to do grander deeds like creating a new article.

  24. Torbjörn Larsson says

    can you really fault people for wanting the article to follow standard references and not the personal preferences of whatever systematist is editing the article?

    And that is of course what irks some specialists.

    Chalmers is such a fool

    I don’t think he makes much sense to physicists. Dualisms, even Chalmers’ kind, are contraproductive to reductionism, which works reasonably well otherwise. Nor are they realistic, since so many are debunked.

    The relevant (physical) realism of p-zombies is a case in point. Any reasonable close (physical) copy of a system with consciousness would be conscious for reasons of continuity and stability. If we don’t know, nor will know, how to make these clones even in principle we can’t make physical assertions with them. This seems to be a rather common argument against them.

    To compound that failure in realism, perfect copies which his argument require seems to be impossible. It is of course well known (‘no cloning theorem’) that we can not duplicate perfect state information in quantum theory. ( http://math.ucr.edu/home/baez/quantum/node4.html )

    But it also seems we can not duplicate perfect classical state information of a system in the corresponding formulation of Hamiltonian mechanics! ( http://golem.ph.utexas.edu/category/2007/01/quantization_and_cohomology_we_10.html#c007314 )

    I think this means that we in principle can make reasonably identical physical clones that work when scientists like Tegmark and Aaronsson discuss physics in a universe or multiverse, for example in anthropic or suicide probing.

    But it seems we (or rather nature) can not make a perfect p-zombie to discuss consciousness as we observe it. Maybe this particular zombie in time will be conscious of the fact that he is dead and stop haunt us. But given his activities it seems improbable that he will get a decent burial.

    Blake:
    Thank you for your in depth comments here and on the “Reality is a constituency” thread – I have just been introduced to Sokal’s latest work on pseudoscience, postmodernism and religion (which I will have to read fully), and here you continue with, for me at least, educating info.

  25. Torbjörn Larsson says

    can you really fault people for wanting the article to follow standard references and not the personal preferences of whatever systematist is editing the article?

    And that is of course what irks some specialists.

    Chalmers is such a fool

    I don’t think he makes much sense to physicists. Dualisms, even Chalmers’ kind, are contraproductive to reductionism, which works reasonably well otherwise. Nor are they realistic, since so many are debunked.

    The relevant (physical) realism of p-zombies is a case in point. Any reasonable close (physical) copy of a system with consciousness would be conscious for reasons of continuity and stability. If we don’t know, nor will know, how to make these clones even in principle we can’t make physical assertions with them. This seems to be a rather common argument against them.

    To compound that failure in realism, perfect copies which his argument require seems to be impossible. It is of course well known (‘no cloning theorem’) that we can not duplicate perfect state information in quantum theory. ( http://math.ucr.edu/home/baez/quantum/node4.html )

    But it also seems we can not duplicate perfect classical state information of a system in the corresponding formulation of Hamiltonian mechanics! ( http://golem.ph.utexas.edu/category/2007/01/quantization_and_cohomology_we_10.html#c007314 )

    I think this means that we in principle can make reasonably identical physical clones that work when scientists like Tegmark and Aaronsson discuss physics in a universe or multiverse, for example in anthropic or suicide probing.

    But it seems we (or rather nature) can not make a perfect p-zombie to discuss consciousness as we observe it. Maybe this particular zombie in time will be conscious of the fact that he is dead and stop haunt us. But given his activities it seems improbable that he will get a decent burial.

    Blake:
    Thank you for your in depth comments here and on the “Reality is a constituency” thread – I have just been introduced to Sokal’s latest work on pseudoscience, postmodernism and religion (which I will have to read fully), and here you continue with, for me at least, educating info.

  26. Caledonian says

    The ‘no perfect copies’ argument doesn’t hold – we can reasonably interpret the point to refer to copies (somehow without ‘consciousness’) that don’t behave differently than copies with ‘consciousness’. Obviously any copy is going to diverge sooner or later – it’s whether there’s a distinguishable difference between the group’s behavior that matters.

    Not that it matters, since a distinction that doesn’t lead to a change in the implications isn’t a distinction.

  27. says

    Carlie:

    No one has yet mentioned Stephen Colbert’s ability to change Wikipedia on a whim? The great elephant coup was probably a bit of an eye-opener.

    I agree that in many (maybe most) cases Wikipedia is as good as any other secondary source out there, but I have to beat it into my students’ heads that neither it nor any other encyclopedic source is good enough for a resource in college level papers. Take the extra 5 minutes to follow the source links, at least!!!

    So, now we have a TV celebrity and national icon showing how information sources can be bollixed. Explain to me how this is a bad thing? At the very least, you should have an easier time explaining to your students why encyclopedias are not a good source for their term papers.

    Why, back in my day we had to walk up two flights of stairs in the snow to find the reference books with primary source information and it was in teeny text in the heavy index books to start with… (mutter mutter mutter)

    Damn that progress! ;-)

  28. jeffk says

    I’m surprised at the amount of flak Wikipedia takes. I find incredibly useful. It’s a great way to instantly know a little something about a subject with almost no effort – I’m a better educated person because of it. And while it’s not scientifically rigorous, its pages on math and physics that I’ve referenced are as good as anything online. It’s not like I’d cite it in a paper, but if I’m doing my homework and want to be reminded of how the jacobian works, it’s the first place to find it. And at the VERY WORST, it’s still the best jumping off point there is because most of the articles are beginning to have good citations.

  29. Dustin Locke says

    Thanks, Cal, I found your one-word explanation very helpful.

    You really ought to take a lesson from PZ here: insults are great… WHEN they’re backed-up by sound explanations and arguments.

  30. says

    Torbjörn Larsson:

    Thank you for your in depth comments here and on the “Reality is a constituency” thread – I have just been introduced to Sokal’s latest work on pseudoscience, postmodernism and religion (which I will have to read fully), and here you continue with, for me at least, educating info.

    You’re welcome. I’m glad at least one person has benefited from my typing.

    Sokal’s paper references the work of Meera Nanda, author of Prophets Facing Backward: Postmodern Critiques of Science and Hindu Nationalism in India (among other things). I read Prophets and found it a pretty good book. (Daniel Dennett also gave it a thumbs-up in Breaking the Spell, as I found out when I bought that book as a birthday gift for my mother.) It suffers a little from organizational concerns, I think: while reading it, I kept having the impression that “the really good part is about to begin” without the “really good part” actually arriving. Instead, the juiciest and most insightful passages are distributed throughout. I still recommend it highly.

    The central thesis of the book is summarized in Nanda’s reply to her first round of critics (PDF link), which also addresses the book’s organizational problem:

    I point out many instances of contact and convergence. But these examples are scattered through out the book, giving the impression to some that I provide “no evidence” (as suggested by Sandra Harding, Jim Maffie and Helen Verran), and to others that I am claiming too tight a fit amounting to an “alliance” or a collaboration between the two sides (as suggested by Vinay Lal and Susantha Goonatilake). If I were write the book again, I would take care to delineate, more clearly, the nature and extent of these connections in order to avoid both extremes of interpretation. For now, let me make amends and indicate, as clearly as I can, what kind of evidence I am basing my thesis upon.

    After Prophets was published, Nanda points out, further evidence came to light providing a clear example to which we can point and say, “Yes, that’s what we’re talking about!”

    Her chapter on Bhimrao Ramji Ambedkar is not to be missed.

  31. Caledonian says

    Thanks, Cal, I found your one-word explanation very helpful.

    I’m going to presume that was sarcastic and respond.

    ‘P-zombies’ are to the study of the human mind what ‘perpetual motion’ is to physics. I’m sure you can find people who will tell you Chalmers has made important contributions to his field. I’m equally sure that you can find people who will say the same of Deepak Chopra.