I worry about Mad Philosophers doing Mad Experiments


We’re all familiar with the Mad Scientist trope. You know, demented but highly intelligent guy pursues insane and obviously dangerous goal of, for instance, breeding giant spiders with which to take over the world, and no one thinks to ask why such a smart guy would think 20 foot tall arachnids would grant him geopolitical domination and power even as his experimental subjects devour him and a band of local villagers armed with flamethrowers torch his work, the end (or is it?). At least you get a little spectacle out of it all, even if the story makes no sense and any sensible person would see the flaws in the plan*.

But what would a Mad Philosopher do? Those people are really dangerous, as the people of Athens realized over 2400 years ago. They play games with people’s heads.

So Aaron Rabinowitz, classic Mad Philosopher type, did a little experiment. He created a private Facebook group, only accessible by invitation or submarine, filled it with lefties and then intentionally invited right-wing goblins to join in, too. He called it Monster Island and said it was an experiment in free speech. Yeah, right. And when I open a jar of angry wasps at a picnic, it’s just an experiment in human agility and speed.

As the resident mad scientist, I saw Monster Island as more than just a containment system for some of the worst people I’ve ever experienced: it was a place to experiment on those monsters, myself included. I wanted to study if it was possible for people from radically different worldviews to debate in an environment that had informal guidelines but no officially enforced rules. It was already clear in 2016 that the regulation of online speech was going to be a rolling disaster, producing an endless stream of rage against the moderators, who were invariably cast as biased against the right. Donald Trump himself has claimed that sites like Twitter and Facebook discriminate against right wing speech. Respectable experts have raised concerns about the lack of oversight for algorithmic moderation. Our failures to address this problem proactively seem to promote a desire in some to return to a “golden age” of the internet, before things got so big that moderation of speech on an epic scale became necessary. I had my doubts that less moderation was really the solution to our problems, and Monster Island presented a perfect chance to test my theories.

Ha ha ha. No. I could predict exactly how this would turn out: in flames and destruction. It did, and worse.

Almost immediately we had to add the guideline “no deleting” as individuals started to delete sections of posts to mess with arguments or cover spots where they’d messed up. I call it a guideline because at this point we hadn’t had to actively enforce anything beyond stating the norms. What became clear though was that even the existence of those unenforced guidelines was an affront to some monsters’ sensibilities, and so they set to work testing the fences for weaknesses. They used all the typical troll techniques. Do things that are very close to breaking the guidelines and then force everyone to argue over whether they count. Look for other horrible things they could do that weren’t technically in violation of any guidelines, just to see if it would force us to develop new guidelines in response. It was always a losing battle, because there is a fundamental asymmetry between order and chaos, and chaos always has the advantage in tempo. One troll named Ryan Balch, whose name I have not changed, for reasons that will become apparent, openly declared his intentions to destroy Monster Island, just to prove he could. Several trolls joined his cause.

The result was several years of the purest banality of evil. We ended up needing to add rules against doxxing, blocking admins, explicit threats of physical violence, and taking photos from people’s personal profiles and photoshopping them into sex acts with military dictators. Meanwhile, the quality of discourse deteriorated from semi-functional, where some folks could have actual arguments or at least do a dance that looked vaguely like presenting evidence, to endless spam of the most disturbing memes you’ve thankfully never seen.

He kept his experiment going for four years. That takes either a level of patience or of masochism to a degree beyond the scope of my imagination. I’ve been running my own Monster Island (it’s a blog called Pharyngula) for almost 20 years, and the only way I’ve been able to cope is with ruthless moderation, keeping the nasties on edge by viciously banning them when they start typing their vileness. Just picture the chaos if I did the opposite, seeding the comment threads by actively inviting creationists, TERFs, racists, and Republicans (oops, repeated myself) to participate. It would be lively, I guess, in the same way that the sack of Rome was an interactive event with an extraordinary degree of audience participation. This was the same kind of thing that happens when you give Libertarians free rein. Someone is going to be eaten by bears.

There was a predictable conclusion to the Rabinowitz Experiment.

By year three it became clear that I’d gotten all the results I was going to get from this experiment, and the toxicity of the island was consuming more and more of my time and life-force, so about a year ago I gave up and swam for shore. Many would say it’s absurd I stuck with it that long, others would call me the worst monster of all for letting the experiment go on as long as I had, and they’re all probably correct. Some of the members seemed to still enjoy the group though, so I passed control over to Peter, set sail, and never looked back. I felt comfortable concluding that unmoderated discourse faces a tragedy of the commons no different than any other unregulated communal resource. There are places online where people who strongly disagree, up to a point, can engage productively. What those groups have in common is substantial rules and heavy moderator enforcement.

There was also a Surprise Twist Ending. Like any good horror story, you may destroy the Island, but the Monsters escape and turn up in the sequel.

It’s hard to believe that a person from a Facebook group with fewer than 400 members ended up directly connected to vigilante violence against protesters. When we saw the news, Peter and I decided it was finally time, and Monster Island sank back into the internet ocean. I’m happy to see it gone, but I’m haunted by the dark irony that Ryan Balch ultimately made good on all his promises. He got out in the streets, and in doing so he destroyed Monster Island.

Go ahead, look up Ryan Balch. He turned up in the news involved in real world bloody violence.

You also get some curious cross-fertilization of bad ideas. They interbreed and recombine and you get strange new monsters that infest the world.

James Lindsay from New Discourses shared this a tweet from Christopher Rufo, a director at the creationist think tank the Discovery Institute

I’m sorry, Aaron. The Mad Scientists’ Union may have to show up at your door with torches and pitchforks, a cup of hemlock, or a trolley. Nothing personal. It’s just that your experiments are terrifying.


*As we all know, breeding small, prolific, fast spiders with aggressive temperaments and gloriously potent venom that you can smuggle onto Air Force One, the Supreme Court, and the floor of the Senate would be far more effective. Bwahahahahaha!

Comments

  1. says

    Turns out that allowing unrestricted access to a group for people whose declared goal is to wreck said group leads to… a wreck. How about that.

  2. Snarki, child of Loki says

    “when I open a jar of angry wasps MURDER HORNETS at a picnic, it’s just an experiment in human agility and speed.”

    Gotta up yer game, if you want to be in the Mad Scientist big-leagues, PZ.

  3. mathman85 says

    One wonders what, if anything, the IRB had to say about this experiment—assuming, of course, that our mad philosopher even bothered to consult with the IRB before engaging in human experimentation.

  4. chrislawson says

    I have no idea what Rabinowitz was trying to learn given that everything he reported was readily observable from existing social networks, going all the way back to Usenet (est. 1980, i.e. 40 years of freely accessible data without the need to set up toxic discussion spaces).

  5. stroppy says

    Trolls. They’re trolls.

    You’ve probably seen them in comment sections elsewhere, stirring up trouble and then bragging, in the most irritating fashion they can muster, about how they’ve been experimenting on the commenters there–a further troll. It’s not surprising that they’d try to lure people into a bottle where they could more conveniently torment their captives. Nor is it surprising that they’d brag about it.

    What I’d like to see, in a classic horror trope style resolution, is the monster they’ve created turn and destroy them. No tears for their fate from me.

  6. F.O. says

    We should really stop using “tragedy of the commons” as if it was a real thing.

    The commons work, but only when the members have skin in the game.

    Internet discussion requires moderation because the participants have nothing to lose by being absolute assholes.

  7. says

    @DexX Most ‘mad scientists’ in fiction are mad engineers. You never hear them suggest a hypothesis they are testing by growing nuclear rutabegas. There is never a control group they are studiously not exposing to the death ray.

  8. raven says

    As already mentioned, this wasn’t even a new experiment. It’s a repeat of one already done many times.

    We know without moderation that any open forum will be overrun and destroyed by trolls.
    It started with the first forums on Usenet, then AOL, then Yahoo. Most news media outlets lately have discontinued comment threads on their articles.

  9. raven says

    I did learn early on that what you see of people on the internet is an accurate reflection of real life.

    Decades ago on the internet, it was still common for people to use their real names. On one forum, one of the members started struggling with real problems in their life. Then they disappeared.

    A few months later someone posted a short news article. They had been shot and killed by the police during a very inept daylight robbery attempt at a LA jewelry store.

  10. says

    Mathman85 @4:

    IRB is bad enough. But one wonders if the experimental data collection system complies with the GDPR, since it does not appear that any steps were taken to exclude EU citizens… and one of the core precepts of the GDPR is that the collector of data must disclose the fact of data collection to all persons whose data is being collected, which would sort of shoot any concept of “experimental design” and “participant nonawareness” in the foot, wouldn’t it?

  11. Grace says

    PZ:

    …when you give Libertarians free rein.

    Love the book reference, PZ. A Libertarian Walks into a Bear is hilarious, informative, and thought-provoking. It’s a rare combination.

    Grace

  12. christoph says

    Here’s a hideous mad science experiment I heard of recently-someone combined human DNA with legume DNA to create a…(GASP)-HUMAN BEAN!

  13. gijoel says

    @9 I know right. If they welded a scythe blade to the sides of the trolley they’d get all of them. Failure of imagination man.

  14. mailliw says

    From the Simpsons

    “I sentence you to a lifetime of horror Monster Island (don’t worry it’s just a name)”

    Lisa pursued by monsters

    “I thought he said Monster Island was just a name?”

    “Monster Island is actually a peninsula”.