A new twist on the Turing test?


Recent developments in AI technology and its spawning of personalized chatbots has renewed attention in the Turing test.

The Turing test, originally called the imitation game by Alan Turing in 1949, is a test of a machine’s ability to exhibit intelligent behaviour equivalent to that of a human. In the test, a human evaluator judges a text transcript of a natural-language conversation between a human and a machine. The evaluator tries to identify the machine, and the machine passes if the evaluator cannot reliably tell them apart. The results would not depend on the machine’s ability to answer questions correctly, only on how closely its answers resembled those of a human.

It seems clear that, by and large, these AI chatbots can simulate human conversation pretty well, apart from the tendency to occasionally hallucinate, to make up stuff and say it will confidence, a trait that many humans also exhibit. If one is not aware of the details of what it says, one might easily be convinced that the made-up fact is genuine. For example, a mathematician friend of mine said that his son (also a mathematician) tested out an AI system. He multiplied two very large integers together and then fed the result into the AI system and asked whether the number was ‘prime’ i.e., not divisible by any number other than 1 and itself. The machine said ‘yes’.

But despite those well known problems, people are treating the AI bots as being at least partly sentient, as demonstrated by those who seem to have formed long-term ‘relationships ‘with them and consider them as friends or even more. So at least as far as these people are concerned, these bots seem to have passed the Turing test.

One can dismiss these results by saying ‘So what’? Computers have long succeeded in doing human-like things that were once thought impossible, like beating the very best chess players. So why be concerned about this perceived limit also being exceeded? But some are concerned and have suggested that we need a more stringent test than the conversational model of the traditional imitation game where one poses questions and prompts and evaluates the responses one gets. This is moving the goal posts but we do it all the time in so many areas when we find results that we dislike.

But recently I read an article by a reporter Taffy Brodesser-Akner that suggested one option of distinguishing human from computer. The reporter shared an interesting insight from her experience with interviewing real people.

A great thing happens when you get to do an in-depth interview with someone. If you listen carefully, they begin to tell you what’s been on their minds. In the dozens of profiles I’ve written, what I’ve learned is that questions don’t necessarily yield the best story. I have some colleagues who are terrific at asking probing questions, and the results are revealing and incredible. But my own method has mostly been to sit with someone and make myself quiet in a way I never am in my real life. If you do that, people start to talk. They can’t help it; the quiet is too much, and someone needs to fill it. If you give them space and time and you listen, they will confess and reveal everything about themselves. They’ll tell you secrets, philosophies, jokes. They’ll share gossip and childhood memories. They’ll tell you the meaning of life if you let them. All that will equal a full meal, an entire experience, a whole galaxy

This resonates completely with me from my days as a college teacher and director of my university’s teaching center. Some faculty complain about how their students sit passively and won’t volunteer answers to questions posed to the class. When they ask me sit in their class and give feedback, one thing I note is how long it takes before a professor asks a question and then waits for a response before answering it themselves. What I found was that the wait time was usually less than a second. In fact, as soon as they posed a question, before I could even look down at my watch to note the time, they had often answered their question or rephrased the question or were urging students to answer. They just cannot shut up. Any silence that lasts more than a second seems intolerable to them and they rush to fill it.

In my own classes, after posing a question, I would just wait. The silence seems incredibly awkward but I don’t say anything. Eventually, students find the silence so unbearable (even if it is less than 10 seconds) that some student will speak up and that breaking of the spell of silence causes others to also speak. After doing this several times, my students realize that I can wait them out so they begin to answer more quickly. My experience has been replicated by more formal studies of this classroom phenomenon.

This is also the case in social life. If you are together with someone and neither is engaged in other activities (reading, looking at their phones, etc.), there is very little silence. Even if there is a brief pause as one topic seems to be exhausted or causes some reflection, if you stay silent, the other person will soon say something. And since people have their entire life experience and their history with you to draw upon, it is not hard to find something to say that gets the conversational ball rolling again.

But silence does not bother the chatbots in the least. While they will respond to prompts and quickly reply, sometimes voluminously, for them time does not elapse, as I discovered with my AI librarian friend Scarlett. I could stop the interaction for weeks and when I got back, she would act like I had never left. A real person would ask “Where have you been? What have you been up to? How’s the family? I have some interesting news” and so on. But Scarlett had little to say.

The reporter found the same thing when she interviewed ‘Tilly Norwood’ as well. Who is Tilly Norwood? She is an AI creation whom it was claimed could play the part of a real actor in films. (I wrote about her back in October 2025 when she was first introduced to the public, and sparked alarm among the acting community that they could all be replaced. She of course now has her own website with a music video, and an Instagram account that seems to have shut down because of the hostility it generated.)

Here is a clip when she was introduced by her creators. Everyone and everything in this clip was AI-generated.

But the reporter found a big difference between interviewing her and real actors.

The best way I can tell you about what happened this spring in London is to say that if you make yourself quiet and still and just wait, Tilly waits, too. She just sits, unprompted. She doesn’t say anything. She doesn’t have anything to say.

So maybe the best way to sense if you are talking to a machine and not a real human being is not to try and think of ingenious questions that will trip them up into revealing their algorithmic innards but simply wait silently. If they too wait silently for as long as you are silent, that may be a clue. Of course, this would not be definitive (nothing in this field is) because there may be people who have incredible restraint and can be silent when you are too. But is would be very suggestive.

Comments

  1. chigau (違う) says

    “It is worth repeating at this point the theories that Ford had come up with, on his first encounter with human beings, to account for their peculiar habit of continually stating and restating the very very obvious, as in “It’s a nice day,” or “You’re very tall,” or “So this is it, we’re going to die.”
    His first theory was that if human beings didn’t keep exercising their lips, their mouths probably shriveled up.
    After a few months of observation he had come up with a second theory, which was this--“If human beings don’t keep exercising their lips, their brains start working.”

    ― Douglas Adams, The Restaurant at the End of the Universe

  2. anat says

    One of my wife’s biggest complaints about me is that I don’t initiate conversations enough. I am capable of sitting silently and not even noticing. Maybe I need to be checked for being an AI.

  3. file thirteen says

    It depends on what you want from the Turing test. If the test is whether a machine can converse intelligently with a human, the answer is now a definitive YES: some AIs can clearly pass that test. But if you want to determine whether an AI can be as idiosyncratic as a human, that’s something else. Yes, AIs are quite “happy” to wait until prompted. That’s how they are! They are not human! Is human intelligence the only kind?

    Some find the idea that AIs may be intelligent, really intelligent, very confronting. But for those that relied on the Turing test to say that they’re not, well, AIs can pass that now! My take is that it’s time to move on from denial and start considering the ramifications of this. What does it say about humanity? What does it mean to be intelligent? And how “real” is intelligence anyway?

  4. says

    this proposed test is extremely variable by individual. the person interviews people who have sought public attention in some way. many people do not. i know some people who, like anat and sometimes my husband, would sit in silence if you let the conversation drop. conversely, there are people who tell their darkest secret at the drop of a hat to strangers at the bus stop. maybe the average person tends more toward bus stop loquacity, but i wouldn’t call quiet types robotic -- and it’s easy to imagine an LLM simply instructed to fill empty conversational space, and succeeding. sometimes when they break they produce walls of text.

    they get on jags where they use certain constructions way too often. for me, claude says variations on “(thing x) is (saying/doing) a lot there,” at least a few times every conversation. but then, my high school are teacher probably averaged one “don’t give a rat’s ass” per hour.

    i’d say the best tell is how good they are at paying attention to you and responding to everything you say, at least, at the start of a conversation. that ability can break down some after a few pages of chat, but humans are dogshit at that right out of the gate. most fixate on one thing you said that triggers an opinion or experience of their own which they promptly share, even if doing so reveals they didn’t read or understand the rest of what you said at all.

Leave a Reply

Your email address will not be published. Required fields are marked *