Now, I suppose the question is “what is a typical number of questions it would take a human to get this?”
Now, watch what ChatGPT does: it reinforces its own prompt by repeating elements of its questions. That’s pretty cool.
I’m reviewing this interaction and I’m not seeing any obviously better question that it could have asked. Let me also drop a mild spoiler: this is a hard item. I deliberately chose something that is used in a wide range of places and ways.
This is a chatbot trained with conversations, not an AI specifically trained to crush one problem (e.g: a chess-bot, go-bot, or DOTA-2-bot) I think it’s doing pretty well. I’m skipping a few questions. It does appear to have been thrown by the multi-purpose nature of the thing I was thinking of.
I threw in the “sort of” because I felt it was a fair response. I’m not going to say ChatGPT was flailing around at this point, but it did not seem to have a clear goal. Which, naturally, makes sense: the goal is concealed.
Now, it’s time for old Marcus to raise the standard for humanity and beat this simple contraption of bits and bytes!
Um, the fate of humanity is not looking good, against SKYNET, here. Then, I started hitting it with curveballs. Usually when playing with a human, you can get clues from how much brow-furrowing or waffling you get with the response. You have to imagine me typing these very fast, not that it matters.
The AI also chose a pretty tricky thing. But then, this gets weird.
It took 10 questions.
I’m going to bask in my temporary superiority.