On LLMs and Intelligence

Given how much I loathe the subject and am annoyed by the fans, you may wonder why I hang around in spaces that are bullish on LLMs. Nowadays, the primary reason is for feedback; people who agree with you aren’t likely to search for weak spots in your arguments, whereas those who disagree will be eager to do so. It really helps sharpen them up… if you get that feedback, of course. More often than not I find my arguments around LLMs are ignored. That’s perfectly fine! No-one is obligated to provide feedback, let alone for free, and it can be a waste of time to push back on a very weak argument with obvious flaws.

But sometimes an argument lands in a weird middle ground. You can tell someone wants to push back, badly, but they can’t find a point to push on. This could be evidence they’re not up to the task, but it could also be a sign that you’ve stumbled on a solid argument. I had another one of those recently, this time centred around “intelligence.”

Typing this up is also an excuse to correct the previous time I ventured into this topic, which now seems a bit problematic. Plus, I got something wrong in the original argument as well.

[Read more…]

The Cult Of LLMs

There’s a line in Minority Report that’s stuck with me. After looking over the crime scene, the detective character turns to another cop. “This is what we’d call an orgy of evidence. Know how many orgies I had as a homicide cop? None.” The logic behind it is simple: the real world is messy, so when you stumble on something that conforms exactly to your biases, be skeptical.

In every sufficiently large business we have observed (say, with 500+ employees), we have noted that continued advancement, and increasingly continued employment, has started to require repeated professions of belief in the transformative power of AI for said business. I am not talking about providing ideas about how to use AI in the business – I mean religious profession, declarations of faith. Overwhelmingly these statements are made by non-technicians, though it is not uncommon for technicians to emit deranged statements to curry favour.

I just stumbled into an orgy, and so my instinct is to take a step back and see if I’ve missed anything.

[Read more…]

LLM’s Shouldn’t Code

My draft for “Loneliness, 3” is currently sitting at 2,600 words. It hasn’t been as hard to write as “Loneliness, 2“, this time around I only redid the intro once. Nonetheless, I haven’t touched it in a few months. The why of it all is complicated, as usual, but one not-insignificant chunk is that I’m starting to doubt my approach. I never expected to find a “magic passphrase” to got people to understand my arguments immediately, but since starting the “Loneliness” series I’ve spent more time with people who love and defend LLMs. The additional evidence and experience suggests that series is shouting into a black hole.

I don’t want to give up on it, but taking a break from it might help get me typing again. Besides, I think I can convince you LLMs should not code. [Read more…]

Aside: Let’s Bisect an LLM!

I previously took a lot of words to describe the guts of Markov chains and LLMs, and ended by pointing out that all LLMs can be split into two systems: one that takes in a list of tokens and outputs the probability of every possible token being the next one after, and a second that resolves those probabilities into a canonical next token. These two systems are independent, so in theory you could muck with any LLM by declaring an unlikely token to be the next one.

Few users are granted that fine level of control, but it’s common to be given two coarse dials to twiddle. The “temperature” controls the relative likelihood of tokens, while the “seed” changes the sequence of random values relied on by the second system. The former is almost always a non-negative real number, the latter an arbitrary integer.

Let’s take them for a spin.

[Read more…]

LLMs and Markov Chains

Pattern matching is a dangerous business, but this is now the second third time I’ve seen LLMs compared to Markov chains in the span of a few weeks.

I think people who want to characterize that as merely the output of a big semantic forest being used to generate markov chain-style output. It’s not that simple. Or, perhaps flip the problem on its head: if what this thing is doing is rolling dice and doing a random tree walk through a huge database of billions of word-sequences, we need to start talking about what humans do that’s substantially different or better. …

One thought I had one night, which stopped me dead in my tracks, for a while: if humans are so freakin’ predictable that you can put a measly couple billion nodes in a markov chain (<- that is not what is happening here) and predict what I’m going to say next, I don’t think I should play poker against the AI, either.

This seems to be an idea that’s floating out there, and while Ranum is not saying the two are equivalent it’s now in the scientific record. Meanwhile, I’ve been using Markov chains for, oh, at least seven years, so I can claim to have some knowledge of them. Alas, I didn’t really define what a Markov chain was back then (and I capitalized “Chain”). Let’s fix half of that.

[Read more…]

Guessing the Next Number

Large language models don’t really work with languages, as we think of them anyway.

At their heart, LLMs are a sophisticated version of “guess the next number in the sequence.” Their input is a long list of integers, and their output is a long list of fractional values, one for each integer they could have been fed. The likelihood of any given number being next is proportional to the value the LLM outputs for it. We can collapse these probabilities down into a singular “canonical” output by randomly picking one of those integers, taking likelihoods into account. If the LLM is being trained, that output integer is compared against what actually came next and the LLM is adjusted to (hopefully!) be more likely to output the correct integer. Want more than one integer? Shift all the input numbers up one space, discarding the first and appending the output integer to the end, and re-run the LLM. Repeat the process until no integer is all that likely, or the most likely integer is one you’ve interpreted to mean “stop running the LLM,” or you just get bored of all this.
[Read more…]