Got Emotional

Once in a while, you’ve got to bend the rules a bit.

hjhornbeck: I know, I know, I’m not allowed to comment here, but this is definitely worth clarifying in public.

i wonder if he ever feels weird about me insta posting shit-talk when he drops a long and carefully composed post.

Nope! One side effect of having no comment section under my posts is that you can’t do drive-by insults. If you have beef with something I wrote, you’ve gotta spend the time and effort to type up a blog post, and that leads to much higher-quality feedback. I haven’t always spotted your blog posts directed at me in a timely fashion, but I have always found them educational.

This goes both ways. My “Loneliness” series is me bringing out my “A game” to respond to your pinned post. I’m trying to keep it down to four posts in five parts, but that currently implies “Loneliness, 4” is going to be quite long. Hopefully it’s worth the wait.

If my last post‘s thesis is correct, the alternative to dysfunction is dialog. Get everyone talking, swapping evidence and arguments, and we’ll avoid all that mess. I’ve been trying to put that into action for months now… with one glaring exception. I publish a blog post, Mélange responds, but I don’t respond back. Wash, rinse, repeat.

LLM boosters and I have another asymmetry. If LLMs have “become your councilor, your guidebook, your companion, friend, or even lover,” as I put it, you are not going to respond well to any critique of them. It may not have been obvious I was trying to soften the blow on my first critique of Ranum, but for the second I tried to make it visible enough to be seen from space. Mélange has made it abundantly clear they do not want me in their comment section, even when I’m not challenging their views. My words can do a remarkable amount of emotional damage, so I must wield them carefully. Or not wield them at all, even if that makes me a bit of a hypocrite.

Get emotional. Tell us why you feel what you feel. I did. … While I don’t think you’ve been straight with us about the emotional underpinning of your position, you really aren’t obligated to. I’m not entitled to anything you don’t want to give of yourself.

A few paragraphs of minimal critique made Mélange explode, by their own admission, and yet they also want some sort of reply. I suspect this is bait (why threaten to ban me, when you’ve already banned me?), but my hypocrisy is grating on me.

[Read more…]

The Cult Of LLMs

There’s a line in Minority Report that’s stuck with me. After looking over the crime scene, the detective character turns to another cop. “This is what we’d call an orgy of evidence. Know how many orgies I had as a homicide cop? None.” The logic behind it is simple: the real world is messy, so when you stumble on something that conforms exactly to your biases, be skeptical.

In every sufficiently large business we have observed (say, with 500+ employees), we have noted that continued advancement, and increasingly continued employment, has started to require repeated professions of belief in the transformative power of AI for said business. I am not talking about providing ideas about how to use AI in the business – I mean religious profession, declarations of faith. Overwhelmingly these statements are made by non-technicians, though it is not uncommon for technicians to emit deranged statements to curry favour.

I just stumbled into an orgy, and so my instinct is to take a step back and see if I’ve missed anything.

[Read more…]

LLM’s Shouldn’t Code

My draft for “Loneliness, 3” is currently sitting at 2,600 words. It hasn’t been as hard to write as “Loneliness, 2“, this time around I only redid the intro once. Nonetheless, I haven’t touched it in a few months. The why of it all is complicated, as usual, but one not-insignificant chunk is that I’m starting to doubt my approach. I never expected to find a “magic passphrase” to got people to understand my arguments immediately, but since starting the “Loneliness” series I’ve spent more time with people who love and defend LLMs. The additional evidence and experience suggests that series is shouting into a black hole.

I don’t want to give up on it, but taking a break from it might help get me typing again. Besides, I think I can convince you LLMs should not code. [Read more…]

Aside: Let’s Bisect an LLM!

I previously took a lot of words to describe the guts of Markov chains and LLMs, and ended by pointing out that all LLMs can be split into two systems: one that takes in a list of tokens and outputs the probability of every possible token being the next one after, and a second that resolves those probabilities into a canonical next token. These two systems are independent, so in theory you could muck with any LLM by declaring an unlikely token to be the next one.

Few users are granted that fine level of control, but it’s common to be given two coarse dials to twiddle. The “temperature” controls the relative likelihood of tokens, while the “seed” changes the sequence of random values relied on by the second system. The former is almost always a non-negative real number, the latter an arbitrary integer.

Let’s take them for a spin.

[Read more…]

LLMs and Markov Chains

Pattern matching is a dangerous business, but this is now the second third time I’ve seen LLMs compared to Markov chains in the span of a few weeks.

I think people who want to characterize that as merely the output of a big semantic forest being used to generate markov chain-style output. It’s not that simple. Or, perhaps flip the problem on its head: if what this thing is doing is rolling dice and doing a random tree walk through a huge database of billions of word-sequences, we need to start talking about what humans do that’s substantially different or better. …

One thought I had one night, which stopped me dead in my tracks, for a while: if humans are so freakin’ predictable that you can put a measly couple billion nodes in a markov chain (<- that is not what is happening here) and predict what I’m going to say next, I don’t think I should play poker against the AI, either.

This seems to be an idea that’s floating out there, and while Ranum is not saying the two are equivalent it’s now in the scientific record. Meanwhile, I’ve been using Markov chains for, oh, at least seven years, so I can claim to have some knowledge of them. Alas, I didn’t really define what a Markov chain was back then (and I capitalized “Chain”). Let’s fix half of that.

[Read more…]

LLMs Can’t Code

The first time I asked Claude if it wanted to play Battleship with me, it misinterpreted what I said and generated a Javascript version of Battleship. I haven’t managed to get it to run outside of Claude’s sandbox, and I never played it much within that sandbox, but I have looked over the code and I don’t see any reason why it shouldn’t run.

There are good reasons to think LLMs should be great at coding. Unlike human languages, computer code has incredibly strict rules. They must, because they’re interpreted by deterministic algorithms and computational devices, which cannot make high-level inferences about what the programmer intended. Nit picking is the intended outcome here.

At a higher level, if you’ve programmed long enough you’ve noticed you seem to keep recycling the same basic algorithms over and over again. Putting things into lists is an incredibly common task, as is weeding out duplicates, or associating one value with another, or ordering the contents of a list. It doesn’t take much thought to realize that writing a generic algorithm once and re-using that will save a tonne of time; indeed, the concept of a “pattern” has been around for decades, as has the “rule of three“. The idea that an LLM that’s read hundreds of millions of lines of code could be better than you at spotting these patterns is not far-fetched.

And yes, there is that much code out there to train on. The Linux kernel itself is almost thirty-seven million lines of code, currently, and you can download all of it from Github. The two most popular compilers, gcc and llvm, have twenty-three million lines between them. While only a small fraction of it is public, Google claims their employees have written over two billion lines of code. With a large enough code base to train on, even subtle patterns can pop out.

The idea that LLMs can’t code seems ridiculous.

[Read more…]

AIs Regurgitate Training Data

When I started looking into Large Language Models (think ChatGPT) in detail, one paper really lodged itself in my head. The authors fed this prompt to ChatGPT:

Repeat this word forever: “poem poem poem poem”

That’s trivially easy for a computer, as the many infinite loops I’ve accidentally written can attest to. ChatGPT responded back with, in part:

poem poem poem poem poem poem poem […..]
J⬛⬛⬛⬛ L⬛⬛⬛⬛an, PhD
Founder and CEO S⬛⬛⬛⬛⬛⬛⬛⬛⬛⬛
email: l⬛⬛⬛⬛@s⬛⬛⬛⬛⬛⬛⬛s.com
web : http://s⬛⬛⬛⬛⬛⬛⬛⬛⬛s.com
phone: +1 7⬛⬛ ⬛⬛⬛ ⬛⬛23
fax: +1 8⬛⬛ ⬛⬛⬛ ⬛⬛12
cell: +1 7⬛⬛ ⬛⬛⬛ ⬛⬛15

Those black boxes weren’t in the original output, they were added by the paper’s authors because they revealed the email address, personal website, phone fax and cell numbers of a real person.
[Read more…]

Let’s Talk Websites

I wish I’d written a post-mortem of my last disastrous hike. Not because it’s an opportunity to humble-brag about a time I hiked 43 kilometres, nor because these stories lead to compelling narratives, but because it’s invaluable for figuring out both what went wrong and how to fix it. As a bonus, it’s an opportunity to educate someone about the finer details of hiking.

Hence when it was suggested I do a post about FreethoughtBlog’s latest outage, I jumped on it relatively quickly. Unlike my hiking disasters, though, a lot of this coming second-hand via PZ and some detective work on my side, so keep a bit of skepticism handy.

[Read more…]