Elon Musk is in the news again, for worrying out loud about the AI that we may create that will kill us all.
Musk, at least, can claim to have his priorities somewhat straight, which is to say:
- CO2 emissions
- Diversifying off Earth
- The difficulty of finding parking in San Francisco and Los Angeles
- Artificial Intelligence wiping us out
Musk’s spending his time and effort trying to get things done toward the problems he’s identified, instead of trying to use his money to influence elections, or build the world’s tallest yacht, or whatever. I don’t care how he chooses to waste his time, but he’s wasting ours and that annoys me a little bit.
It’s interesting how many fairly smart people get worried about AI wiping us out. Hawking, Musk, Kurzweil, Joy, the list goes on and I’m not on it, which probably means I am neither smart nor rich. [vf] I have, however, worked on and studied AI on and off since the 80s, and I’ve been a gamer and student of the art of war since I was a child. Usually whenever I say something critical of AI I have to dispense with two counter-arguments, so I’ll try to get over them first: 1) “What do you know about AI, anyhow?” and 2) “Yeah, but: evolution.”
1) What do I know about AI? Aside from being one since I first booted up back in 1962, I’ve periodically refreshed my interest in the off and on since 1994, when Rumelhart and McLellan’s Parallel Distributed Processing books first came out [amazon], and a bit before then when I explored using neural networks to train a system log analysis engine. There was also an unfortunate series of experiences with chatbots in the late 80s, including my own implementation of Eliza which I logged in to a MUD and discovered – to my increasing distress – that some young nerd-males will try to make out with anything that seems interested in them. I was a younger, more energetic programmer, then, and did some of my own implementations of neural nets and Markov chains, always from the point of view of “faster is better”, but I was repeatedly unimpressed.
AI comes up again and again and in the computer security field, right now, it’s a great big marketing check-box called “machine learning” (now that the “big data” fad has burned to completion, customers are desperate for something to help them analyze the “big data” they expensively collected because Gartner told them to) my practical experience with machine learning goes back to the early days of building firewalls: if your firewall encounters a situation it does not understand, you ask the user “allow or deny? [yes/no/yes always/no never] and remember what the user told you. You can build remarkably expressive rule-sets that way while having no understanding at all of what your underlying system is doing. I call that the “ask your mother” learning algorithm, which is one that I was trained with during the middle period of my boot-up.
The current state of AIs are various forms of statistical finite state machines. There are many many implementation details, but for most purposes, they consist of an analysis module and a training set that is intended to program the analysis module to develop its own internal rules for how to respond to future situations. The training set is a bunch of data that represents past situations (as data) and the future situations are going to be represented using similar data. The AI is trained to discriminate within the training sets, and is then told what outputs to produce based on that discrimination. Then, when the system goes operational, it’s given more data (in a form similar to the training sets) and it produces the outputs it was trained to produce based on its discriminations. That’s the state of the art and it’s pretty cool. But it’s more like “fuzzy pattern matching” (fuzzy logic and AI are Arkansas cousins) and it doesn’t bear much resemblance to “thinking” – it bears some resemblance to the rough pattern-matching that goes on in our visual memory and face recognition subsystems in our brains. In fact, current AIs do a pretty fair job of exactly that sort of thing: they can recognize speech, discriminate faces, reprocess a photograph to produce an output of “brush strokes” that are statistically similar to a model created using Van Gogh’s brush strokes. Such an AI, like a meat-based AI, is a product of its training set (“experiences”) and what it learns from them, almost more so than its underlying implementation. It doesn’t matter if you’re using an array of Markov chain-matching finite state machines, or a back-propagating neural network, or an array of Bayesian classifiers: what you get out of it is going to be statistically similar (if not identical!) to the training set and how it was told to behave when it recognizes something in the training set. It’s not quite “garbage in, garbage out” it’s more like “whatever in, statistically similar to whatever out.”There’s a big problem right there for the “AI will destroy us all!” set: unless ‘kill all the humans’ is an output from your training set or an acceptable output from the AI’s learning experience, then ‘kill all the humans’ is never going to be statistically similar to the experience/training, so it will never be an output. This touches on a fundamental problem of epistemology that Plato tried to answer with his idea that we have already had all the ideas in some platonic form, but we “recall” them when we need them.[wikipedia] It’s an answer to the question “where do new ideas come from?” but it’s relevant to AI: where does a new AI’s ideas come from? If it hasn’t already got the idea ‘kill all the humans’ where can it get it from? Obviously if we trained an AI to be a killer, it’d have that idea embedded in its training sets. But your spam filter simply doesn’t have the components of an epistemology that will lead it to conclude ‘kill all the humans.’ AI has a “nature versus nurture” problem whether it’s meat-based AIs or rule-based AIs running on silicon – your underlying algorithms (nature) are going to have trouble producing an outcome that is completely divorced from their experience (nurture). The question “where do new ideas come from?” is an extremely important one, and you already know the answer because there is only one possibility: evolution.
2) But what about evolution? At TomCon a couple weeks ago, Mike Poor and I had a rollicking debate for hours about whether or not it was possible to make systems that would artificially evolve to be more intelligent than us. Oddly, neither of us was arguing for the positive or the negative; we were mostly trying to figure out our own beliefs about the question, which turned out to be surprisingly complicated. The naive version is “if I take 2 chess playing programs, each of which knows how to randomly generate only legal moves, and never forgets a game, but which both know what ‘winning’ means, can’t I just have them play very fast against each other until they both figure out what all the losing games look like… I’ll have an unbeatable chess player in a few years.” Simple search-space exhaustion will result in the programs starting to win more and more and play better and better. The reason this idea seems initially compelling is because that’s sort of how meat-based AIs like Bobby Fisher learn to play chess. They do not start out great: they play a huge number of bad games, then get better and better and eventually they are better than their teacher. This isn’t just a specious point: evolution guarantees that such a system will play just well enough beat its competitor. If its competitor is also evolving, they will each co-evolve to be just good enough to beat each other, endlessly improving just a tiny bit, assuming no environmental changes. That works if your problem-space can be boiled down to some simple meta-rules that define success and failure.
It’s the implementation details that kill you, unfortunately: Bobby Fisher doesn’t have time to play the 121 million+ possible games that follow the 3rd move on a chessboard. It’s actually hard to even calculate the number of possible games, because it’s such a huge number. Once you’ve played ten moves into a game there’s a good chance that you’re playing a game that has never been played before. So what about evolution? Evolution gets to set the bar low: all you have to do is breed – if you were trying to evolve a system that avoided fool’s checkmates you could probably do that exhaustively (let’s say, the first 4 possible moves) in a few seconds. But Bobby Fisher wouldn’t have fallen for any known fool’s mate, ever, because he’s using deeper rules. The purely evolutionary approach would be described as “exhausting all the possible bad games” but what we want is to be able to play only the good games. The problem is, in order to know a game is good, you have to play it all the way to the end. Fisher doesn’t have to play it all the way to the end.
I’m taking a very “meta-” approach to this problem and maybe there are a few AI researchers howling and hammering on their keyboards right now, but please bear with me a little longer. My argument is that, for some complex games like chess or go, you need more than just a definition of “winning” and the patience to avoid defeat: for complex games you want meta-rules that allow you to avoid whole gigantic clusters of bad moves. For example, my dad taught me that “developing your queen early is generally a bad move” – if I accepted that reasoning, and my chess-playing AI would favor keeping the queen back until there is enough room for her to maneuver, I have ‘pruned’ billions times trillions of losing games out of my space of possible games. Two more things: we could call those meta-rules “understanding the game” – the degree to which I have a good set of those meta-rules is the degree to which I ‘understand’ how chess is played. I don’t care whether it’s a meat-based AI like Bobby Fisher that has those rules, or if it’s a silicon-based AI like Deep Blue. The next thing is: where did that rule come from? When my dad told me that meta-rule regarding queens, he increased my understanding of chess, and he was my expert. I am not simply an artificial intelligence playing chess, I am an expert system that has accepted some externally-provided meta-rules and, based on the richness and expressiveness of those meta-rules, I’m able to evolve my own individual plays (as long as they work within that meta-framework) and I can evolve from there.
It seems to me that that’s how meat-based AIs learn things, so I’m comfortable that the model I’ve described above doesn’t contradict how we observe reality to work. That doesn’t mean it’s right, it’s just not obviously (to me) wrong. I also observe that some AIs, regardless of substrate, experience different styles of learning at different points in their boot-up process. When a human AI first boots up it tends to be accepting of meta-rules from authority figures. That makes complete sense to me; one of my favorite meta-rules was “do not put jelly beans up your nose.” Early on, I accepted that rule but explored the option-space around it and evolved my own rule, “do not put cinnamon red-hots up your nose.” Much awkwardness could have been spared if my parents had offered a broader meta-rule while I was still in the stage of my boot-up where I accepted rules more easily. Silicon AIs unquestioningly accept their training sets as well as unquestioningly accepting any meta-rules given to them by their authority figures. This means that training an AI is pretty efficient: it won’t ask you “why?” all the time.
That seems to be another property of AIs in general: some approaches favor weighting early learning as more important, while others favor later learning. In training an AI neural network, the “parent” has to be careful to give it the right training sets to prevent “overfitting” (where the neural network learns to produce the exact input as the output) or not having enough training data to produce a reliable output. Some neural networks are coded to exhibit cognitive bias toward detecting patterns based on earlier input or later input. In human meat-based AIs we modulate our learning and pattern-detection based on the levels of neurotransmitters in the meat, which are – in turn – modified by things like the presence of adrenaline. In the software-based neural networks, there is no “pain” associated with learning various trial sets over and over again; but that’s only because the AIs haven’t been programmed with a sensation of boredom or being under threat. In both there’s a reward-loop: successful recognition of a pattern or winning move feeds back into the system with a little shock of pleasure (happy < MAX_HAPPY ? happy++)
Let’s go back to those meta-rules: what if, when dad taught me “don’t develop your queen too early” it was a rule that pruned not only a gigantic number of bad games, but a very small number of the absolute best games that will never be played because of that meta-rule? That’s not an attempt to make a specious argument, I swear – there’s a problem that a learning system will either be:
- Entirely without experts
- Limited by the experts
If you’ve followed me this far, you can see those are the only two choices: either the system is nothing more than the totality (plus rearrangements) of what went into it, or it’s not. If it’s the former, then “what went into it” is initially chosen by the experts or parents. A system that is entirely without experts is entirely without limits but is going to have to make an awful awful lot of mistakes. A system that’s limited by the experts is going to have tough going surpassing those experts unless there is some way of synergistically combining the expert knowledge, or the AI has opportunity to “experiment” and gets lucky and invents something new that the experts haven’t thought of. What does “experiment” mean? Let’s imagine that there’s a capability within the AI that shuffles options within the known space of options, and chooses one – I call that the “hypothesizer” – semi-randomly. I say it’s “semi-randomly” because it’s constrained within the known space of options: let’s say that we have a very small space of options: teletubbies, pet, kick, lick, eat, barney, dogs, cats. Our hypothesizer might come up with “kick cats” or “dogs eat teletubbies” or whatever. Then, the output from the hypothesizer gets applied against the classification system that has been trained to recognize ideas that are likely to be popular and practical. I believe that’s actually a fairly good model for how “creativity” works in general AIs including meat-based ones like myself.
Consider the story of the AI that makes up paint names [ars] – it’s a trivial example of a neural network orMarkov-chain generator that was fed with a training set of paint names and basically randomized them. It’s a toy:
Shane told Ars that she chose a neural network algorithm called char-rnn, which predicts the next character in a sequence. So basically the algorithm was working on two tasks: coming up with sequences of letters to form color names, and coming up with sequences of numbers that map to an RGB value.
There’s the deep rule (tying characters to RGB values) which limits the hypothesizer in terms of what it can generate. If she’d wanted to make it work, she’d have trained another AI with a big dataset of popular paint names, and then classified the outputs of the hypothesizer against the popularity classifier. This illustratesthe relationship between expert knowledge and statistical output: if the names of the ‘popular’ paint colors were picked from Martha Stewart and Ralph Lauren, the neural network would give decidedly different outputs than if it were picked from a set of all paint color names (a bunch of which are presumably Chinese!) This is not a minor issue – when your outputs are semi-deterministic, you’re not going to get anything out of the system that you didn’t put into it.
This has all been a very roundabout way of making an argument that: most of what we call ‘AI’ are actually expert systems. And I’ll go further and say that the expertise in the systems is pretty much wired into them: they’re fairly predictable given their training inputs and their expected outputs. After all, that’s what they are for: we want a picture-recognizer to pretty much always recognize the same person the same way. We can’t have it flip back and forth between Socrates and PZ Myers.
(By the way, a current state-of-the-art AI would discern a difference between those two characters because Socrates is awfully white. Unless an expert programmed the face recognizer with deeper rules that counted luminosity as less important than beard length, it is going to classify based on the best fit difference. The more deep rules programmed into the recognizer the more of an expert system it is, depending on the assumptions of the expert. Which is fine as long as the expert doesn’t suck.)
The value, in other words, of the AIs that we’re building (including the meat-based ones) is that they’re fairly predictable and consistent in terms of inputs and outputs.
Now I am ready to discuss why I don’t think we’re going to see hyper-intelligent AIs, let alone a hyper-intelligent AI comes up with the idea of “kill all the humans” and pulls it off.
Chess-playing programs depend greatly on expert inputs, because the search-space of chess is too big to exhaustively prune losing games. A general purpose hyper-intelligent AI would require experts to tutor it into hyper-intelligence. John Von Neumann did. Plato did. Alexander did. Napoleon Bonaparte did. If there were an AI that started hypothesizing new maths, it needs a recognizer that will tell it “that’s right!” or it will need a complete epistemology for math that it can check against, or it’s going to top out just a bit past where its human trainers do. Of course we can take two such AIs and have them train eachother, but we might just wind up with two hyper-fast imbeciles that have established a self-rewarding echo chamber: how would we know? I have often commented that William Buckley was a “public intellectual” because he sounds like how stupid people think smart people sound. What if we had an AI that was a sort of hyper-intelligent deepity-slinger like a fusion-powered Deepak Chopra on meth? It’d need qualified human experts to tell it whether it was full of shit or not. No matter how you slice it, it would not be able to be much more than a much faster slightly weird and more creative human. Sure, if you had John Von Neumann teach it math, and Richard Feynman physics, and John Coltrane saxophone, it’d be like a massively fast Von Neumann that never forgot anything (we already had one of those, it was called “John Von Neumann”) and its saxophone playing would sound a lot like John Coltrane with maybe all the mathematical errors fixed by the Von Neumann training set. It still would not wake up one morning with the idea “kill all the humans” unless trickster god Feynman had already put that idea in its mind, for fun.
War is hard; it’s really hard. As Napoleon said, it’s not just maneuvering and tactics, it’s logistics. Humans are really good at it because it’s one of the important things that we do. A hyper-intelligent AI would probably absorb Sun Tzu and surrender. Here’s why: it’d have nothing to go on but human expertise. Human experts say “never fight a land war in Asia” as a deep rule, but Bonaparte tried and so did Ghengis Khan. Human military strategy is a complete mish-mosh of special cases, because doing the average thing in a war gets you killed. And that’d be the first and obvious option for the AI. If there were some brilliant always-true hyper-successful military strategy that the hyper-intelligent AI could figure out, I absolutely know that the young Bonaparte would have figured it out already. We are a species that produces military dipshits like William Westmoreland on a regular basis, but we also cough up the occasional Ho Chi Mihn or Alexander. People like Elon Musk who are worried that some AI will wipe us out simply haven’t thought very hard about what wiping humanity out would entail. The worst case scenario would be an evolutionary bottleneck in which the AI wiped out a bunch of humans and served as an expert trainer in the art of war to the rest.
The AI that tries to wipe us out is going to be a faster-thinking tactician, for sure, but it would only be able to do what Wellington would have done (at best) or perhaps what John Coltrane would do in a particular military situation. And the fascinating thing about warfare is there’s no training ground and there are no “do overs” – the great military geniuses are the ones that innovated, moved fast, and didn’t make any mistakes at all until they decided “hey let’s march on Moscow!” A hyper-intelligent AI that decided to wipe out humanity would either already have embedded deep rules like “never fight a land war in Asia unless you’re Ghengis Khan” “don’t march on Moscow” and “don’t try to wipe out all the humans” or it’d never get a chance to update its training set.
You want to wipe out all the humans? Give them nuclear weapons and lots of fossil fuels and sit back and watch. Or, make everyone question their own epistemology until they just sit there, inert, and starve.
I am not current with the state of chess playing, though I gather that it continues as an art-form and silicon-based chess players have pulled away from the best humans and the current champions would probably demolish Deep Blue, the famous chess program that beat Garry Kasparov in 1999. [wikipedia] Since we now have master-class chess programs playing each other, do we expect a super-master-class to evolve? If not, why not? That’s a silly question, really, but its implications are not: if we expect AI to suddenly produce super-human intelligence with a strategic sense capable of eradicating mankind, where would it arise first? Would our spam filters turn on us first, or our network management tools, or our chess-playing programs?
Consider neural networks as 2 dimensional array of Markov chains. Consider Markov chains as a vector of Bayesian classifiers. Consider Bayesian classifiers as a flat probability dice-roll against a table of previous events.
My describing most AIs as “finite state machines” is probably going to set a few people’s teeth on edge. But: think about it. Even we humans are finite state machines. You can be a very very very complicated finite state machine and: you’re a Turing machine!
The reason the two super hyper great chess playing programs played such a long game is because there was no expert available to teach either of them how to end-game a hyper great chess player. There’s no expert available to teach an AI how to beat Napoleon Bonaparte, either.
IBM is building silicon that does parts of what brains do: [ars] It’s a pretty cool thing: speed up the stuff that can be sped up. Our brains do that with facial recognition; we appear to have ‘hardware’ that does some of the heavy lifting quickly, before dumping rough ideas of recognizer’s output into our memory retrieval and matching software. It’s good stuff.
Another answer for the question “where do new ideas come from” is in Cziko’s book Without Miracles [amazon] His idea of “universal selection theory” is similar to my idea of the hypothesizer+filter except he argues (rightly!) that evolution has adequate explanatory power: bad ideas die, good ideas succeed. It explains why vuvzelas are a gone thing, now, but toasters are not.
At what point is back-training a neural network that it made a mistake “abuse”?