This Kevin Kelly dude has written a summary that I find fully compatible with the biology. Read the whole thing — — it’s long, but it starts with a short summary that is easily digested.
Here are the orthodox, and flawed, premises of a lot of AI speculation.
- Artificial intelligence is already getting smarter than us, at an exponential rate.
- We’ll make AIs into a general purpose intelligence, like our own.
- We can make human intelligence in silicon.
- Intelligence can be expanded without limit.
- Once we have exploding superintelligence it can solve most of our problems.
That’s an accurate summary of the typical tech dudebro. Read a Ray Kurzweil book; check out the YouTube chatter about AI; look at where venture capital money is going; read some SF or watch a movie about AI. These really are the default assumptions that allow people to think AI is a terrible threat that is simultaneously going to lead to the Singularity and SkyNet. I think (hope) that most real AI researchers aren’t sunk into this nonsense, and are probably more aware of the genuine concerns and limitations of the field, just as most biologists roll their eyes at the magic molecular biology we see portrayed on TV.
And here are Kelly’s summary rebuttals:
- Intelligence is not a single dimension, so “smarter than humans” is a meaningless concept.
- Humans do not have general purpose minds, and neither will AIs.
- Emulation of human thinking in other media will be constrained by cost.
- Dimensions of intelligence are not infinite.
- Intelligences are only one factor in progress.
My own comments:
-
The whole concept of IQ is a crime against humanity. It may have once been an interesting, tentative hypothesis (although even in the beginning it was a tool to demean people who weren’t exactly like English/American psychometricians), but it has long outlived its utility and now is only a blunt instrument to hammer people into a simple linear mold. It’s also even more popular with racists nowadays.
-
The funny thing about this point is that the same people who think IQ is the bee’s knees also think that a huge inventory of attitudes and abilities and potential is hard-coded into us. Their idea of humanity is inflexible and the opposite of general purpose.
-
Yeah, why? Why would we want a computer that can fall in love, get angry, crave chocolate donuts, have hobbies? We’d have to intentionally shape the computer mind to have similar predilections to the minds of apes with sloppy chemistry. This might be an interesting but entirely non-trivial exercise for computer scientists, but how are you going to get it to pay for itself?
-
One species on earth has human-like intelligence, and it took 4 billion years (or 500 million, if you’d rather start the clock at the emergence of complex multicellular life) of evolution to get here. Even in our lineage the increase hasn’t been linear, but in short, infrequent steps. Either intelligence beyond a certain point confers no particular advantage, or increasing intelligence is more difficult and has a lot of tradeoffs.
-
Ah, the ideal of the Vulcan Spock. A lot of people — including a painfully large fraction of the atheist population — have this idea that the best role model is someone emotionless and robot-like, with a calculator-like intelligence. If only we could all weigh all the variables, we’d all come up with the same answer, because values and emotions are never part of the equation.
It’s a longish article at 5,000 words, but in comparison to that 40,000 word abomination on AI from WaitButWhy it’s a reasonable read and most importantly and in contrast, it’s actually right.
Regarding IQ, is it fair to say that it still provides a useful comparison of individuals within the same culture?
As to making human-like computers; what would be the point? We make “units” capable of thinking like humans by the billion already. Maybe money spent on such a project would be better spent on maximising the potential of those biologically-produced units.
Lately I’ve been working on a project that involves AI-like properties – namely, supervised machine learning expert systems with the ability to propose possible new rules to the expert supervisor. In researching it, I’ve come to realize that many of the people who are interested in machine learning seem to forget the part where you need an expert to train it, or you wind up with an artificial racist troll, like Microsoft’s chatbot. So, if you have a machine learning system that is trained by Socrates you’re going to get very different results than if you have a machine learning system that is trained by random people on Reddit. It’s funny to me that some people point to the success of machine chess-playing software and don’t acknowledge that the machine was trained by a collective of chess masters (with input from historical masters’ games)
I spent several months researching this, and doing some test implementations of some typical machine learning algorithms against log data, and I’ve concluded that humans aren’t very smart, either. Humans are expert systems, too – massively parallel meat robots – that are incredibly dependent on the expert that trains them. Humans take 30 years or so of constant feedback training from experts to be moderately effective. I’m not sure what it’d take to train an AI to be a great general to wipe out humanity, but you’d need an expert capable of deriving such strategies to teach the AI to figure out how to do it.
Daz: Uffish, yet slightly frabjous@#1:
Regarding IQ, is it fair to say that it still provides a useful comparison of individuals within the same culture?
It compares their performance on an IQ test, sure.
As to making human-like computers; what would be the point?
The idea would be to make a human-like computer that was able to train itself to be better than a human at doing certain tasks. That would be valuable, if such a thing could be made. I think it might be possible, but to do it you’d need to simulate an environment that was as complicated as reality, then train it with rules that led it to interact in the desired manner with that reality. Otherwise, you’d wind up with a 2-year-old that might, 90 years later, be interesting. Or, it might be a horrible smug old fart, if it read too much Sam Harris.
I really blew my explanation of a point in #3: the reason you’d want to simulate complex realities to train your AI in, is so that the AI would be able to learn how to handle something more than humans can. We’re already pretty good at handling our reality, because we evolved in it and learned to deal with it. If you wanted an AI to grow up to be a great human-eradicating general, you’d want to train it in a reality simulation where that was a desirable and plausible goal, because if you trained it in this reality it’d be killed by the humans while it was still learning how to be a general. The AI ‘Hitler’ in the 1940s is an example of how that learning process can get sidetracked by intangible details.
Hmm. But if it can handle something more than humans can, it’s not human-like? Or am I missing “jargon” meanings that differ from general usage?
PZ:
Because humans have a perverse need to make everything in their own image – look at all the gods we’ve created and discarded. Why not do the same to servants and slaves?
Daz: Uffish, yet slightly frabjous@#5:
But if it can handle something more than humans can, it’s not human-like? Or am I missing “jargon” meanings that differ from general usage?
I don’t think I’m jargonning you, and I think we’re probably on the same page about human-like. The article PZ references seems to me to be spot on the money: people think that an AI will be human-like, but more so. So, if you can teach a human how to play piano, an AI will be able to play piano, except it’ll somehow be better than Mozart because “Mozart was only human.” I take by “human-like” you mean that we might have an AI that plays piano as well as I do. Which AI you really do not want to hear unless you like ‘chopsticks’ in the wrong key.
Perhaps this is a way of thinking about it: human-like things do stuff like create (within domains) and communicate and get pretty good at a few tasks and so on. I think that by “human-like” we mean that the sets of tasks the AI would do are the sort of tasks a human would do. Now, the next question is “does it do them better?” and “better” is vague: faster? more skillful at that particular domain? (What if I made a chess-playing robot that was “better” than a human because my criterion for “better” was that it makes its moves in microseconds! so fast! It doesn’t make legal moves, of course, it just shoves pawns around. so fast!) If your question was about making an AI that does the same things that humans do, with the same skills that humans do those things, at the same speed that humans do them, then yeah it’s interchangeable with a human unless it’s going to be pretty expensive.
Marcus Ranum #7:
Thanks Marcus. And I agree; how does one define “better,” when we can’t even define what intelligence actually is?
If you ever hear me sing (I use the term loosely), you will agree that human-like AI is, in this limited respect, already with us.
Kelly is founding executive editor of Wired magazine, so presumably he’s had to read through a lot of breathless fetishizing etc. of AI over the years, and seen several cycles of “We’ll have human level AI within the next 10 years.”
Something worth remembering: AI intelligence is fragile. When it fails, it fails abruptly, whereas human intelligence fails less catastrophically.
For this reason, AIs will be more a tool than a replacement for human intelligence. The best illustration of this was a chess tournament held a few years ago, where there were three types of teams: grandmasters, the best chess AIs, and excellent chess players with lesser AIs that the players using them understood. The latter did the best — the combination of an AI and the good chess player who understood how the AI worked. They did best because those players understood the limits of the AI helping them, and knew where its blind spots/failure modes were.
AI overlords aren’t in our future. Human overlords augmented by AIs? A lot more likely.
Another way of thinking about it might be: is Google translate a better translator than a human? Well, it’s a whole lot faster, it’s always there when you want it and never complains, and it can translate a lot of languages – I imagine there is no single human translator that can translate Latin to Finnish to Arabic – especially not in milliseconds. So in those respects, it’s a better translator than a human. Would you want your best poetry translated by it?
In some ways an AI is already superior to a human, while being inferior at the same time. It makes it hard for me to sensibly talk about “intelligence” let alone “better” in that sense. If we say that “better” means “more capturing the beauty of the underlying text in translating Lao Tze to English” I bet a human will be “better” than an AI for a very very long time. If we say that “better” means “cheaper and faster” we’ve had AI that are “better” than humans for a long time.
Put differently, Sam Harris is clearly highly intelligent, so why does he say such stupid shit?
We have the same problem with figuring out our criteria before we can talk about intelligence or superiority, and when I see someone try to jump to talking about ‘intelligence’ without laying out their criteria for what comprises intelligence, I assume I’m going to need both palms free for my face, and a clear spot on my desk to pound my head on.
My bugbear is killer AI. I was watching the new Dr. Who episode last week called ‘Smile’. SPOILERS
It bugged the crap out of me that they didn’t build a robots that had the “Don’t kill people” command hard wired into them. These were domestic robots for crying out loud.
I’m generally with you on the notion that most speculation about potential AI is baseless, but I don’t find any of the above remotely compelling.
1. So intelligence is not one dimensional? So what? So long as none of those dimensions are mutually exclusive, which seems implausible if not impossible, anything which is smarter than you in every dimension of intelligence is, by any reasonable standard, smarter than you. Regardless of how many ways you want to slice it, some people are just smarter by every conceivable measure. Not many, because most people are around the middle of the curve on every measure, but sometimes you do roll a Yahtzee.
2. I’m not sure general purpose means what you think it means. Certainly, human minds were ‘designed’ (forgive the phrase) for survival in a particular environment, but it seems to me that any intelligence evolved on the plains of Africa which is adept at surviving in a modern urban environment is pretty damned general purpose. In fact, what I believe is meant by the phrase is ‘capable of completing any task we can think to give it’, which is a reasonable expectation of human intelligence in the abstract. I’ll concede that we probably can’t conceive of the problems we can’t (in principle) solve, but we’re presumably talking about machines made to solve the problems we give them. Is it not reasonable to suppose that a machine created to solve any problem given to it by a human will necessarily need to think at least analogously to a human?
3. This is actually the most interesting one. In some sense it must be possible to artificially reproduce the function of a brain, since anything that can’t be reproduced in silicon can be simulated by something which can, but it seems like the overheads involved in simulating a few trillion? (google says more than 10^26(holy shit)) subatomic particles is probably prohibitive. It may actually be impossible to make anything like a brain which is not actually a brain.
4. There is presumably an upper limit to everything, within a physical universe, but so long as it’s sufficiently far away I don’t think it’s much of an objection. An AI need not be infinitely intelligent to be significantly smarter than us. If the metric is the ability to solve any given problem faster and more effectively than any human, it’s not a particularly high bar to reach, on the grand scale. To believe that humans are the smartest thing there can be seems both arrogant and unlikely, in which case it follows that a smarter thing can be brought into existence. Whether it then immediately becomes a being of godlike omniscience is hard to guess at without anything to base the guess on.
5. I would generally define intelligence as the ability to solve problems. If that is the case, then a sufficiently advanced intelligence can indeed solve any problem. At the upper end you probably run into problems that cannot be solved by any intelligence which can physically exist, but ‘solve the problems of a bunch of hairless apes’ is, I would suggest, not anywhere near that limit. If we assume that we can solve the problems we currently face, then an intelligence which thinks somewhat like us, only better, can surely do the same, only better. If we can’t solve those problems then we’re all screwed anyway.
Is it AI research with a view to the creation of a general purpose AI which is better than us at fixing our shit that you have a problem with, or just the breathless anticipation of a sci fi utopia which is supposedly just around the corner? I certainly agree with you about the latter bullshit, but I think there’re probably substantive conversations to be had about resource allocation for the former.
I agree the main question is: “Why?”
I reject arguments that mechanisms made out of silicon as opposed to meat can’t be conscious or intelligent out of hand as rank dualism. Searle’s Chinese Room argument was the stupidest thing I’ve ever seen written down in black and white until I hit this “Boltzmann brain” crapola lately.
I also totally reject the idea that we’re smart enough to design such an intelligence from the top down, 1950s-style. Some kind of neural network/machine learning arrangement could work, but not without being out and about and dealing with the real world (simulated or not). I agree it might not take 500 million years this time, but it would take a good long while–centuries for sure.
To me the real ethical dilemma would be shutting off iterations that weren’t quite as smart as you wanted….
PZ:
Maybe I don’t understand the point of this particular argument. Do you want to claim that’s not part of any conceivable equation? Presumably not, since as a naturalist, you’re claiming such phenomena are reducible to physics — not that we know exactly how to do such a reduction of course (not yet and maybe never) but that this is how the world itself is.
So, somebody may propose making a not so “Vulcan” sort of AI, which doesn’t much resemble a calculator, one which is more like a human, with emotions and so forth. That is obviously the sort of position you’ve been complaining about, and as you’ve pointed out, there are plenty of reasons to think that’s not at all practical for us. But for some, that is nevertheless their stated goal or “ideal” so to speak. So this kind of objection can’t be used against such people, since that’s not even a conceptual problem for them (whereas somebody else would be very confused if they thought making a calculator-like object would essentially be equivalent to making a human brain).
That is, not unless you have some genuinely good reason to believe any conceivable AI must be like your PC, a Vulcan, a stereotypical “robot” from a 50’s sci-fi movie (or most sci-fi for that matter), or whatever it is you’re imagining instead of the type of thing they’re actually proposing. As I said, this sort of argument presumably won’t work, because we’re physical objects (like any AI would be), which is enough for us to have all of the sophisticated features you care to name, including values and emotions and so forth (however neglected or underrated they may be by certain people in certain situations).
And AIs can just as well have whatever complicated set of features people are gesturing at when they talk about “intelligence,” which as you’ve argued isn’t appropriately measured by so-called IQ tests. If you even wanted to go so far as to say that “intelligence” just plain doesn’t exist — however that should be understood, though I’ve never heard anyone make this claim and don’t see why they’d want to — then the goal is still roughly to make an “artificial person” with whatever interesting features a person actually does possess, including the things we were incorrectly characterizing as “intelligence.”
So, the objections seem to entirely an issue of practicality, about whether or not we’re going to do it, or about whether it’s likely to happen in the near future. Which is fine, there are certainly serious questions about that, and it’s good to take stock of where that leaves us. There’s apparently no principled reason to believe that it can’t physically be done (including, for example, the idea that brain tissue has some secret special sauce that other physical systems can’t have). And there’s no obvious reason to think proponents of it must somehow be confused about what it is that they think could be done. I’m sure some of them are confused about lots of stuff, of course; but that isn’t by virtue of being an AI proponent due to the idea itself being somehow wrong-headed or contradictory or what have you.
The really funny thing, to me, is that the same people who will defend tooth and nail the accuracy of IQ as a measurement of a person’s overall intelligence and potential (because minorities they don’t like average lower IQ scores) will also dismiss psychology and sociology as SJW elitist loser college professor pseudoscience when it comes to explaining why those scores differ.
So the field is apparently advanced enough that we can routinely administer a test that will filter out all cultural, social and environmental influences on something as complex as “intelligence”, while simultaneously being completely wrong about how those cultural, social and environmental influences work.
I think any discussion about “why would we want to do this” ought to start with the premise that we’re talking about something which is genuinely a person in all relevant respects. If the idea is “we can make it our slave, so as to solve chess, write better music than us, and so forth” then that is a truly awful idea. We should not do anything like that. The conversation ought to be more like two people who want to have a baby. We’ve got a responsibility to take care of this thing and respect it and so forth, if one ever gets made. If you can’t morally “use” a person for some purpose, because it violates their rights, autonomy, etc., then you shouldn’t use a human-equivalent AI for that either.
So…. what’s supposed to be the point of making a baby? Why babies? Is there a scientific or technological purpose to that?? I couldn’t really say, I guess — I’m not in the baby-making business — but presumably that isn’t why it’s done.
The Very Reverend Battleaxe of Knowledge:
Well I don’t what crapola you’re talking about. You can’t coherently believe you’re a Boltzmann brain, if that’s the sort of thing you’re worried about. Searle’s argument was simply a load of crap, pulled directly out of his ass…. very much unlike statistical mechanics, which is what motivates the discussion of Boltzmann brains.
Typically the importance one attaches to IQ comes, miraculously, after the score. As for chess, I see little wrong with saying computers are smarter at chess than humans, although that would be speaking American. Coached or not in development, they are now objectively superior to the greatest humans in history. I don’t see a cost problem here, or in the replacement of humans in any similar field. Twas ever since the caveman by his wheel – creation its creator fast outran. Not true AI they say, but better than human ‘genuine’ intelligence anyway. How narrow is this field then? Non-AI better than human-GI? Perhaps not as narrow as accountants are hoping. But to make a computer dream…I suspect the wrong archetype is on the case.
The motivation for human emulation is mostly the quest for immortality, which even the most ardent evopsych sceptic may have noticed is quite popular – even mentioned long ago in the three best-sellers God saw fit to telepathically dictate.
@ consciousness razor:
I was calling the Boltzmann brain argument stupid because its essence is that a conscious, thinking entity randomly coalescing out of the vacuum is more likely than having it as the result of evolution on a planet, when we know it’s happened once that way, and that there are umpty mega-kajillion googolplex stars and planets between likely instances of Boltzmann brains–it’s a mathematician’s argument about biology.
At the time, Searle’s argument mainly offended me by its ignorance of how language works, but you’re right–it’s idiotic on so many levels that it takes the prize.
Then there’s Roger Penrose’s “We need Quantum gravity to understand how microtubules in the brain work”. Difference is, I have a great deal of respect for Penrose….
@ Zeppelin – usually they point to their team’s claimed average, and deeply suspect this makes them, as an individual, inevitably superior. Has anyone worked out the average supremacist IQ? I think a guess is probably good enough.
No, it’s a statistical mechanic’s argument about statistical mechanics. A Boltzmann brain doesn’t come to exist by means of biological processes, since it fluctuates from thermal equilibrium. We obviously have no observations of this occurring, but they have not done the math incorrectly. You seem to be saying, for anthropic reasons, we know the probability of Boltzmann brains must be lower than indicated; but that doesn’t make any sense, since our knowledge of our own situation says nothing about the probability of anything fluctuating from equilibrium. Because we know very well that we didn’t appear that way. Yet the fact remains that something else could, given sufficiently crazy physical conditions that must certainly obtain over an infinite period of time.
I’m not sure what mean by “the Boltzmann brain argument” (there isn’t really an argument to be had here), but to repeat: there’s no claim that this is how you or I or Earth or the Milky Way actually appeared, since that was a long process tracing all the way back to an extremely low entropy state in the Big Bang. Everybody understands that. You might ask yourself why it was so extremely low. Anyway, Boltzmann brains have their probabilities as determined by statistical mechanics, which may be a lot higher than you’d like to admit, but your knowledge of us and our history (including planet formation, evolution, etc.) plays no role here, because we’re talking about a very different thing with a very different sort of history that simply isn’t captured by the probabilities for any of those processes.
The only context I’ve seen the Boltzmann brain idea in is as an argument that the evolution of complex life, consciousness, and intelligence is so spectacularly unlikely that Boltzmann brains are more likely examples thereof than the evolution of such through natural processes–it’s the “Tornado in a junkyard” argument on steroids.
“The whole concept of IQ is a crime against humanity”.
Hyperbole, much? Racists? Given IQ has proven to be a solid predictor of success in a variety of ways, I don’t see how such an attitude is justifiable. Do we burn the books Catholic style?
Re: 2. I don’t see how you can object to the premise that “abilities” and “potential” is hard-coded into us. Would you object to this in areas such as height, and athleticism.
What makes 5 especially bizarre to me is that nearly all of our problems are social rather than technical. E.g., the scientific understanding of climate change provides us with some pretty clear guidance on what human societies should be doing. The fact that people who want us not to do those things have huge amounts of political influence at present cannot be solved by any reasonable “super intelligence”. We already know what we need to know on the subject.
Furthermore, where our scientific understanding is lacking, most of the time that’s not due to a lack of sufficient thinking, but rather because we just don’t have the data. Constructing a “super intelligence” won’t help at all with this problem of data collection: data collection requires funding. Maybe such an artificial intelligence can help at the margin by improving efficiency (e.g. by coming up with more efficient machines or algorithms). But this is the kind of thing that AI is doing right now.
b1rdofprey@#23:
Given IQ has proven to be a solid predictor of success in a variety of ways
The number of ways is irrelevant, if it predicts success in a lot of useless ways.
Mostly it predicts success at IQ tests.
Secondly it may predict that you’re from a particular culture or have been educated – which may predict that you have $$ to afford a good education, or not.
As soon as you get into “it may predict stuff, but you don’t really know what it predicts” its usefulness as anything but an ego ornament is over.
Using height to predict success in certain things is reasonable, but it’s not a universal predictor of success. 6’3″ people make good basketball players (regardless of race or gender) but suck at being fighter pilots, for one example.
If the end result of a test is a whole lot of “… it depends” then it’s not much of a test and it’s not much of a predictor.
The Very Reverend Battleaxe of Knowledge, #22:
Well, okay… The creationist nonsense you have in mind boils down to a design argument, but the real thing is not like that. It comes from perfectly respectable math and physics, and it’s certainly an odd thing that some are understandably trying to deal with in serious physics and cosmology.
An interesting related question is why entropy was so incredibly low in the past. Would you even consider potential physical explanations of that, or are you simply going to assert that the probability of that is 1 because it’s what we observed and leave it at that? The latter seems a bit … suspicious, I guess, and a bit lazy. Why don’t we get to do this all of the time for all sorts of shit? Just say it’s 1, without doing any actual calculations and without meaning very much of anything by it, so you can move on to something you really care about. It’s not obvious at all that that’s the best we can do here.
Meanwhile, people also go around saying evolution straight-up violates the 2nd law of thermodynamics. It looks like they want entropy to one way or another fuck some shit up and prove Jesus for them — not sure why that looks like a good candidate for the job, but it’s already clear they have no idea what the fuck they’re doing. Anyway, your conclusion better not be that the 2nd law is therefore wrong or that evolutionary theory is therefore wrong, because some creationist said some confusing things to you once.
IMO, the only real danger of AI is that AI-driven automation will make many, if not damn near all, jobs redundant, and society will remain wed to the idea that you have to work hard in order to earn the privilege of continuing to live. There aren’t many jobs, even the so-called “creatives'” jobs, that can’t be done effectively by a well trained computer, but it’s usually just too expensive today, but those prices will go down and it’s already happening.
If the world is destroyed by intelligent robots, I’d bet the root cause will be human greed, avarice and shortsightedness rather than an AI that “turned evil”. That’s already the cause of the global climate change catastrophe that we’re currently still ignoring, so plenty of precedent there.
ck, the Irate Lump@#27:
If the world is destroyed by intelligent robots, I’d bet the root cause will be human greed, avarice and shortsightedness rather than an AI that “turned evil”.
It’s always seemed to me that’s a bigger threat. For example, imagine a “war minion” bot that was a great killer and a good strategist, capable of following orders, but with limited creativity except for battlefield tactics. Then imagine a “global dominator” bot – it would have to be a superset of the “war minion” bot because, in addition to doing what the “war minion” would have to do, it would have to be creative enough to come up with an idea like “hey let’s kill all the humans, and I can run this place!” So we’d be more likely to build a “war minion” which means that the first human with a “war minion” fleet gets to run the world and if humans are eradicated it’s done when the first human with the “war minion” has a bad day. After the slaughter is over, the “war minion” bots simply … sit there. In fact I should pre-emptively build a “war minion” because if someone builds a “global dominator” then I’ll get eradicated. So it makes sense for me to wipe out all humanity well before we develop an AI that’s capable of coming up with that idea.
ck, the Irate Lump #27:
Evil In, Evil Out.
Marcus Ranum #25
Flynn (Flynn effect):
“IQ is also important because some group differences are large and predictive of performance in many domains.
Much evidence indicates that it would be difficult to overcome racial disadvantage if IQ differences could not be
ameliorated. IQ tests help us to track the changes in intelligence of different groups and of entire nations and to
measure the impact of interventions intended to improve intelligence”
Given the Flynn effect show that it’s possible that IQ can be dramatically improved through changing environmental factors, some argue that focus on IQ has value to achieve greater income equality.
So I don’t think you can dismiss a whole area of study quite so flippantly.
Good post overall, but you’re just wrong about IQ, PZ. It is not a “crime against humanity,” and it has not “outlived its usefulness.” Yes, it has shady origins and has been misinterpreted and misapplied to horrific ends, as many technologies have. Take nuclear fision, for instance. Yet IQ measures are extremely important in the fields of mental health and neurology. Ask any psychologist or neurologist; almost all will tell you that IQ tests are critical for accuratley assessing and differential diagnosis of conditions, from dementia to TBI to developmental disorders. IQ is among the most useful indicators of cognitive function that we currently have. Also, IQ is age and education normed, so it is rather robust to education level and usually remains pretty stable over the entire lifetime. Almost always it is given alongside an academic achievement test as well as other cognitive measures, and these data as well as education and other history are taken into account when IQ tests are interpreted. Only licensed psychologists should be interpreting IQ tests, and they have to do so following practice standards. I agree with the criticisms of how racists misinterpret and misapply it. But intelligence testing is reliable, valid, and useful.
b1rd0fprey@#30:
So I don’t think you can dismiss a whole area of study quite so flippantly.
I wasn’t being flippant; I pointed out a couple of serious problems with what you said.
The test questions have changed since the original IQ tests, so testers now have the problem of demonstrating that the new test is measuring the same underlying property as the old test, but the new results are a consequence of the property changing – rather than maybe that the tests are getting easier or there is a cultural bias in the testing. There is a tremendous amount of wrong in the Flynn Effect, starting with the prominent assumption that IQ measures something called “intelligence” – we don’t know that until we have a definition of “intelligence” and solid evidence that the test isn’t measuring something else. Then, and only then, can we even start to think about the question of whether the test scores (of different tests) going up or down means anything. Good luck with that, given that we’re not even talking about the same tests, or the same culture, and we’re probably not even talking about the same samples (so there might be different bias)
Flippant would be me saying, “what a load of bullshit, you must be clueless as a bag of cinders.”
Which, by the way, I now say.
PS- Here’s a scary thought: maybe they are measuring more people of color and that’s why the test scores are going up.
PPS- Bias matters.
Only licensed psychologists should be interpreting IQ tests, and they have to do so following practice standards
Disclaimer: I have a BA in psychology from an ivy-covered university in Baltimore.
This is all backwards. Results are not interpreted. When a scientist is measuring something, the measuring tool ought to measure what it purports to measure accurately enough that there is no room for or need for “interpretation” — one does not “interpret” a yard-stick, nor does one “interpret” an IQ test.
Secondly, anyone with an education in psychology ought to have taken enough course hours on testing methods, and enough course hours of statistics (I had 3 semesters of each) that they’d have enough understanding of the problems in IQ testing that they wouldn’t take it seriously at all. Unless they’re handing out degrees in “pop psychology” nowadays, that is.
Ask any psychologist or neurologist; almost all will tell you that IQ tests are critical for accuratley assessing and differential diagnosis of conditions, from dementia to TBI to developmental disorders
I am not a practicing psychologist, but I have some qualifications and I am telling you that IQ tests are critical for assessing your ability to score on IQ tests. Anything else is hard to say. For example, IQ tests appear to correlate with wealth. (depending on which test you’re talking about, and which biased sample it’s applied against) and I don’t mean that in the sense of “people who score high on IQ tests tend to get wealthy” but rather in the sense that “kids that grew up in wealthier families scored better on IQ tests.” Now, there’s the question: was it nutrition, protection from environmental toxins like lead, better education, private tutoring, better access to books – there could be a lot of factors – and since none of those factors are captured in the information about the sample, it’s impossible to say anything about its impact on the test.
Marcus Ranum #32
It’s s shame your comment appears directly below such an objective and well reasoned comment as skepticalmat #31, making your babyish remarks all the more obvious.
Given your attachment to PZ folk logic that “IQ Test – BAD”, & “IQ Test – Crime against humanity” (never mind sarin gas on children), you’ll refuse to even read the following article highlighting the issues your raise, the difficulties in using IQ results, and the substantial progress that’s been made in removing cultural bias from IQ tests and their ultimate usefulness in many respects.
Because of course, the APA, Amercian Psychological Association, wouldn’t know. You’d rather trust a discredited and universally reviled atheist blogger who knows nothing about IQ. Who just calls people scum and asshole. If you want to be a troll just admit that you’re uninterested in the facts. You really just want to call people names and the slink back into the shadow of your own ignorance.
http://www.apa.org/monitor/feb03/intelligent.aspx
“Critics of intelligence testing often fail to consider that most of the alternatives are even more prone to problems of fairness and validity than the measures that are currently used, says APA President-elect Diane F. Halpern, PhD, of Claremont McKenna College.
“We will always need some way of making intelligent decisions about people,” says Halpern. “We’re not all the same; we have different skills and abilities. What’s wrong is thinking of intelligence as a fixed, innate ability, instead of something that develops in a context.””
@ Daz, 1
Perhaps, but maximizing potential without profit is not a venture that interests people looking not just to count their units but own them outright; instead, they’d start at the top to control the means of production and then set it loose at something narrow and specific. Communities, groups, couples, and individual people exercising full bodily autonomy when making or choosing not to make children cannot be controlled, hence, erm, patriarchy and, when applied by folk with capital to spare, slavery.
Things are a bit less clear-cut in quantum mechanics, as you may know, but that’s because the word “measurement” has been so widely misused in that context.
The “measurement of a person’s IQ” is supposed to reveal what the person was like prior to “measurement.” It’s supposed to tell you how intelligent they were, like a yardstick tells you how long something is. Yardsticks don’t force something to become that long — it already was that long, and that fact isn’t significantly changed by your use of the yardstick on this measurably-long thing. It seems incredibly simple and hardly worth noting, but things can go off the rails very badly and very surprisingly when that isn’t abundantly clear to everybody.
Anyway, these tests are not supposed to tell you how smart you made them as a result of your intervention. Then again, people tend to get “more intelligent” (score higher on the test) after repeated interventions…. So if that is what they’re doing, then for fuck’s sake don’t call that a “measurement” of anything. You may as well throw darts at a board and say you’re “measuring” the locations of your darts — no, you’re throwing them, as we would say in ordinary English, which is an altogether different activity, so we need to reason about such activities differently. The darts had a location before you threw them whether you know it or not; and wherever they may happen to land, that sort of procedure is clearly not how you’d go about measuring a property like that. Sometimes, that’s the best you can do, but that’s all it is — there’s no reason to get yourself into a lather about it or to describe it using completely inappropriate language anyway.
b1rd0fprey,
Did you actually read the APA article you linked to? The entire thing is about psychologists developing alternatives to IQ testing, that show less cultural bias and allow for a more nuanced and complex description of cognitive abilities. Yes, they’re still measuring various aspects of “intelligence.” No, they’re not boiling it all down to a single value and claiming that value accurately represents a person’s generalized smartness.
That sober essay is most refreshing in a most zesty way – like opening up the windows to let some fresh air circulate through the subject room to banish the unimaginative stale and stink of the techie-whoopee contingent. Good stuff at last.
consciousness razor@#36 –
I used “measure” instead of (what, maybe “quantify”?) some other word, in reference to SJ Gould’s use of it in The Mismeasure of Man not in reference to quantum mechanics. The word “measure” is used differently by woodworkers, psychologists, physicists, and tailors. So, I understand it if my comment wouldn’t make sense to a physicist, but then I wouldn’t expect a physicist to use an imprecise tool like a yardstick, anyway. (Mentally picturing the LIGO, which needs better than a yardstick for its measurements).
The “measurement of a person’s IQ” is supposed to reveal what the person was like prior to “measurement.” It’s supposed to tell you how intelligent they were, like a yardstick tells you how long something is.
Agreed about the measurement before and after and the potential change in measurement. But we’re talking psychology (or, I’d argue “pop psychology”) not physics. Though your comment suggests an experiment that would probably vex IQ proponents: if a group of subjects took IQ tests (2 different ones) in a row, would the second run tend to score better? If it did, we’d have demonstrated that IQ tests are affected by learning not entirely by an innate capacity in the subject, which would raise the question of to what degree IQ tests were measuring something innate or whether learning was taking place.
The discussion about IQ keeps swirling around without a definition of “intelligence” that we can measure – that’s problematic because, hypothetically, your definition of intelligence might shrug off a bit of learning, whereas someone who was concerned with proving there was an innate property (such as, say, one might have because it’s genetic…) would be wanting to measure just the genetic component of the result, to amplify their particular point, whereas you might be interested in probability of certain social outcomes.
Anyway, these tests are not supposed to tell you how smart you made them as a result of your intervention. Then again, people tend to get “more intelligent” (score higher on the test) after repeated interventions…. So if that is what they’re doing, then for fuck’s sake don’t call that a “measurement” of anything. You may as well throw darts at a board and say you’re “measuring” the locations of your darts — no, you’re throwing them, as we would say in ordinary English, which is an altogether different activity, so we need to reason about such activities differently.
I see what you’re saying but I don’t agree with your analogy. This would be more like saying that your ability to throw darts into the center ring was a measurement of how coordinated you are. It would certainly measure your dart-throwing ability, but it still leaves the problem of “what the hell is ‘coordinated’ mean?” and we can certainly agree that your dart-throwing score predicted your future capability to throw darts well.
I don’t think it’s reasonable to try to enforce the physicists’ definition of “measurement” onto the dart player, or me, or a woodworker, tailor, sailmaker or a pop psychologist. I hate to play dictionary but google returns:
I didn’t originally use the term “measurement” in the sense of making an observation of quantum states – the fact that I mentioned a yardstick ought to have been a clue.
Anyway, these tests are not supposed to tell you how smart you made them as a result of your intervention.
My point is that since we don’t have a unit of measurement for “intelligence” (i.e.: we don’t know what “intelligence” is) these tests can’t measure anything except performance on the test. Unless you want a circular definition, such as that “intelligence is what is measured by IQ tests” and then we’re really stuck.
there’s no reason to get yourself into a lather about it or to describe it using completely inappropriate language anyway.
My language was not inappropriate. There’s a long history of using the word “measurement” in psychometry, psychology, and pop psychology. Not to mention tailoring, architecture, and woodworking. There is no point I am aware of when other fields of human endeavor signed away their rights to use the word, and wholly reserved it for the physicists. (and I thought physicists preferred “observation” for collapsing quantum superpositions)
Since I apparently wasn’t clear, I’m going to elaborate a bit more. When I wrote:
one does not “interpret” a yard-stick, nor does one “interpret” an IQ test.
I was thinking about one of the big problems in psychology right now, which is that there’s a great deal of “interpreting” going on and it’s why the field has a replication crisis. That’s one piece of it (and I didn’t say anything about it in my comment; that’s just what I was thinking outloud to myself as I wrote…) but, the larger point is that if IQ tests actually did measure “intelligence” – whatever that is – there’s nothing to interpret other than “the test says you’re smart” or maybe “the test says you have the same IQ as my cat.” There’s comparative framing you can put around such results and that’s interpretive but the results are what they are.
Now, let’s imagine a case where “interpret” fits with regard to the results of an IQ test: suppose I took an IQ test last year and this year I suffer an NDE and a doctor gives me another IQ test and I score something very different. One might “interpret” that as that I suffered some kind of damage in my NDE. Or one might argue about the interpretation – maybe it’s a sign of alzheimer’s, or maybe I just drink too much. There, interpretation makes sense, and a comparative analysis might make sense – but I’m skeptical even of that, because there are too many factors that might affect my measurement: I might still be on painkillers (oops!) and the IQ test actually measured something about how I perform on IQ tests when I’m on opiates versus when I’m not.
Let me drop a few historical bits, too. Binet, when he first set out to measure “intelligence” – literally did it with a tape-measure. His first experiment (though he appears to me to have lept to that conclusion without any supporting theoretical framework) was to try to measure cranial volume to see how smart people were. I don’t recall him ever offering a definition of “intelligence” either, maybe it was like “porn” and he just knew it when he saw it (i.e.: if you were well-spoken and French you were “intelligent” but if you were a horrible French-mangler from Alsace with a German accent – you were probably less “intelligent..”) He gave up on craniometry and instead decided to use “psychological” measurements: he broke down a few common tasks into common categories, and ordered them in terms of what he thought was difficulty, then observed subjects’ performance on them and tried to flatten those results down to a score. Here’s the problem I was referring to earlier: he was trying to measure something without knowing what it was, so there was no way he could tell if he was measuring that, or something else. And, as we still see today with IQ tests, his tasks were horribly culture-biassed: counting coins (helps if you’re French!) assessing which faces were prettier (no possibility of racial bias there!) etc.
It was all gross pseudoscience, and it’s a shame psychology hasn’t completely repudiated it. They should teach about IQ in the first semester of psychology the same way – and for the same reason – that a budding physicist might be taught about Blondlot’s N-rays or phlogiston: as a warning.
Anton Mates #37
No, they talk about intelligence testing and IQ tests interchangeably. (Citing Wechsler – which was the classic IQ test).
They are talking about improving IQ tests which is a good thing, and saying “most of the alternatives are even more prone to problems of fairness and validity than the measures that are currently used…”
b1rd0fprey@#34:
making your babyish remarks all the more obvious.
“babyish”? Lolwut? Is that what you consider a refutation? Because, even if my comment had been written by a 12-year-old instead of a 54-year-old that wouldn’t change the truth value of it one iota. If that’s the best you can do, I’m not going to take you seriously and will switch to making fun of you.
Given your attachment to PZ folk logic that “IQ Test – BAD”, & “IQ Test – Crime against humanity”
I have been making my own arguments, not PZ’s. Here’s a hint, if I were leaning on PZ’s arguments my comments might resemble “PZ U R SO RITE!” rather than the kind of arguments against psychology’s epistemology that I’ve been making. Besides, as I mentioned above, I had to study this stuff. That was back in 1981-1985, and I’d formed a thorough set of objections to much of psychology by the time I got my degree. I only heard of PZ Myers back in 2002 or so (back in the scienceblogs days) and most of my opinions on IQ testing were established well before then. That’s probably why I haven’t been referencing PZ’s arguments at all except in passing.
Because of course, the APA, Amercian Psychological Association, wouldn’t know.
The APA exists to promote psychology, they’re unlikely to offer a strong critique of their field while they’re in it. I don’t want to go to a full critique of APA’s many failings, but I’ll mention that you appear to be setting up an ‘argument from authority’ in the form of “the APA says: ${thing the APA says} therefore it’s true.” And you tried to characterize my comments as childish?
You’d rather trust a discredited and universally reviled atheist blogger who knows nothing about IQ.
No, I trust myself, and the knowledge I gained about psychology and testing while I was getting my BA in the field. I’m happy to agree with PZ when I agree with him, disagree with him when I disagree with him. But follow him?
You appear to have started to set up an ‘argument from authority’ regarding APA, and next appear to be accusing me of accepting PZ as an authority. Will you please make up your mind about whether or not you think argument from authority is good?
In the meantime I will continue to argue from knowledge. (Or, actually, since we’re talking about IQ tests and how psychologists don’t even know what they’re measuring, I should say “argue against ignorance”)
You really just want to call people names and the slink back into the shadow of your own ignorance.
You got a brief boggle out of me, there.
“Critics of intelligence testing often fail to consider that most of the alternatives are even more prone to problems of fairness and validity than the measures that are currently used, says APA President-elect Diane F. Halpern, PhD, of Claremont McKenna College.
You got a great big boggle out of me, there!! You quoted one of the dumbest things APA could have said to defend IQ testing, translates as: “Sure a lot of people think IQ testing is bogus, but it’s the best measure we’ve got!” which translates into Marcus as: “Sure a lot of people have blown smoking holes in IQ testing but it’s the best measure we have that we can use to measure that thing that we have no idea what it is. But it is remarkably accurate at measuring how people perform on IQ tests.”
“We will always need some way of making intelligent decisions about people,” says Halpern. “We’re not all the same; we have different skills and abilities. What’s wrong is thinking of intelligence as a fixed, innate ability, instead of something that develops in a context.””
Ways of making decisions about people sounds like a great idea. I support that. I support using science for doing that, which means you need a solid epistemology underlying your decisions.
Note “intelligent decisions” doesn’t say anything about “decisions about intelligence” which is what we’re talking about.
What’s wrong with thinking about intelligence as a fixed, innate quality instead of something that develops in a context? That’s not my problem. It’s not my theory. I’ll tell you one thing that’s wrong: you can’t think about something as fixed and innate if you don’t know what it is. I mean, you’re welcome to act as though it is – but until you can develop a measurement that demonstrates that it’s fixed and innate, it’s just as likely to be clam chowder as far as I am concerned.
And, I skimmed the rest of that piece (which you should have done) and – well – if I was accepting the argument from authority that APA was right, I’d hammer all over you with quotes like this one, from that article:
Translation: oops, critics were right. We are scrambling to salvage a money-maker.
Translation: Of course the IQ tests are still full of bias but we ‘reduced’ it. We think.
Translation: The gap between how the theories are used and the tests is huge – people keep using IQ tests as if they measure something about people’s intelligence, and we keep trying to downplay the fact that they don’t but we’re stuck because – did we mention? – it’s a money-maker.
Translation: We’re working on making IQ testing better and less of a pseudoscience, but the pseudoscientific tests are still widely used. Did we mention – it’s a money-maker? And don’t ask us about Myers Briggs, please, no, don’t ask us about that.
Naglieri’s own test, the CAS, is based on the theories of Soviet neuropsychologist A.R. Luria, as is Kaufman’s K-ABC.
Translation: We have worked desperately hard to catch up to the 1960s
I have to stop here. Pretty much every paragraph of that piece is arguing my point for me (they’re just wording it very very carefully) I think that you really picked the wrong authority to argue from.
You’re going to have to do better than this if you’re going to glare down your nose at me and imply that I’m babyish. Because if I’m babyish you just showed that you’re not even a blastula.
Wechsler – which was the classic IQ test
Wrong. Binet was the classic IQ test. Wechsler tried to fix some of the glaring flaws in Binet.
PS – I am hugely in favor of tests that are used to measure before/after differences in things like strokes, or to measure comparative performance on tasks for people who are suffering from neurodegenerative disorders, or even before/after self-reported surveys to compare PTSD interventions or psychopharmacological interventions. Those sort of tests are interesting because they normalize (mostly) your results of your present self against your past self, and that ought to filter out a lot of cultural biasses. The things psychologists have learned from the desperate fight to repair IQ tests have helped them improve their methods so that some of those tests look really promising.
#41
What a laugh.
Translation? Seriously?
First, you failed to understand the article, but then compounded your error by accusing me of failing to understand it. Now you presume to translate the clear and unequivocal statements of the APA (accusing them of being “dumb”) – which you didn’t understand in the first place.
No thanks. Slink back into the shadows, baby.
b1rd0fprey #40
“The” classic IQ test was Stanford-Binet. Wechsler scales don’t give you one score, they give you several scores in various cognitive domains: three on the original WISC, seven on the WISC-III, up to fifteen on WISC-V. (Note the direction of that trend.) Yes, WISC also gives you a composite “Full Scale IQ” score, but that’s precisely the measure that even the intelligence testing supporters in that article recommend dismissing:
“For them, the problem with the discrepancy model is that it is based on a fundamental misunderstanding of the Wechsler scores, which were never intended to be used to as a single, summed number. ”
A single, summed number–IQ, general intelligence, g factor–is what PZ, Kevin Kelly, and most of the posters in this thread are objecting to. I haven’t seen anyone here argue against quantitatively testing people’s cognitive skills and abilities at all; I’m pretty sure PZ gives his students exam scores for a reason.
Again, all of the improvements they mention involve moving away from a single generalized IQ score, to an array of scores that measure different aspects of cognitive ability, plus additional behavioral observations on the person in question. If you still want to call that an IQ test, you can do so, but it’s not the test that folks like Murray and Harris are defending, and it’s not the test that PZ has criticized.
(I don’t think, anyway. PZ and other posters are welcome to correct me on that.)
Well, that’s quite true. But “better than most of the alternatives” is a low bar; most of the alternatives are terrible.
Ach, it’s quite good, and quite useful, and 50 points down means you’re ill. In the wrong heads it’s terrible. Certainly it’s finite. Now you’re going to have trouble thinking beyond the limit of your thought. But you can see it. Just as you have trouble seeing beyond your visual perception, but you can think it. Plenty would buy a TV beyond human perception to show their neighbours an impressive, superior, number written on it. This sort of idiocy is the way to do futurology. Unless, looking around you, you think the most intelligent humans are going to be running the show. A brave call in America at the moment. The limit of thought looks ilke htis, and beyond looks kethsili.
Spelling tests are similar to IQ tests, even a part of them, but spelling can only be corrected if the correct meaning has been successfully imparted. Wigner thought the grammar of the universe miraculous, but couldn’t read between the lines. If you look at a single word, you can see what we’ve been sentenced to. Reading left to right, or right to left, is a matter of taste, but reading before and after is a matter of reality. ‘Solipsism’, if you look closely, holds both space and time, and if it didn’t, it couldn’t be spelled. If you try and read the universe, you find, after say 500 years, the minimum two sentences, written in incompatible languages. Both work fine, as that is merely the reversal of their deduction. They are net-spelled correctly, but fundamentally spelling mistakes. You can read just a bit beyond this, since every pencil writes in stencil, or rather, you can see the limit of reading, which is better spelled empiricism.
Sorry I’ve forgotten what I was going to say. The next bit was good. Forgetting very important if you’re planning to be eternal. Also, a good spam filter.
I got distracted reading the intro to Under Milk Wood, when I saw it had been translated into 30 languages, and wondered how.
It’s late, I’m still recovering from seven hours of driving to and from a distant city, while a passenger of a vehicle, whose driver left my knuckles white, my shoulder (mild separation there) in agony and three hours of dealing with US government idiocy. All, courtesy of the DHS, as I had to get an ID card from one of their agencies, since they accepted my clearance a year ago.
Oh, I also didn’t sleep the night before, as I had my computer returned from baselining, due to IP stack not communicating with the physical layer, after corporate ordained that I install Virtual PC, Virtual Box would’ve been better. Alas, no contract with Oracle on their free product, Microsoft has a full court pass.
So, 240+ patches for the VM, several blue screens on the VM, one fix of an oddity that prevented a certain VPN product client install to the virtual machine *and* 60+ patches (next meeting of our shop is going to be rather loud, in riot act terms. That’s unacceptable!), I got the damnable laptop in proper working order and secure two hours until my coworker was due to pick me up (my car died, just today, since a couple of deals feel through and my schedule for the past month was utterly impossible, I managed to purchase a cheap $2200 car. At the time, we very nearly ran out of food, but I digress). So, zero sleep for me.
While three hours from home and going through the Stations of the Cross, via a minefield, razor wire and armed smurfs shooting at me, we managed to get our sponsors to wake the hell up, sponsor our ID cards and access, then go back to not answering their FEMA 24/7/365 internal hotline support line.
Then, pass through the *major* thunderstorm that we watched build over the building (OK, area, but work with me on this) and expand.
Once we finally, after being delayed, despite an appointment time schedules with their office, to which we arrived a full three hours early, by onboarding employees, my ID card finally was blessed. Nearly three years after I had access to the program.
We then drove with much of the storm, despite the damnable thing moving at twice our velocity, its vector was consistent for a long drive in stormy weather, with 74 MPH gusts and bands of it overtaking us, once we were satisfied that the main band moved beyond us. A full nine motor vehicle accidence we observed, most being idiots driving too fucking fast and ending up in the median ditch, the remainder, going into some Texan County Car Wash, on the shoulder.
That all offloaded and *not* relating agony in my lumbar spinal region, I’ll go back on topic.
AI, Artificial Intelligence. Frankly, I don’t see any *other* form of intelligence on this planet and the AI type is conspicuously absent.
Google is about the best of it and facial recognition software. That’s not weak tea, it’s homeopathic tea!
Beyond certain, highly specific areas, such as facial recognition software, search engine technologies and text to speech (on the high end, *seriously expensive* package) level, there is nothing.
As for IQ tests, I disagree with PZ, based upon some really technical terms.
IQ, as a gauge of a human, as useful as eugenics, which isn’t even fit to wipe my ass, as I’d be doing so with a bare finger. A bidet would be far preferred.
As a range of skill set specific tests, it can be highly valuable.
An example, a decent encyclopedia software set, when properly queried, would give infinite facts, as fare as humanity’s knowledge is concerned and in one form of IQ test, it’d be a genius. But, utterly incapable of accomplishing any other task. Even “idiot savants” of old and now, more appropriately diagnosed did better.
However, I’m also retired military and recall quite well, my ASVAB test. Which, quite literally, was an IQ test. Testing general knowledge, mathematical knowledge, spacial reasoning knowledge and more. Basically, a functional psyche test, for a lack of a better term available to me in my recovering state. (Odd, as I scored junior college level vocabulary in 7th grade, but it’s late, I’m still not feeling well and I’m growing a bit older.)
*That* sort of test is useful, to some extent. While I was in school, it was rigorously utilized and rejected, universally, due to the rigidity of the usage attempts (odd, considering how often “zero tolerance” is embraced today). That said, it’s a useful guide for advancing, strengthening and adding additional support flagging strengths of a student or individual out of school.
The only test that I’d add would be, for prospects, a test for dyslexia. Something recognized early with me and when stressed or fatigued, still shows up in spades with me (thanks mom, for all of that work!), to address those in need of *proper* training. I’ve recently come upon an entire community of dyslexics, many with phenomenal artistic skills, many of whom now have felony records.
As, that community is disjointed, due to disproportionate laws, adds to their plight.
As for Artificial Intelligence, at the current state of the art, I’d not trust current AI to even wipe my ass, after my morning shit, as shifting to one side or the other might both emasculate me and sever my rectum from my brain.
Despite what some readers here thinks, that’d be a notable occurrence, due to the margin of error of over a meter.*
*Yeah, I have long played the obscenity laden idiot. TBI, yeah, had more than a few. Smoke? Yeah, damned addiction, try to quit, a heat stroke induced issue ensues, v-tach, due to an irritable locus (which my cardiologist agreed with me and ignored my primary physician’s misdiagnosis). Drink too much? Yeah, better than high dosage narcotics. But, I’m approaching worrisome tolerance. Thankfully, now that we have a reliable vehicle, I can get diagnostic imagery and figure out which, of two conditions are a problem.
We already know my wife’s issues, which are dire. Advanced osteoporosis, complete cervical disc degeneration and spinal stenosis, advanced cauda equina syndrome.
So, yeah. A small chunk of our lives, totally OT.
Now, to go to bed. I have a physician’s appointment in 11 hours. I suspect pancreatitis, secondary to gallstones, long after a gallbladder was removed, making things rather, erm, interesting. **
** One skill that has long been wondered at, at many levels, both governmental and various medical associated levels, was my diagnostic skills.
Only once did I miss a diagnosis, due to a similar and also present condition, which had no observable symptoms.
@ wzrd1 – It would be difficult to diagnose dyslexia from that. But not impossible.
To begin after the beginning, 1 = one, so words are fine. If you wanted to spell infinite correctly, you wouldn’t stop typing. That is why it is finite. It may have taken 4 billion years to get to xmph, then much faster faster, but there is a speed limit. Thought short-cuts are as geometry, with multiple calculations inherent. A lot of human work can be rendered obsolete, or rather, cheaper, with such fudging and cheating. Cheaper is cheaper, and all is well if you take over distribution, and remember to be green. Exaptations of medicine will drive surprising ‘advances’ – it doesn’t take much to surprise a human – and a fool’s paradise is as good as a real one, since selfs are virtual anyway. Not dying will remain popular, and any betterlatethanevers won’t be playing. Humans transparently can’t see what they are, as this is how they see. I doubt many psychotics do AI. That seems to me the problem. Programming the illogic. The pre-filter tourettes. Writing Thomas would be a start in the right direction – I doubt he would pass a conventional grammar test, writing like that. But what beautiful mistakes he makes.
This is yet not the real deal. It seems almost unnecessary to say humans can be made within Physics. How biological it must be is fun to guess at the moment.
Marcus, way up there near the top:
“is Google translate a better translator than a human?”
For a fun experiment try typing repeated short strings into Goofle Translate (See Languagelog: More Deep Translation arcana)
An example (Romanian detected):
gives
Marcus Ranum, #39:
I’m not sure if we have much of a substantive disagreement. I wrote “you” several times in my #36; but rather than you personally, Marcus, it was meant as a collective/indefinite “you,” since as I said I think the two of us are mostly in agreement. The “you” was generally addressing psychologists, who should want to avoid mistakes in their methodology and theorizing. Sorry if that made anything unclear. Maybe I should’ve used the “royal we” instead.
The point I was trying to make was not especially about physics, although that is a place where some people had to learn this lesson the hard way in recent history (some I suppose are still struggling with it). It was a point about how it’s prudent to use a potentially misleading English word very carefully, in order to avoid confusion. That’s because people (even smart, knowledgeable experts) may get their logic all mixed up and draw invalid conclusions, even wildly absurd conclusions, if encouraged to take a sloppy or ambiguous usage of a word like “measurement” more seriously than it ought to be taken, or as if the word represented something other than what it’s actually and literally meant to represent.
Okay. You’re right that a yardstick is very crude and imprecise, and you shouldn’t expect modern physicists (Newton would be another story) to go running for their yardsticks when asked to measure lengths, if they had a choice in the matter, since they have better modern equipment for that now.* However, that is nevertheless a good, simple model of what people ordinarily think of as a measurement of something. It is a crystal clear example of a physical device for physically measuring a physical property (length), which hasn’t changed in physics or anywhere else, although indeed, many devices we have now are able to measure that very same thing more precisely.** So, it’s not as if what a yardstick does somehow means something different to physicists or isn’t even an applicable concept in physics anymore, but that we simply have better stuff for doing that now. As you said, LIGO more or less amounts to using a better yardstick,*** which is to say it’s a means to obtain information about the same familiar physical properties as a less precise instrument for those things, as opposed to doing something else that isn’t even like a measurement of those properties.
I don’t see why this should be very specific to the context of physics. To the extent that something isn’t like that, to the extent the process which is being called a “measurement” doesn’t work that way and doesn’t tell us that kind of information about that kind of property, then to that extent we should be ready for people to be confused by that usage of that particular word. That’s all I was really trying to say, and it doesn’t seem like you’d disagree with that.
Fair enough, but yours is problematic as well. The outcomes of these IQ tests depend a great deal on how well the experimenter performs the test, how the questions/activities were designed, and what if anything they’re “interpreted” to say about the subject (as opposed to the experimenter or their testing methods).
There might be some trouble defining exactly what “coordinated” is supposed to mean, and you might very well question whether a dart-throwing score is a good indicator of whatever that is. But beyond that, if your testing methods involved getting people drunk and then throwing those darts, or if they involved insulting the person or disorienting them or in any way making them feel uncoordinated (whatever that means) or less confident, then your test is probably not going to be a very reliable indicator of the thing which is ostensibly being studied. Even as a measure of dart-throwing ability, it might fail spectacularly. Some people respond to such factors differently, and that is independent of whatever “coordination” (or “dart-throwing”) is supposed to be about — our concept of coordination is at least clear enough to make distinctions like that and to separate it from other things like alcohol tolerance, emotional stability, etc. So, something like this wouldn’t tell you much about how the subjects typically perform in real-world situations outside this highly-artificial “testing” environment, where you’ve introduced all of these confounding variables.
In this case of IQ tests and intelligence, it’s not obvious that anybody could conceivably eliminate all sorts of other variables like that, or anyway I’m sure that’s always going to be a major issue for such research, no matter how people might attempt to do it.
* Of course, inches and yards are also not units physicists use, as a matter of convention. So, make it a meter stick if you like, since I guess that’s more sciencey.
** Not infinitely precisely or to any arbitrary finite precision, since we learned you can’t actually do that, just more precisely.
** I don’t understand some of the details, but I think their interferometers were technically more like a time-measuring clock than a distancing-measuring rod; but in any case, distances can be inferred from that with no real problem, if you’re interested in those. We now have better clocks and stopwatches and so forth too, compared to sundials or hourglasses or whatever, so that’s a decent example to use as well.
On that note, in contrast, a paper wall calendar just isn’t the sort of thing that you’d properly say is “measuring” time or anything else. That’s just not what calendars do, that’s not what looking at a calendar does, that’s not what anybody in the business of printing calendars is in the business of doing, and so forth. They’re more like bookkeeping devices or something along those lines. If somebody were talking about the “measurements” they made of time with their calendars (not a “calculation,” as in figuring out the number of days until next Tuesday), it’s right to be very suspicious of that or to think maybe they’re just speaking incorrectly about whatever it is that they’re actually doing. I take it that this is not just an issue with calendars, because it’s a much more general fact about what English-speaking people are typically thinking in the back of their heads, any time they come across that word.
Wow- this sure blew up quick. My post was to point out that PZ’s dismissal of IQ is out of touch with the field of psychology. I am licensed clinical psychologist and a tenure-track professor with a PhD from an APA-accredited program. Not trying to pull rank, just trying to say that I am not some fringe person with a lay knowledge of psychology or IQ. I am an expert. And I am telling you that IQ tests are used in virtually every cognitive evaluation a psychologist does. They are extremely useful. Yes, there are limitations. Every measure has limitations. And yes, it is a measure. The comparisons to yard sticks fail to bring up the issue of measurement error. A yard stick has a set resolution that is capable of reliably measuring things like dressers, but it does not have the resolution to measure the thickness of a piece of paper. Does that mean it is not a measurement? Of course not. We can measure things like working memory capacity (how many individual units of information you can hold in your mind at once). This is extremely stable over the lifespan. Really, this comment thread has descended into something PZ frequently criticizes- armchair skepticism that dismisses the consensus of researchers and other experts based on a superficial level of knowledge.
What is being measured by such a thing, other than how one performs at that specific IQ test?
No, it doesn’t. It’s understood perfectly well that they’re not as precise as other fancier distance-measuring instruments, as Marcus and I were explicitly discussing. If that’s a relevant thing to worry about when conducting a measurement of one sort or another, then the comparison to yardsticks is apt, because it very clearly brings up such issues and doesn’t neglect them.
Do you think the problem with IQ tests are their imprecision or “measurement error”? What is being measured by such a thing, however precisely, other than how one performs at that specific IQ test?
Sure it does. You stack up paper sheets of the same thickness, until it is 1/16th of an inch (or an 1/8th or however far apart the marks on your stick are). Then you count the sheets in the stack and divide by that number, because everybody is allowed to use math whenever they like. That gives you an approximate answer about the things being measured with the yardstick. There are better ways to do it to get better approximations, but that is any case what you can do with yardsticks, since there’s no sensible reason to actually and literally believe the thickness of paper (a length) is a fundamentally different thing which simply can’t be measured appropriately with yardsticks (which measure lengths).
Of course not. I don’t see any of the critics here saying IQ tests are merely imprecise ways of measuring intelligence. Instead, the criticism is more that it’s misguided to think that you’re actually measuring intelligence with them, whatever it is that you may or may not be measuring with them. They may be useful in psychological evaluations and such (or at least they are widely used). But the issue is still how seriously we ought to take them as being a genuine measure of intelligence, instead of some other potentially-useful or interesting constellation of attributes that a person may exhibit, which for whatever reasons psychologists/etc. may want to study.
Okay. I’m a musician. I’ve known hundreds, maybe thousands, of musicians who have great memory capacities. I have no idea how stable that may be for individuals over their lifespan, or to what extent that can be developed as a skill over time, with the right kind of practice or the right environment or teachers or who knows what. In my limited experience, it’s not so “extremely” stable as you seem to suggest, but of course I’ll take valid psychological research about it seriously, if presented with anything like that.
Anyway, they’re definitely not all intelligent, and that doesn’t in any meaningful sense suggest that they are, because that’s simply not what people are talking about when they’re talking about intelligence. Some of these people have been astonishingly fucking stupid, to be perfectly frank, even though they’re quite skilled as musicians, probably in part due to factors like that, since all sorts of memory play a very important role. So, I don’t consider that equivalent to intelligence, for a variety of good reasons I would say, because there’s a whole lot more to intelligence than that.
#32 Marcus Ranum said (in quotes)
“Results are not interpreted. When a scientist is measuring something, the measuring tool ought to measure what it purports to measure accurately enough that there is no room for or need for “interpretation” — one does not “interpret” a yard-stick, nor does one “interpret” an IQ test.”
Marcus, what are you talking about? Every scientist interprets data. Recognizing that interpretation is happening is the first step to recognizing that we can look at data and make mistakes due to bias, etc. When a psychologist performs a cognitive assessment, they typically administer a battery of tests, including an IQ test, along with a detailed interview to get history about development, education, medical, etc. For issues like ADHD, where an individual’s perspective may be biased, psychologists also get information from other reporters, like teachers and parents. When looking at testing results, interpretation needs to take into account all of the data. Let me illustrate (caveat: this example is made up and should NOT be taken as an example of a valid interpretation for any particular person). Suppose I have an adolescent patient with a VCI (verbal intelligence) of 70, but there is a large discrepancy between scores on the Information and Vocabulary subtests, with much higher scores on the latter. Looking at the educational history, I see this person was never been in formal schooling and learned almost nothing about history at home. Knowing that the Vocabulary test loads highest onto general intelligence and that Information is more dependent on education than other subtests, I would not interpret a VCI of 70 as evidence of intellectual dysfunction (formerly known as mental retardation). If scores on other domains, like perceptual reasoning, working memory, and processing speed, were in the normal range, then I certainly would not interpret the tests as evidence of intellectual dysfunction. Instead, I would interpret the low Information score as likely due to limited education. I would compare this to the academic achievement testing I administered to see if there is evidence of limited academic skills or limited ability to apply them. If all VCI subtests were very low and consistent and the educational history evidenced sufficient opportunity, I might consider the history to see if testing for autism is warranted. So can you see now why it takes a psychologist to administer and interpret this? Also, when I say that only licensed psychologists can interpret IQ tests, I am right both in terms of legality and competency. It is just the same as saying that only medical doctors can interpret your bloodwork. Yes, certain biologists could look at the data and draw conclusions, but the doctor has extensive training on all of the possible factors in an abnormal test. A high count on X marker can mean a lot of things, and it is actually illegal for people who are not properly trained and credentialed to interpret these kinds of tests.
“Secondly, anyone with an education in psychology ought to have taken enough course hours on testing methods, and enough course hours of statistics (I had 3 semesters of each) that they’d have enough understanding of the problems in IQ testing that they wouldn’t take it seriously at all. Unless they’re handing out degrees in “pop psychology” nowadays, that is.”
I’m a licensed psychologist with a PhD. Is that enough education in psychology to meet your standard of expertise? Who taught your statistics courses? Why were they lecturing about the limitations of IQ testing in those courses? It seems like a crank’s pet issue that does not belong in the curriculum except in a psychometrics/tests & measures course. IQ is not “pop psychology.” It is mainstream. And it has been heavily researched. There is a ton of scientific literature behind its development. Psychologists like myself are well aware of its limitations. We were the ones that identified them in the first place! We are the ones who take those limitations into account when interpreting IQ tests. Knowing those limitations allows us to apply IQ tests in prescribed way that minimizes assumptions and risk of drawing wrong conclusions. Your instructor is simply out of touch with the field and did you a disservice by going on a tangential screed on a topic about which he/she knew very little.
“I am not a practicing psychologist”
Exactly. So why are you acting like you know something about this? Undergrads learn almost nothing about intelligence testing, and neither do terminal master’s students. Training in cognitive assessment is part of PhD curriculum.
“IQ tests are critical for assessing your ability to score on IQ tests.”
Wrong. IQ is an imperfect, yet robust predictor of many important outcomes. There is an extremely large body of literature on this topic. Here is an example, I might be doing a worker’s comp eval for a worker who sustained a closed head injury. The worker claims to be unable to understand simple instructions now. Assessing his working memory using an IQ test is crucial to determining whether or not he is telling the truth (by the way, this and other tests have an embedded test for determining whether someone is just faking).
“Was it nutrition, protection from environmental toxins like lead, better education, private tutoring, better access to books – there could be a lot of factors – and since none of those factors are captured in the information about the sample, it’s impossible to say anything about its impact on the test.”
You can’t determine whether environmental toxins, such as lead, are harmful to intelligence if you don’t measure intelligence. Seriously, that is how we can quantify the impact of such things. Yes, education, tutoring, and other privileges do have some correlation with intelligence (actually, not much). As I said before, your IQ test scores are derived by comparing your performance to grade-matched or age-matched norms. So all of the factors you listed (and many others, such as social and medical history) are captured and taken into account when interpreting IQ scores.
Marcus, your beef is with media personalities, non-psychologists, and a few fringe psychologists who have made inferences about intelligence that go beyond the science. I agree with you on that point. Murray and Hernstein were wrong and racists. Psychologists know this. And we still administer IQ tests. Because you shouldn’t throw out an important scientifically-derived clinical tool just because some idiots misuse it to pretend that they are sociology and political science experts. Just as the pseudoscience in evolutionary psychology (yes, I agree with PZ it is a mostly bogus field) does not mean the theory of evolution is wrong, please do not let those who are misapplying intelligence discredit this important psychological construct for you.
#53 consciousness razor
Your question is about the construct validity of IQ tests. What are they measuring? What they aim to measure are the trait-like cognitive abilities that enable flexible learning and success. Is this a fuzzy construct? Absolutely. Welcome to psychology. Anxiety, depression, etc.- they are all abstract categories describing a subset of phenomena that are closely related. This is an issue in all of science. Your yardstick is affected by barometric pressure and temperature (though very minutely). Does that question its validity as a measure of length? Of course not. Those other factors are measurement error. We have more measurement error in psychology than in most other fields. In the case of IQ, I cannot directly measure someone’s actual intelligence, but I can assess their behavior by administering a performance-based test. I can then compare performance to an education or age normed sample of the population to determine whether an individual may have unusual impairment or giftedness in these cognitive abilities. Hopefully that answers your question. So why assess IQ at all? Well, let’s say I want to know whether lead exposure is impacts cognitive abilities? Maybe I am interested in finding out whether a child who lives in Flint may have been impacted by lead exposure. This question is very real and relevant to the child’s ability to seek recompense and assistance with education, etc. Without IQ or other performance-based tests, how would you answer the question? With an IQ test I can answer the question. I can do so by testing the child’s performance on a number of tasks which rely on different cognitive abilities.
You said “But the issue is still how seriously we ought to take them as being a genuine measure of intelligence, instead of some other potentially-useful or interesting constellation of attributes that a person may exhibit, which for whatever reasons psychologists/etc. may want to study.”
Actually, your definition here is pretty close to how psychologists DO conceptualize intelligence. We operationalize it as a constellation of cognitive abilities measured through behavioral tests.
Your example about musicians you know is interesting. Working memory capacity is pretty stable (as are the other abilities that load onto the latent factor we call intelligence). Stroke or injury can impact it, though. Anxiety can impact it somewhat. People can functionally improve working memory performance using techniques such as chunking. Working memory is one of those cognitive abilities that form part of the cluster of abilities we call intelligence. Others include processing speed (how fast you process novel information), perceptual reasoning (how adept you are at applying logic and rules in the manipulation of visuospatial information), and verbal comprehension (how adept you are with language and semantic memory). There is ongoing debate in the literature about whether other factors should be included. IQ tests sample each ability with several subtests and examine consistencies or inconsistencies in performance. A psychologist would not say someone is particularly intelligent just because of a high working memory capacity. The musicians you know might have good working memory, but poor crystallized knowledge or poor critical thinking skills. Again, with intelligence we are not trying to get at “street smarts” or whether or not someone is a skeptic(TM). Instead, we are assessing a constellation of cognitive abilities. In some cases a person is particularly strong or weak in one area. This does not mean the person has high or low intelligence, necessarily. For example, slower processing speed relative to other scores is commonly seen in individuals with high anxiety or ADHD.
skepticalmat@#52:
I am licensed clinical psychologist and a tenure-track professor with a PhD from an APA-accredited program. Not trying to pull rank, just trying to say that I am not some fringe person with a lay knowledge of psychology or IQ. I am an expert.
Then please tell us what “intelligence” is.
@consciousness razor – I don’t think we disagree either. I did mis-parse a bit but we’re on the same track.
@Marcus Ranum #55
Good question. See my response #54.
@Marcus Ranum #32
“Results are not interpreted. When a scientist is measuring something, the measuring tool ought to measure what it purports to measure accurately enough that there is no room for or need for “interpretation” — one does not “interpret” a yard-stick, nor does one “interpret” an IQ test.”
-Marcus, what are you talking about? Every scientist interprets data. Recognizing that interpretation is happening is the first step to recognizing that we can look at data and make mistakes due to bias, etc. When a psychologist performs a cognitive assessment, they typically administer a battery of tests, including an IQ test, along with a detailed interview to get history about development, education, medical, etc. For issues like ADHD, where an individual’s perspective may be biased, psychologists also get information from other reporters, like teachers and parents. When looking at testing results, interpretation needs to take into account all of the data. When I say that only licensed psychologists can interpret IQ tests, I am right both in terms of legality and competency. It is just the same as saying that only medical doctors can interpret your bloodwork. Yes, certain biologists could look at the data and draw conclusions, but the doctor has extensive training on all of the possible factors in an abnormal test. A high count on X marker can mean a lot of things, and it is actually illegal for people who are not properly trained and credentialed to interpret these kinds of tests.
“Secondly, anyone with an education in psychology ought to have taken enough course hours on testing methods, and enough course hours of statistics (I had 3 semesters of each) that they’d have enough understanding of the problems in IQ testing that they wouldn’t take it seriously at all. Unless they’re handing out degrees in “pop psychology” nowadays, that is.”
-Who taught your statistics courses? Why were they lecturing about the limitations of IQ testing in those courses? It seems like a crank’s pet issue that does not belong in a stats course. IQ is not “pop psychology.” It is mainstream. And it has been heavily researched. Psychologists like myself are well aware of its limitations. We were the ones that identified them in the first place! We are the ones who take those limitations into account when interpreting IQ tests. Knowing those limitations allows us to apply IQ tests in prescribed way that minimizes assumptions and risk of drawing wrong conclusions. Your instructor is simply out of touch with the field and did you a disservice by going on a tangential screed on a topic about which he/she knew very little.
“IQ tests are critical for assessing your ability to score on IQ tests.”
-Wrong. IQ tests assess performance on a number of tests that reflect cognitive function. IQ tests are an imperfect, yet robust predictor of many important outcomes. There is an extremely large body of literature on this topic. Here is an example, I might be doing a worker’s comp eval for a worker who sustained a closed head injury. The worker claims to be unable to understand simple instructions now. Assessing his working memory using an IQ test is crucial to determining whether or not he is telling the truth (by the way, this and other tests have an embedded test for determining whether someone is just faking).
“Was it nutrition, protection from environmental toxins like lead, better education, private tutoring, better access to books – there could be a lot of factors – and since none of those factors are captured in the information about the sample, it’s impossible to say anything about its impact on the test.”
-You can’t determine whether environmental toxins, such as lead, are harmful to intelligence if you don’t measure intelligence. Seriously, that is how we can quantify the impact of such things. Yes, education, tutoring, and other privileges do have some correlation with intelligence (actually, not much). As I said before, your IQ test scores are derived by comparing your performance to grade-matched or age-matched norms. So all of the factors you listed (and many others, such as social and medical history) are captured and taken into account when interpreting IQ scores.
Marcus, your beef is with media personalities, non-psychologists, and a few fringe psychologists who have made inferences about intelligence that go beyond the science. I agree with you on that point. Murray and Hernstein were wrong and racists. Psychologists know this. And we still administer IQ tests. Because you shouldn’t throw out an important scientifically-derived clinical tool just because some idiots misuse it to pretend that they are sociology and political science experts. Just as the pseudoscience in evolutionary psychology (yes, I agree with PZ it is a mostly bogus field) does not mean the theory of evolution is wrong, please do not let those who are misapplying intelligence discredit this important psychological construct for you.
skepticalmat
Doing this
<blockquote>paste copied text here</blockquote>
Results in this
It makes comments with alot of quotes easier to read.
Thanks chigau. I am clearly a rookie here. :p
Here’s my question: why is everything about AI always about skipping to a genius supercomputer with human-like sapience? You never hear people talking about trying to figure out how to program AI on the level of a vole or even lower. To simply create an artificial organism as advanced as a bacterium would be an amazing leap of progress, but everyone seems to act as though we’ll skip right to making a computer that can have conversations with us about Descartes; it’s like people talking about calculus without any understanding of basic arithmetic.
zibble, I’m guessing part of it is that some people wouldn’t find AI equivalents of a small rodent interesting enough, while other people wouldn’t find AI equivalents of a small rodent scary enough.
PZ, I dispute your point #4, though not in a way that matters for the larger argument. There are at least 2 species (with multiple subspecies, or a couple more species, depending on classification schema) on Earth with human-like intelligence: H. sapiens and P. troglodytes. Possibly more, depending on how you define “human-like”: cetaceans and elephants, for example. Non-mammilian species with advanced problem-solving abilities, from avians to cephalopods to the emergent collective intelligence of insect colonies, are decidedly NOT human-like, though they can also address similar problems and possibly reason in terms of abstractions.
@zibble #61:
You’re universalizing your personal experience: you clearly don’t hear about it (I suggest framing statements to that effect using “I” statements as opposed to “you” statements, because even/especially with a general “you”, you’re presuming your personal experiences are universal or at least majoritarian), but most of what I hear about AI is about mimicking emergent insect colony intelligence and/or individual insect intelligence for autonomous drones/rovers or swarm AI for data analysis of everything from social media use and advertising return to facial recognition.
skepticalmat@#58:
Marcus, what are you talking about? Every scientist interprets data. Recognizing that interpretation is happening is the first step to recognizing that we can look at data and make mistakes due to bias, etc.
Wouldn’t the time to have done that been before the test was used on the general public for decades? I interpret “interpreting” as “trying to figure out how we can repair a horribly flawed test” – which is a legitimate goal, but given that the test has been widely acknowledged to have cultural and educational biases, it’s a flawed instrument and it shouldn’t be relied on for any real-world measurements, until it’s fixed. Which also appears to be a pretty difficult ongoing problem. I know that the usual response to that objection is “yeah we’re working on it” but how hard do you have to squeeze and interpret a bunch of pseudoscience in order to try to get a few drops of science out of it? Throw the whole mess out and start talking about how to measure those “trait-like” “cognitive abilities that enable flexible learning and success” and what they are and maybe you can claim to be doing science. By the way, WTF is “trait-like”? Do you mean something that appears like it may be innate but also may not? That’s a mighty nice hedge.
When a psychologist performs a cognitive assessment, they typically administer a battery of tests, including an IQ test, along with a detailed interview to get history about development, education, medical, etc.
I know. Now, what does the IQ test show in that situation? Are you taking it to be measuring something about the “trait-like” “cognitive abilities that enable flexible learning and success”? Or are you treating it as a proxy for … well, you tell me. I fully understand that IQ tests are used often in psychology. I know that phrenologists used to measure their patients skulls, too. How are you able to tell if you’ve got a patient that’s maybe not very well educated, or maybe dyslexic and has reading problems, from a patient that performs badly on the test because they lack those “trait-like” whatevers?
For issues like ADHD, where an individual’s perspective may be biased, psychologists also get information from other reporters, like teachers and parents. When looking at testing results, interpretation needs to take into account all of the data. When I say that only licensed psychologists can interpret IQ tests, I am right both in terms of legality and competency.
Hang on, when you’re making a subjective assessment of reports from other sources and “interpreting” test results in with that, you’re not interpreting the IQ test, the IQ test and a bunch of other stuff are subject to interpretation collectively. I was leaping like I’d been hit by a cattle prod because I understood what you were saying as being that the IQ tests were being “interpreted” somehow rather than just being scored and factored in to your overall assessment of the patient. In that case, I can see that if you have, for example, a patient who is so frustrated and distracted that they don’t even finish the test and ball it up and throw it away, you might interpret that overt behavior surrounding the test as part of your subjective assesment of all the information you’ve got to look at.
Or are you telling me that someone might score badly on an IQ test and you might use that to support a conclusion that they have ADHD (whatever that is) because something something mumble trait-like mumble something? Wouldn’t MMPI be better? (of course MMPI has well-known flaws, and once again has been subject to psychologists trying to squeeze the pseudoscience and collect a few drops of science from it…)
A high count on X marker can mean a lot of things
That’s really the nub of my objection. If you’ve got a test that is trying to objectively measure a phenomenon, it shouldn’t require a fortune teller to read the yarrow stalks afterward and tell you which of “a lot of things” the Queen of Spades means when Jupiter is in the 7th house. A test that doesn’t accurately measure an understood and objective criterion, isn’t much of a test.
In #53 you equivocate:
This is an issue in all of science. Your yardstick is affected by barometric pressure and temperature (though very minutely). Does that question its validity as a measure of length? Of course not.
A yardstick is also a bit of circular reasoning: it objectively measures, with some degree of accuracy, a defined quantity. There are potential margins of error, but you don’t have to have a professional carpenter come make a certified assessment of a yardstick measurement. Like you admit is necessary with these psychological tests. I can’t tell if you’re being disingenous in trying to imply that psychological test measurement is remotely on epistemological par with a yardstick, or maybe you just don’t know how yardsticks work. I introduced the question of yardsticks into this discussion early on because of that interesting circularity: yardsticks measure one yard, how do you know? Because that’s the definition of a yard. What does an IQ test measure? “Trait-like” something mumble learn waffle something? I’m OK with saying “IQ tests measure your ability to score on an IQ test” and that’s completely defensible, just like yardsticks measure yards. But the people promoting the use of IQ tests have two problems:
1) what does it measure? “trait-like something mumble waffle learning something mumble” – ok, what is that?
2) then stop calling it fucking IQ and say “IQ has nothing to do with intelligence” – yet there is still a huge amount of conflation of IQ tests with intelligence tests. If you’re using it to see something about ADHD then call it an “ADHD something trait-like mumble waffle something test battery” because you damn sure aren’t measuring “intelligence” (and I clearly think your definition of “intelligence” needs some work in order to be merely ridiculous)
Who taught your statistics courses? Why were they lecturing about the limitations of IQ testing in those courses?
Dan Naiman. He was awesome. You missed the part in my sentence that you quoted: “enough course hours on testing methods” – that was professors David Olton and Jason Brandt. My particular “spin” on psychology was probably a result of reading a bunch of skeptical philosophy (I was taking a dual major in philosophy and psych, but I only got the psych degree course hours so I graduated with just a psych BA) at the same time as I was studying test design and – yeah – psychology didn’t stand up very well. Professor Naiman didn’t talk about psychology at all. I got the impression that mathematicians and statisticians don’t think psychologists are much better than cargo cultists, but that was the late 70s attitude, maybe psychology is perceived better now. I hope so.
IQ is not “pop psychology.” It is mainstream. And it has been heavily researched. Psychologists like myself are well aware of its limitations. We were the ones that identified them in the first place! We are the ones who take those limitations into account when interpreting IQ tests.
It is being used in the mainstream. It has been heavily researched. Psychologists like you are well aware of its limitations. You identified them. You take those into account. Great. Do you also use Meyers-Briggs? Because the exact same argument applies to MBTI and I sure as hell hope you’re gonna say “no that load of codswollop is pseudoscience.” And we agree. But it’s mainstream, it’s been heavily researched, and it’s still codswollop. Your argument that IQ testing is a flawed instrument and yet you’re somehow capable of using it by “interpreting” it … you’re starting to sound like a medicine man, to me. If you’re aware of its limitations why not throw it out?
Your instructor is simply out of touch with the field and did you a disservice by going on a tangential screed on a topic about which he/she knew very little.
Your failure to read my comment and parse it correctly doesn’t say anything about the otherwise excellent professor Naiman.
My interpretation of psychology’s problems are a result of studying perhaps a bit too much David Hume while reading some of the theater of error that is psychological testing. Unlike you, I am not willing to excuse the field or interpret its flawed instruments. If psychologists know these instruments are flawed, why are they still using them? Oh, wait – they’re trying to fix them, right? Why haven’t they fixed them. I bet the answer is “it’s hard” – well, if it’s hard to fix those flawed instruments then psychology can’t claim any understanding of what they are objectively measuring, or they’d have fixed them long before now.
You can’t determine whether environmental toxins, such as lead, are harmful to intelligence if you don’t measure intelligence. Seriously, that is how we can quantify the impact of such things.
Uh, what? Lead is a neurotoxin. “Neurotoxin” means it’s harmful to brain activity. You ought to be using exposure to quantify impact.
Let me turn that around into a thought experiment:
1) patient presents at your clinic
2) you give them an IQ test and do a bunch of other stuff
3) the IQ test is low, so you conclude they have lead exposure
nonsense, right? If you agree that’s nonsense then you can’t tell me about lead exposure from the IQ test. Let’s try another thought experiment:
1) patient takes an IQ test
2) patient is exposed to lead
3) patient takes an IQ test
can you quantify the lead exposure? Of course not. For one thing you have no idea if the patient is also doing meth and didn’t report it.
Sure there are lots of external pieces of data that you might actually use to actually quantify the impact, but then do that and don’t waste your time doing IQ tests.
Yes, education, tutoring, and other privileges do have some correlation with intelligence (actually, not much)
Oh, really? Not much?
I’ve probably read too many eugenics textbooks… (Oh, by the way, here’s a hint for psychologists: when you start sounding like eugenicists, you’re probably putting the cart before the horse somewhere) There always seem to be factors that got overlooked. You conveniently mentioned education, tutoring, other privileges, but there’s just a whole lot of external factors to control for: what about that poor people tend to live in neighborhoods nearer to highways and kids born in the 60s and 70s were exposed to more airborne lead from car exhausts? Or poor people living in swampland or who don’t have as much political control get sprayed with malathion more often? Or alcohol abuse (which is across demographics and may be unpredictable based on a parent’s wealth) or cigarette abuse, etc, etc. Whatever “intelligence” is. Can an IQ test tell a genius whose brain was damaged by lead paint growing up, so instead of being a great physicist they were a mere marketing executive? Let me guess, you’ve got to interpret all that.
As I said before, your IQ test scores are derived by comparing your performance to grade-matched or age-matched norms.
Yeah, that was one of the big fixes for IQ tests. I admit that works – it avoids having to try to figure out an epistemology of intelligence by letting you say “this kid is way smarter than all the other kids” or even “this kid is better at our verbal tests than other kids.” Tell me, do SATs and AP exams measure “intelligence”?
Marcus, your beef is with media personalities, non-psychologists, and a few fringe psychologists who have made inferences about intelligence that go beyond the science.
You have no idea what my beef is.
Yes, I do have a beef with media personalities, pop psychologists, and pseudoscientists. The problem is that, in general, I lump psychologists under pseudscientists. And I don’t do that because someone went on a mind-tainting rant to me about psychology’s poor testing methods – I got to where I am by being a student of psychology, looking at the bullshit that the field is built on, looking at how the field keeps trying to squeeze drops of science out of pseudoscience, and the kind of horrifically stupid and pointless experiments that psychology has done and the stupid and ill-founded tests they continue to apply.
zibble@#61:
To simply create an artificial organism as advanced as a bacterium would be an amazing leap of progress
There was a bunch from MIT who were trying to behaviorally reverse-engineer cockroach algorithms. I wonder whatever happened with that effort – they wanted to be able to make a robotic roach that would run from the same things a roach would, in the same directions, etc. I believe that ought to be pretty doable since roach options are limited (they tend to run some kind of forward) so you could probably model them statistically. Markovroach is just the beginning.
@timgueguen
Frankly, I’d be impressed by an AI emulating an ant or even more impressively, a honeybee.
The funny thing is, a decent workstation has more circuit elements and logic units than either creature, but has not been able to be programmed anywhere near to the processing complexities of a simple ganglion.
Part of the problem is, our computers are simple binary logic units, biological neural processing uses multiple neurotransmitters and neuromodulators, making direct equivalents utterly impossible.
Additionally, a computer uses a master clock and sub-clocks that are few in number, biological processing uses many, many, many clocks, some, variable in timing.
So, emulating neurological processing with binary computers is already a process fraught with using an incompatible tool to create its peer, analogous to using an electron microscope in the place of a hammer to hammer a nail into the wall to hang a picture – after using that same electron microscope as a hammer to build a house that contains that wall.
Worse, we still don’t quite understand much of neurological processing biochemistry, as in, what modulates what, at what level, interactions between different neurotransmitters and neuromodulators. In short, we still haven’t even come close to understanding how the simplest brain processes stimuli and both controls a body and reacts to all of the bombarding stimuli it receives.
So, in short, we don’t know how a brain operates fully, yet some expect us to emulate one faithfully, via a technology that is spectacularly incompatible with the original model.
@Marcus Ranum #65, honestly, I’d have suggested emulating an ant. Social behavior would’ve been easier, smaller brain size and hence, number of neurological circuits, social behavior being a fair bit more of mirror neurological processing, etc.
Roach behavior has some social insect behavior and a lot of individual behavior. As an example, I once watched a colony of cockroaches attack a preying mantis and kill it. They teamed up in an organized fashion and attacked the eyes first, leaving it defenseless, to be consumed at their leisure.
But, the problem remains that our computers use binary logic, neurons can utilize multiple neurotransmitters and those can be modulated by other neuromodulators. So, one for one functional logic emulation isn’t quite possible.
I suspect, we’ll not even emulate such a simple organism until we both fully understand neurology at a cellular/group level and we have quantum computing well, truly and commonly in use, as the multiple logical states would simplify processing complexities.
While, we do have a few quantum computers, they’re far simpler than even an ant’s processing capabilities and still experimental. Some of the latter, I cannot discuss, due to an NDA.
For both, we’re at the equivalent of ENIAC knowledge, seeking to create a modern supercomputer.
I saw some effective Bee-IQ the other day, on Zuckerburg’s platform that AI’d me the video. I’ll recount it in the manner of a professional entomologist. There were some bees. Perhaps as many as a lot. The boffins had them buzzing over some dishes of bee drink/food that were under glass fixed too low to crawl under to the prize. The dishes were on strings that protruded from the glass. Well now, two maverick bees pulled the dishes out with the string. They buzzed off to tell the others back home (somewhere else in the lab), and then the subtitles on the video said the audience bees then went and did the same. Unfortunately, the clip stopped before that point, and I must go and watch the rest of it sometime.
Now they must have been she-bees, because they asked directions. It would be rather dramatic if they had explained the new abstract combo-concept in jiggle-lingo. I expect they instead said come and copy and paste. What would you call this sort of thing? hox-meme perhaps. I was impressed, but then I know people who couldn’t do that.
skepticalmat
I disagree, upon extremely narrow grounds. Tutoring can help when dysfunction would interfere with education. For example, I have dyslexia. My IQ scores and knowledge base are high because of intensive training to compensate for my dyslexia.
But, that’s a very narrow quibble. For the rest, I agree with you. I think some of the disagreement is also entrenched within some narrow areas as well, along with some misunderstandings of key concepts.