Finally, a tiny voice of caution speaks out against the genetic testing hype.
The Food and Drug Administration has ordered DNA testing company 23andMe to stop marketing its over-the-counter genetic test, saying it’s being sold illegally to diagnose diseases, and with no proof it actually works.
The heavily marketed test includes a kit for sampling saliva, and the company promises to offer specific health advice. “Based on your DNA, we’ll provide specific health recommendations for you,” the company says on its website. "Get personalized recommendations."
In an unusually scathing letter dated Friday, the FDA says it’s been trying to work with the company to get some sort of evidence that the test can do that with any accuracy.
I had no idea that 23andMe was making any health claims, and that’s deplorable. You can’t do that. That’s naive billiard-ball-biology, and it’s never going to be as simple as testing a few markers and then declaring that you understand physiology.
I prefer the approach of the National Genographic project, where the results are used to infer relationships rather than leaping to biomedical conclusions. We have far more accurate tools for determining your medical condition — it’s direct and involves examining your health, rather than indirectly looking at genes that have a remote connection to your health.
Which brings me to an essay that had me gawping in disbelief. A neuroscientist, James Fallon, noticed the results of a PET scan of his own brain.
“I got to the bottom of the stack, and saw this scan that was obviously pathological,” he says, noting that it showed low activity in certain areas of the frontal and temporal lobes linked to empathy, morality and self-control. Knowing that it belonged to a member of his family, Fallon checked his lab’s PET machine for an error (it was working perfectly fine) and then decided he simply had to break the blinding that prevented him from knowing whose brain was pictured. When he looked up the code, he was greeted by an unsettling revelation: the psychopathic brain pictured in the scan was his own.
OK. If this happened to me, I’d place the most importance on my personal experience — if I were a successful professional with no history of unethical behavior, I’d say “uh-oh…maybe these scans aren’t such a reliable indicator of personality after all.” I would not say, “uh-oh, I must be a psychopath.”
But guess what interpretation Fallon put on it? He got genetic tests.
But when he underwent a series of genetic tests, he got more bad news. “I had all these high-risk alleles for aggression, violence and low empathy,” he says, such as a variant of the MAO-A gene that has been linked with aggressive behavior. Eventually, based on further neurological and behavioral research into psychopathy, he decided he was indeed a psychopath—just a relatively good kind, what he and others call a “pro-social psychopath,” someone who has difficulty feeling true empathy for others but still keeps his behavior roughly within socially-acceptable bounds.
Wow. And then he starts self-rationalizing. He’s aggressive when he plays games, therefore his diagnosis must be true. He admits that maybe this isn’t as clear-cut as he thinks.
But the fact that a person with the genes and brain of a psychopath could end up a non-violent, stable and successful scientist made Fallon reconsider the ambiguity of the term. Psychopathy, after all, doesn’t appear as a formal diagnosis in the Diagnostic and Statistical Manual of Mental Disorders in part because it encompasses such a wide range of symptoms. Not all psychopaths kill; some, like Fallon, exhibit other sorts of psychopathic behavior.
But one thing he doesn’t consider? That maybe PET scans and genetic tests aren’t as robust and interpretable as he thinks. What I find personally chilling is that he so blithely considers a scan or a gene so definitive that he will defend a diagnosis of psychopathy in himself; does he also judge the subjects of his research on the basis of these abstractions rather than on their behavior?
So if you are intelligent and have lots of information at your fingertips and have access to even more information …
you can fuck up really bigtime.
Comforting.
Chigau:
Keep in mind, lots of conspiracy theorists are anything but stupid. They just have an extremely scrambled concept of logic.
“But one thing he doesn’t consider? That maybe PET scans and genetic tests aren’t as robust and interpretable as he thinks.”
______________________
In the article it is clearly mentioned that he does give consideration to this.
For some reason he elects to go along with it, make the vague term psychopathy even vaguer by introducing pro-social variants and then proceed to give TED Talks about it. All a bit baffling.
I saw a documentary on psychopathology on the Beeb a year or so ago which featured Fallon quite heavily. It seemed to me that he made quite a good case for his diagnosis, and it wasn’t as though it wasn’t supported by his colleagues. And in the quote above, he says “several” markers rather than just “one gene”. It’s possible that he’s convinced himself of something that ain’t so, but on the other hand, perhaps it is true. he, of all people should be aware of the deficiencies of brain scanners and the wooliness of psychopath profiling.
Not every psychopath is a Hollywood spree-killer, some — maybe most — are quite externally socialised and pragmatically committed to not making trouble in this society of aliens they find themselves in. I encountered a forum for sociopaths a while ago; while some of the members were Internet-nasty and proud of their lack of empathy and some were genuinely disturbing, some just wanted to get along to get along. Again, it’s possible that they were all self-deluded into thinking that everyday alienation was psychopathy, but they seemed genuine to my layman’s eye.
And we know that there are people outside of Hollywood movies who get along with us social apes quite remarkably well, becoming respectable members of society, like teachers, police, bankers, dentists, doctors — until the day comes when the social veneer, already thin in places, peels off entirely and the ugly truth is revealed. Why not a neuroscientist?
Of course, Fallon also has a book out, The Psychopath Inside.
how do you know he didn’t? does the man seem irrational? no? then why wouldn’t he?
this is likely because he trusts the data he has seen with his own eyes. My guess is he has not only participated in, but read hundreds of studies supporting the correlative data.
strong correlation does not mean perfect correlation, you know this.
it is just as wrong to say the data has no meaning whatsoever. I hope that isn’t the case you are trying to make here, PZ, because THAT would be the truly chilling thing… you seem to want to reject the correlative data out of hand, without any knowledge of the research involved.
frankly, you scare me far more than he does!
for some reason?
that the best you can do?
I actually think the FDA is wrong on this one. 23andme is not claiming to diagnose any disease. All they do is genotype your SNPs using commercially-available technology (microarrays from Illumina), then provide information on studies (GWAS mostly) that link those SNPs to particular diseases. Now, that means that a lot of the information is not terribly helpful (oooh… I have a 17.3% risk of gout!), but they give it in the context of the heretibility and population-wide incidence of the disease. They also give good information on those studies–how many people in the study, from what populations, and whether the studies were replicated.
That all seems perfectly reasonable to me. What I don’t know is what the FDA wanted in terms of clinical data–the microarrays are a standard technology that have high repeatability. The GWAS studies are peer-reviewed. It really seems to me that all 23andme is doing is providing a service that genotypes you then provides an organized literature review. Seems a lot more legit than a lot of those alt med people that just make wild claims about curing cancer.
Did you know that it is possible to copy/paste the nym and number of the comment to which you are referring?
You seem to have that whole <blockquote> thing worked out.
Ichthyic, I have no idea why you choose such a hostile approach. What on earth do you mean “Is that the best you can do?”.
For goodness sake, I was making a brief comment about the strange bait and switch the man took. Surely you can see that he has moved the goal posts by redefining psychopathy so that his findings continue to fit within it – rather than abandon his hypothesis.
You are the one making lazy assumptions regarding the allegedly strong correlation. Why don’t you turn your critical eye on yourself. Is that really the best *you* can do?
I can’t help but note how fortunate it was that his deliberate unblinding of his own experiment lead to a new definition of psychopathy and also a book and lecturing circuit.
“I actually think the FDA is wrong on this one. 23andme is not claiming to diagnose any disease.”
Are you kidding me?! Read the copy on their website! It’s full of prose touting health benefits. They’ve gone way, way over the line. It’s Kevin Trudeau level bullshit.
“A simple DNA test led to an unexpected diagnosis for a mother and daughter, changing their lives forever.”
That’s a direct quote describing a video on the website. That kind of prose is everywhere.
Well, “led to” is probably fully accurate…
well I think PZ is reacting to the hubris of Fallon.
he is rather proud of his ability to make the interpretation of the data to prove an obviously pathological diagnosis.
He is taking so many things as true including the definitions of his terms so much so that he can coin new terms that include himself even when he does not really fit the original meanings.
Same can be said about the other thing 23andMe while we may be able to know a lot about humans from objective testing of things like scans and genetic analysis not to day.
so the FDA finally said it prove it or shut down.
uncle frogy
I have these dystopian nightmares where the Power That Be end up imposing PET scans and genetic testing to toddlers and invent bullshit interpretations of the results to rigidify the social hierarchy even more.
I am a 23 and Me client as are a few cousins. We didn’t get their genetic testing for health reasons, we got it for the study of our relationships/ancestry. I would venture to guess that is their primary business.
However, they do claim to provide info on 240+ health conditions and traits – but most of that concerns traits not health. They rate ‘health risk’ on something like 122 conditions. In doing so they rate their confidence on their rating of risk and most of those conditions (in my case) do not rate well on confidence. They also rate 53 inherited conditions, 60 traits and 25 drug responses.
Here‘s a piece of the BBC documentary I mentioned, with Fallon talking about his discovery. Of course, he just sounds like an asshole, but maybe that’s what’s an asshole is, a socialised psychopath. Some of them, anyway.
And, of course, Bernard Hill’s narration makes it all sound so much more authoritative.
It’s fortunate it turned out to be his own brain scan. Not pleasant to think of what he might have done to any other family member if it had turned out to belong to someone else (if only to the extent of changing his relationship with that relative. Having, say, your own father or partner or brother, who just happens to be an expert in the field, sincerely convinced that you are a psychopath is probably not good for your mental health).
@hilltop, #10:
“Psychopathy” is not adequately defined in the first place, so I’m not sure anybody can redefine it. It’s a very general term that means all sorts of different things to all sorts of different people in all sorts of different contexts. It is certainly not a specific diagnosis.
laurentweppe @ 15: Me too, although I do believe the nightmare is likely to happen in the current climate. I imagine that such genetic tests will be the requirement for employment, for example, and who is to say it can’t happen? On top of those pointless “psychological profile” pre-employment tests that do not predict anything about the individual and those wonderful efforts to allow employers to force employees and applicants to submit their private usernames and passwords for their private emails and social networking sites for constant employer monitoring, we already have normalized invasive physical tests that have little to no effect on what they claim to, that people don’t think twice about.
And the ACLU a few years ago fought a policy at Akron University to require that all applicants submit genetic material–not even as a post-hire test, but ALL APPLICANTS! For the honor of even applying for the job, with no guarantees! Sure, AU lost against the ACLU case, but the same thing happened with drug testing–it consistently got shot down right up until the Reagan administration, where they decided that the Constitution only counted when it was convenient for them, starting one of the more dangerous precedents that plague us today–and it could only be a matter of time before we’re submitting to pre-employment genetic testing. You know, to make sure they’re not hiring “criminals” or perhaps the much more PC sounding “people with criminal tendencies” because, after all, it “makes sense” and it’s only a private business’s “right” to “protect itself” from the rest of us, right? Or perhaps to “protect” themselves from hiring people who are genetically pre-disposed towards, say, being incompetent or pre-disposed to non-conformity and questioning–after all, the former is a detriment to company productivity and safety, and the latter is a detriment to company unity. Also, for the real reason, to avoid hiring people with inconvenient health problems they don’t want on the insurance, but that doesn’t sound as noble, does it?
I mean, there was a time when phrenology was taken seriously as perfectly logical and scientific and totally acceptable way to make decisions about other people.
These days, those dystopian nightmares are sounding much more likely to happen in the near future, and that scares the hell out of me. Call me a conspiracy theorist all you want, but just judging from how things are and have been going in this country for the last thirty or forty years, I just don’t think it’s that far-fetched.
‘Book Sales’ lights up on my BS monitor along with a flickering of ‘Financial Interest in Related Devices’ And that’s with the ‘psychpathery’ filter turned off and the ‘histrionics’ boost set on low.
It still sort of croggles me that the US allows drug testing of employees who have no need of a certifiable level of attention. I mean, I get it for people like pilots and air traffic controllers, and for police maybe, train engineers, watchkeepers on ships and such.
But why it’s legal to just casually test data entry clerks or cleaning crew or retail salespeople is just baffling, outside the whole “the USA is now a wholly-owned subsidiary of Megacorp Inc., a proud Cayman Island business since 2010!” thing.
And I would bet that most USans don’t actually know that such a thing is generally not legal anywhere else in the western democracies. I’ve never been drug-tested in my life, not even when I was in the Canadian Forces, and certainly never for any job, and health insurance is government-run here, so not there either. It’s just weird.
On the OP, I could see there being some valuable things you could find out, like if you had a predisposition to Parkinson’s or the BR2CA (if I’ve got that right) thing that means such a high likelihood of certain cancers. But even those are just tendencies, not absolute rulesets.
I don’t reject the science — I just have some standards. Notice what he does: he claims to have genetic and neurological markers for psychopathy. But wait, psychopathy is a vague, poorly defined trait! Oh, well, he says, I am kind of an asshole, therefore that’s what I’m detecting. So now he has a genetic test that can spot assholes? There’s this bizarre slip-slidey attitude towards what he’s measuring that puts all of this on a par with psychic reading. Only difference: he’s wearing a lab coat.
Any reason we should take the word of a self diagnosed psychopath?
Oh, PZ, you’re just jealous because Fallon has a better work-related anecdote than any of your zebra fish stories. ;)
Notice, though, that this isn’t Fallon’s article, it’s him being interviewed by Salon, so of course the article’s going to be a bit journalistic and anecdotal, rather than rigorously scientific, even by the standards of squishy sciences like psychology. There may be substance elsewhere to criticise the whole neurological-psychopathology approach Fallon represents, but this article isn’t going to have it, and it isn’t fair to criticise Fallon for saying “I’m an asshole, therefore I’m a psychopath” (I paraphrase). Even if he’s wrong (and this layman can’t tell) it was a little more involved than that.
I’m going to agree with Katie Marshall that 23andme doesn’t claim to diagnose disease. For example, I have an allele that allegedly increases my risk for gallstones. Here is how 23andme explains that:
All very interesting. None of it actually diagnostic. Plenty of recognition of multiple causal factors, and interactions among them.
Well Ryan, I think if a genetic test shows you have a predisposition to a particular disease and you follow that up with your doctor, and the doc diagnoses you with that disease, then it’s perfectly fine to claim that a “DNA test led to an unexpected diagnosis”. They didn’t say a “DNA test diagnosed” them.
While I am not going to comment on the issue of reductionism and whether a genetic test or MRI captures the essence of who you are, I will comment on the issue of genetic testing. I am in the middle of an exome sequencing project to try to identify genes that are linked to a disease. This is a hellishly difficult task. One of the things that the sequencing of the human genome showed us is that our DNA is a complicated mixture of single nucleotide substitutions, copy number variations in which whole chunks of DNA are moved around, or deleted. Each of us has thousands of variations from the reference genomes that are used for comparison. Trying to single out one of those variants and attach it to a disease is not easy.
And I have spent a lot of time in the Genome Wide Association world looking at results from various studies that have looked at both normal properties of people(height)and diseases(depression, bipolar disease, schizophrenia.)While these studies have identified a few (let me stress few)SNPs that seem to be linked to the disease or phenotype, there has been no causal link made between the presence or absence of any SNP and the disease. Indeed, given the odds ratios for the hits found, there are many people in the control populations that carry those variants.
The neuroscientist, Fallon, who made a decision about his brain scan(MRI?)and a psychopathology, and then bolstered his decision with a genetic test just shows how ignorant he is of the utility of these technologies.
CaitieCat @22
Although even then, it would be helpful if they used a test that, you know, actually measured your level of intoxication, like a breathalyzer does (of other popular recreational drugs, cocaine is the only one UA can reliably detect the current presence of.
I haven’t checked the “health recommendations” yet, but I think that for the results they present on health risks and other traits they do an exemplary job of acknowledging, visualizing and explaining probabilities, confidence, along with the underlying science. I know of no other service of any kind that goes that deep in making sure to get the message right.
The FDA shutting them down now seems really weird. If there is a problem with one product, like recommendations, I do not see how the whole service can be affected. Also, it’s 6 years old, they notice that now? Something feels very wrong.
@30 Joerg
The FDA’s letter is culmination of a back-and-forth that they’ve had with the company since 2010. What’s happened is that 23andMe has generally failed to work with the FDA on their concerns, and has completely stonewalled the FDA since May while introduce new marketing campaigns for their testing services. I can’t blame the FDA for being frustrated, although I think they’re over-reacting (there aren’t any case examples yet of 23andMe people getting unnecessary treatment because of their test, despite the service being six years old).
I also think the FDA might be feeling heat from the clinical and insurer side of things. The insurance companies in particularly are likely leery about any tests that might encourage patients to get expensive care without a doctor doing the actual screening.
I find it concerning that he broke the anonymity of his research test. How many people did he test that he would know this was a member of his family? Isn’t he also excluding this result from his study? If this is the type of thing he is looking for he just screwed the results of his entire research project. Kt all seems unethical to me.
Got this link on a bioinformatics site…
Essentially, the 23andMe customer had a heterozygous mutation in each of two different genes. As it turned out, certain homozygous mutations in each of those genes can lead to the same nasty condition, so the software decided to send him a terrifying letter telling him that people with mutations in those two genes “typically” have a devastating medical disease.
He taught himself enough genetics, and learned how to analyze the raw genotype data, to figure out that he had one ‘good’ copy of each gene, so was probably fine, and got 23andMe to correct their assessment, but as a commenter on biostars pointed out, scary assessments like that should be made by clinical geneticists, not computer programers.
http://mntmn.com/pages/23andme.html
It is utter stupidity on the part of 23andMe not to work with the FDA. Why? Abbott received a 125 million dollar fine for repeatedly ignoring the FDA about one of their diagnostic (“we’ve always done it this way, and we haven’t had any problems” is no defense when continuous improvement is required by law), who does have the authority to padlock a facility. They prefer to get cooperation first, but they won’t be ignored. Those who do so better have some really good lawyers who they wish to give a large amount of money to.
Here’s a doctor’s take on 23andMe: http://slatestarcodex.com/2013/11/26/a-letter-i-will-probably-send-to-the-fda/
Wow — a rare double pooch-screwing from PZ Myers.
23andme.com really doesn’t push the genetic determinism line. I see Scott Alexander’s letter on his blog slatestarcodex has already been linked. Short version: he’s heterozygous for an allele with a prevalence of 1 in 30 among Ashkenazi Jews that causes a horrible neurological disease in homozygotes. This is a really useful thing to know.
James Fallon is the real deal: He “has made significant scientific contributions in several neuroscientific subjects, including discoveries of TGF alpha, epidermal growth factor, and the first to show large-scale stimulation adult stem cells in the injured brain using growth factors.” Here’s his presentation of his “pro-social psychopathy”.
And aren’t there mechanisms already in place to suss out such possible problems? There is a difference between the person having the testing done and their possible progeny, in case you hadn’t noticed. Different ethical situations arise. And the Jewish community already has means of saying this marriage shouldn’t take place.
So was Linus Pauling, winner of two Nobel prizes, prior to his advocating large doses of vitamin C. Argument from authority can be argument from a foolish authority. Isaac Newton believed in alchemy and biblical inerrancy for example….
I don’t know about brain scans, but I do know that genetic markers are pretty flimsy as evidence. Not only is it merely telling “somewhere amongst these hundreds of genes in this area something is different in people with…..”. But the step from having a gene and getting it expressed is far from inevitable.
Still if the reliability of a brain scan is good and his own experience confirms then he might have a pretty strong case anyway…
A cursory Google search suggests the answer is: not really. Even as late as 2012, philanthropy was supporting genetic screening for the under- and uninsured in the Jewish community.
A company called Counsyl offers genetic screening for Ashkenazi heritable diseases; in 2011 the price was $349 — not sure what it is now. I think at the time 23andme’s kit went for $299 — a reasonable offering in an established market. 23andme may have been the first to offer genetic info commercially, but it has healthy competition now.
Of course. I offered the info about his research not as definitive proof of his correctness, but as evidence for those interested that they are probably not going to be wasting their time if they click on the link I provided to Fallon’s talk on the topic.
Gee, the old adage is only a fool has himself as his lawyer. Same applies to medicine….
Actually, I believe the adage in medicine is that doctors make terrible patients. And of course he’s not a medical doctor and makes no pretense of being one…
But in any event, I’m guessing you didn’t bother to watch the video — if you had, you would know that he didn’t come to this conclusion lightly, nor did he self-diagnose. He’s relying on multiple strands of evidence, including, in particular, the assessment of professionals in psychiatric research. (These are people who have collaborated with him in various research projects.)
Well just like you’d expect from a sociopath.
Or someone who biases their research.
Again the term itself is very vague and definitions and diagnosis can be stretched pretty far with it.
I also suspect that it’s a rather comforting ‘diagnosis’ for some. “Oh I’m not an asshole, I’m a sociopath. One would be a serious character flaw, the other is just a condition I can’t help”
We also amusingly see people promoting neurotypical people adopt and act more like sociopaths so it’s apparently a horrible condition…that can lead to happiness and success.
IMO it’s vagueness is such that we should be skeptical of any claims related to the condition.