A few days ago, I posted a review of a study on the genetic signatures of a population of individuals who attained extreme old age (95-119). The study was based on a genome-wide array analysis. Briefly, genetic material obtained from many genes can be arrayed, using a robot, on a glass-based wafer, very much like a silicon chip in electronics. This is called a DNA chip. By exposing the chip to fluorescent probes, only the genes that are structurally complementary to the probe will fluoresce a green color. Of the 25,000 genes that were studied in this paper, 150 SNP (Single Nucleotide Polymorphism) genes were Longevity Associated Variants.
The purpose of this follow-up post is to show you how science works. Every paper goes through peer review before being accepted for publication. But this is not the end of it. Once published, the scientific community at large has a chance to critique and challenge assumptions, techniques, analyses, and interpretations. The authors then respond. But again, the process is not finished. If the findings are important enough, there will be other investigators who will try to replicate the study. Only then will a study be universally accepted. This process ensures that science is self-correcting. This is how errors are found, how preliminary results are made robust, and, yes, how fraudulent results are uncovered (would that our political dialogue be subject to such rigor).
The self-correcting process is beginning with respect to the Longevity Study. Here is an interview with the study’s authors published in the prestigious journal, Science:
Authors of Controversial Longevity Study Discuss the Furor
by Jennifer Couzin-Frankel
A week ago, Science published a paper whose title says it all: “Genetic Signatures of Exceptional Longevity in Humans.” The work identified a cluster of 150 genetic variants called single nucleotide polymorphisms (SNPs), single DNA bases, that appeared to have remarkable prognostic value. As a group, the SNPs helped predict whether 2322 people in the study were centenarians 77% of the time. (1,055 were centenarians and 1,267 were not.)
Within days of publication, controversy erupted. In a lengthy article posted on Newsweek, as well as blog posts, researchers expressed concerns about the paper. One potential problem cited by critics was that the authors used a relatively small sample size for the type of analysis conducted, a genome-wide association (GWA) scan of the genomes of centenarians and those who were younger. (To increase statistical power, many GWA studies use DNA from 10 times as many people or more.) Another concern was about the paper’s technical accuracy—in particular, how the use of a digital SNP chip called the 610 Illumina array, that some say is problematic, might have impacted the results.
Science Insider spoke with the paper’s two lead authors this afternoon, Paola Sebastiani (PS) and Thomas Perls (TP), both of Boston University. They are currently rechecking their data for accuracy. The conversation has been edited for brevity and clarity.
Q: What do you make of the controversy? Did you expect the paper to be controversial?
PS: To be honest, I expected this to be controversial for the type of analysis, and not for some issues with the data. There are very innovative ideas in the paper, …the idea of not really looking at the effect of individual SNPS, but the global effects of several SNPs, …the idea of looking at patterns rather than individual variants. I expected that to generate discussion.
Q: GWA studies usually focus on common diseases. Since centenarians are so rare, is GWA the right technique for studying them?
PS: I think one has to be careful about how we interpret the sample sizes. The power is not determined [only] by the sample size but also by the method you use.
TP: I think there’s another couple of features that made the centenarians a particularly potent sample to perform this study with. We had evidence for quite some time that this [longevity] ran very strongly in families. …One of the problems with GWA [studies] …is that the phenotype can be very different among different people in the study. [Here], we have these very old individuals that [are more homogenous].
Our primary purpose in conducting this work was to understand the genetic basis of exceptional longevity. …It has nothing to do with trying to get a bunch of people to be 100 or older.
Q: Had you heard about problems with the 610 SNP chip that other researchers say they’ve experienced?
PS: Honestly, I didn’t. However, we are taking this very seriously. I know of a lot of other studies that use this chip, so I was really surprised that it was such a major issue that you never heard about. It could be my fault. We are looking at this carefully; the [paper’s] analysis can be rechecked.
Q: What kind of rechecking are you doing?
PS: We want to look at the results [and] how robust they are.
Q: How do you do that?
PS: We are working on this [now], so we prefer to discuss it once we’re done.
…When you come out with a provocative paper, people are going to scrutinize it very carefully. …If concerns are raised, it is certainly very, very important for us to follow up on those concerns.
Q: How did the concerns come to your attention?
TP: It was via the media.
Q: In media reports, critics have said that if the SNP chip produced unreliable results, that could affect many of the variants you reported, but in a statement released yesterday, Science said only two of 150 the variants might be affected. Can you explain this discrepancy?
TP: Some concern has been raised about the technical validity of the SNPs and the chip, and so we are looking at the effect of that on these signatures. Until we’ve done the proper analysis, we really can’t speak to what that is, and we’ve just asked people to be patient and let rigorous science take place. We should have the answer very soon.
Q: Would it have made sense to do this rechecking before the paper came out?
TP: If we had known about the problem, we certainly would have taken that into account.
PS: I think there are errors in every paper. This is part of the scientific debate—from errors, sometimes you can come up with very good ideas.
Q: Do you feel that the results in the paper are generally correct, that they’ll hold up?
PS: We want to wait until all the analyses are done.
TP: That’s one of the problems that we do not want to contribute to. There have been very premature and harmful proclamations about what the data show or don’t show. I think people need to know that good science takes time, and while people are anxious to know and we completely understand that, we want to be able to do a thorough…analysis.
This is not going to take months or even weeks. We will have this available fairly soon to the editors of Science, and then they will do a very rigorous go-through…and make sure that scientists around the world will be satisfied with the analyses, whatever they show.