An honest mistake.

Urederra

Mostly harmless
Joined
Apr 22, 2004
Messages
5,310
Location
Sea of tranquility
Yet another scientist who published some papers in Science/Nature and he has to retract his findings.

http://www.sciencemag.org/cgi/content/full/314/5807/1856

[size=+2]A Scientist's Nightmare: Software Problem Leads to Five Retractions[/size]

Until recently, Geoffrey Chang's career was on a trajectory most young scientists only dream about. In 1999, at the age of 28, the protein crystallographer landed a faculty position at the prestigious Scripps Research Institute in San Diego, California. The next year, in a ceremony at the White House, Chang received a Presidential Early Career Award for Scientists and Engineers, the country's highest honor for young researchers. His lab generated a stream of high-profile papers detailing the molecular structures of important proteins embedded in cell membranes.

Then the dream turned into a nightmare. In September, Swiss researchers published a paper in Nature that cast serious doubt on a protein structure Chang's group had described in a 2001 Science paper. When he investigated, Chang was horrified to discover that a homemade data-analysis program had flipped two columns of data, inverting the electron-density map from which his team had derived the final protein structure. Unfortunately, his group had used the program to analyze data for other proteins. As a result, on page 1875, Chang and his colleagues retract three Science papers and report that two papers in other journals also contain erroneous structures.

"I've been devastated," Chang says. "I hope people will understand that it was a mistake, and I'm very sorry for it." Other researchers don't doubt that the error was unintentional, and although some say it has cost them time and effort, many praise Chang for setting the record straight promptly and forthrightly. "I'm very pleased he's done this because there has been some confusion" about the original structures, says Christopher Higgins, a biochemist at Imperial College London. "Now the field can really move forward."

The most influential of Chang's retracted publications, other researchers say, was the 2001 Science paper, which described the structure of a protein called MsbA, isolated from the bacterium Escherichia coli. MsbA belongs to a huge and ancient family of molecules that use energy from adenosine triphosphate to transport molecules across cell membranes. These so-called ABC transporters perform many essential biological duties and are of great clinical interest because of their roles in drug resistance. Some pump antibiotics out of bacterial cells, for example; others clear chemotherapy drugs from cancer cells. Chang's MsbA structure was the first molecular portrait of an entire ABC transporter, and many researchers saw it as a major contribution toward figuring out how these crucial proteins do their jobs. That paper alone has been cited by 364 publications, according to Google Scholar

Two subsequent papers, both now being retracted, describe the structure of MsbA from other bacteria, Vibrio cholera (published in Molecular Biology in 2003) and Salmonella typhimurium (published in Science in 2005). The other retractions, a 2004 paper in the Proceedings of the National Academy of Sciences and a 2005 Science paper, described EmrE, a different type of transporter protein.
Crystallizing and obtaining structures of five membrane proteins in just over 5 years was an incredible feat, says Chang's former postdoc adviser Douglas Rees of the California Institute of Technology in Pasadena. Such proteins are a challenge for crystallographers because they are large, unwieldy, and notoriously difficult to coax into the crystals needed for x-ray crystallography. Rees says determination was at the root of Chang's success: "He has an incredible drive and work ethic. He really pushed the field in the sense of getting things to crystallize that no one else had been able to do." Chang's data are good, Rees says, but the faulty software threw everything off.

Ironically, another former postdoc in Rees's lab, Kaspar Locher, exposed the mistake. In the 14 September issue of Nature, Locher, now at the Swiss Federal Institute of Technology in Zurich, described the structure of an ABC transporter called Sav1866 from Staphylococcus aureus. The structure was dramatically--and unexpectedly--different from that of MsbA. After pulling up Sav1866 and Chang's MsbA from S. typhimurium on a computer screen, Locher says he realized in minutes that the MsbA structure was inverted. Interpreting the "hand" of a molecule is always a challenge for crystallographers, Locher notes, and many mistakes can lead to an incorrect mirror-image structure. Getting the wrong hand is "in the category of monumental blunders," Locher says.

On reading the Nature paper, Chang quickly traced the mix-up back to the analysis program, which he says he inherited from another lab. Locher suspects that Chang would have caught the mistake if he'd taken more time to obtain a higher resolution structure. "I think he was under immense pressure to get the first structure, and that's what made him push the limits of his data," he says. Others suggest that Chang might have caught the problem if he'd paid closer attention to biochemical findings that didn't jibe well with the MsbA structure. "When the first structure came out, we and others said, 'We really don't quite believe this is right,'" says Higgins. "It was inconsistent with a lot of things."

The ramifications of the software snafu extend beyond Chang's lab. Marwan Al-Shawi, a biochemist at the University of Virginia in Charlottesville, says he's now holding on to several manuscripts he was about to submit. Al-Shawi has been using Chang's MsbA structure to build computer models of an ABC transporter involved in human cancer drug resistance. David Clarke of the University of Toronto in Canada says his team had a hard time persuading journals to accept their biochemical studies that contradicted Chang's MsbA structure. Clarke also served on grant panels on which he says Chang's work was influential. "Those applications providing preliminary results that were not in agreement with the retracted papers were given a rough time," he says.

At Scripps, colleagues are standing behind the young researcher. "He's doing some really beautiful work, and this is just an absolute disaster that befell him," says Chang's department chair, Peter Wright. "I'm quite convinced he'll come out of it, and he'll go on to do great things." Chang meanwhile has been reanalyzing his original data and expects to submit papers on the corrected structures soon. The new structures "make a ton of sense" biologically, he says. "A lot of things we couldn't figure out before are very clear.

Ummm... computer models going wrong. This time is not like the fraud made by Hwang Woo-Suk or the one made by this german nanomaterials scientist whose name I can't recall. This time seems to be an honest mistake and the guy really is as brilliant as their colleages say and I hope he will be able to perform as well as he has been doing so far.

But there are way too many mistakes the science/nature reviewers should have caught and they didn't. Is it just me or some reviewers are just too lousy?

Does these type of news damage the credibility of scientific peer-reviewed publications? Does the peer-review proccess need a change, at least in extraordinary claims?
 
I also believe that it's an honest mistake. I don't think that the peer review process needs to be changed, I think that's sort of an unreasonable proposition. I mean, what would you do? Peer review is at the very foundation of all brances of science, and all mistakes are bound to be caught.
 
A point not made in the article is that a scientist, just like any other person, will defend a mistake until he's actually proven wrong.

Chang wasn't proven wrong, another scientist just published a paper whose findings contradicted his. Yet Chang immediately adopted the position that he should assume his findings were the wrong ones, double-checked his own work and found the error almost right away.

He sounds like he has exactly what it takes to be a great scientist.
 
But there are way too many mistakes the science/nature reviewers should have caught and they didn't. Is it just me or some reviewers are just too lousy?

Does these type of news damage the credibility of scientific peer-reviewed publications? Does the peer-review proccess need a change, at least in extraordinary claims?
No, there was absolutly no wrongdoing on the part of the reviewers.

Reviewers don't and shouldn't line for line check the researcher's code.

A lot of science, good non-junk science, that gets published is wrong! This is fine because they are tested and found to be flawed. If we made sure everything that was published was 100% correct all the time nothing would get done!
 
Man, in some lines of work, you'd lose a digit off of your pinkie for something like this ...

Doesn't this group now get to publish five new papers? While it's very sad to happen (I've had columns disappear sometimes when massaging data :) ), it shouldn't set them back too far. Maybe it will damage his career though, which sounds rough.
 
I think this illustrates the strength of the scientific method perfectly.

But I believe extraordinary claims do get more attention by journals and peer-reviewers. Perelman's recent proof of the Poincare conjecture, for instance.
Or Wiles' proof of the FLT. I'd be surprised to discover that it is done differently in the sciences.

Hats off, however, to the human involved in this story, who did not try to cover up his mistake. Scientists and non-scientists alike could learn an awful lot from this.

Take the climate change debate, or evolutionary biology, for instance. A lot of peer-reviewed work has been published. Some of it will no doubt turn out to be erroneous. But it's not clear that the current political climate encourages an "oops, my bad" response.

As I said in another thread, a long time ago:

The Anti-Science faction won a long time ago, when we switched the emphasis from reason to emotion/feeling in the schools. As luck would have it, the Hippies gave the Fundies their biggest boost.
 
A point not made in the article is that a scientist, just like any other person, will defend a mistake until he's actually proven wrong.

Chang wasn't proven wrong, another scientist just published a paper whose findings contradicted his. Yet Chang immediately adopted the position that he should assume his findings were the wrong ones, double-checked his own work and found the error almost right away.

He sounds like he has exactly what it takes to be a great scientist.

My thoughts could not have been expressed more succintly :)

I think there should be an award out there for any scientist who 'blows the whistle' on him or herself.
 
OMFG all of those retracted papers were the basis for some lectures in one module and now I'll have to do a whole load of extra reading to find out what on earth's going on with that!!! :run:
 
If the scientific community has any heads not currently residing in backsides, then he should be regarded as great asset to scientific knowledge...
 
Time to stir up things a little bit. ;)


Chang wasn't proven wrong, another scientist just published a paper whose findings contradicted his. Yet Chang immediately adopted the position that he should assume his findings were the wrong ones, double-checked his own work and found the error almost right away.

I see this case from another point of view. Sorry but Chang was proven wrong. And it wasn't just one scientist who pointed the error, other biologists complained before that his structure didn't fit well with their biological results. His first and most importan paper has been cited more than 300 times, and some of the citations expressed some doubts about his structure. The Toronto team, for example, had a hard time to publish their results because they didn't fit well with Chang's structure.

It had to be not just another scientist, but another chrystalographer who pointed out the error for Chang to start thinking that he might be wrong. I have worked with some chrystalographic data taken from the Protein Data Bank, (http://www.rcsb.org/pdb/home/home.do a site that stores the structures of biomolecules resolved by either chrystalographic, NMR or modelling means) and, although I haven't worked that much with it, I have found a couple of mistakes. And believe me, it is hard for some chrystalographers to admit that their structures are wrong, specially if a non-chrystalographer points out the error. (One of the mistakes we found was the structure of an apo-enzime published on Science, and when you download the data file you see right away the His-tag in the binding site of the enzime, I simply cannot believe that the reviewers didn't catch that. The His-tag is too big of a molecule to remain unnoticed. But the data sheet doesn't say anything about that and the paper claims that it is an apo-enzime)

He sounds like he has exactly what it takes to be a great scientist.

Indeed, He is a great scientist, but maybe a bit too arrogant. Hopefully he will learn from this mistake and his career will be as good as it was until now.

My boss told us a story about a very well known chemist who become so arrogant and rude after receiving the Nobel Prize that the director of the institution he was working in had to tell him to behave properly. I cannot remember the name of the chemist, though. It shouldn't be very difficult to find out who was he.
 
My thoughts could not have been expressed more succintly :)

I think there should be an award out there for any scientist who 'blows the whistle' on him or herself.

Ask anyone who has struggled with an Excel spreadsheet that keeps giving the wrong answer... it's not that easy to find the error in an equation or model that you developed yourself ;) :goodjob: That's why we have peer review in the first place.
 
My boss told us a story about a very well known chemist who become so arrogant and rude after receiving the Nobel Prize that the director of the institution he was working in had to tell him to behave properly. I cannot remember the name of the chemist, though. It shouldn't be very difficult to find out who was he.

Sophie will no doubt bite my head off for saying this but... that's chemists for you :mischief: :lol:
 
Reviewers don't and shouldn't line for line check the researcher's code.

Usually the code of data analysis programs isn't even published. Reviewers are not expected to even receive it. Sometimes they jealously guard it from colleagues/competitors.

It's not even surprising to find investigators using some old piece of software that runs on an ancient workstation and whose source is long lost, hoping that said workstation continues working… I have one such case to deal with soon (rewrite the damn program and publish the source). :(
 
Back
Top Bottom