For Science's Gatekeepers, a Credibility Gap

[Guest guest]

SSRI-Research@

Tue, 02 May 2006 22:20:11 -0000

[sSRI-Research] Science's Gatekeepers, a Credibility Gap_Dr.

Lawrence Altman, NYT

 

 

 

 

 

ALLIANCE FOR HUMAN RESEARCH PROTECTION (AHRP)

Promoting Openness, Full Disclosure, and Accountability

http://www.ahrp.org/cms/

 

FYI

 

Wouldst that New York Times science reporters would learn how to

critically report about science and medicine as does Dr. Lawrence

Altman. " For Science's Gatekeepers, a Credibilty Gap, " is an

unflinching, sobering appraisal of science journal editors'

performance--they don't make the grade.

 

" Virtually every major scientific and medical journal has been humbled

recently by publishing findings that are later discredited. The

flurry of episodes has led many people to ask why authors, editors

and independent expert reviewers all failed to detect the problems

before publication. "

 

This persuasively argued critique goes beyond the headlines and the

self-serving rationalizations of the fraternity of journal editors

who demur when asked why they don't establish meaningful review

mechanisms to prevent fraudulent articles from passing peer review.

Altman lifts the curtain to reveal that the scientific fraternity has

vested interests in maintaining the illusion that " passing peer

review is the scientific equivalent of the Good Housekeeping seal of

approval. "

 

Instead, Altman lets lay readers in on the intricate interwoven

conflicting interests--financial and political--that the system

protects.

 

" They include economic pressures for journals to avoid investigating

suspected errors; the desire to avoid displeasing the authors and the

experts who review manuscripts; and the fear that angry scientists

will withhold the manuscripts that are the lifeline of the journals,

putting them out of business.By promoting the sanctity of peer review

and using it to justify a number of their actions in recent years,

journals have added to their enormous power. The release of news

about scientific and medical findings is among the most tightly

managed in country. Journals control when the public learns about

findings from taxpayer-supported research by setting dates when the

research can be published. They also impose severe restrictions on

what authors can say publicly, even before they submit a manuscript,

and they have penalized authors for infractions by refusing to

publish their papers. "

 

Scientists, academic institutions, the pharmaceutical industry, and

the journals are all invested in " keeping up appearances " but not the

integrity of science. Journal editors recoil at suggestions that the

reviewing process for scientific reports itself be tested in

accordance with scientific methods. They reject conducting random

audits " like those used to monitor quality control in medicine, "

citing costs and " the potential for creating distrust. "

 

Such excuses betray an underlying elitism among this elitist

fraternity that has still not woken up to the fact that the products

of its endeavors are tainted-just as the blockbuster drugs that have

enriched all those involved, are defective.

 

Altman gently but surely disabuses lay persons of their illusion:

 

" A widespread belief among nonscientists is that journal editors and

their reviewers check authors' research firsthand and even repeat the

research. In fact, journal editors do not routinely examine authors'

scientific notebooks. Instead, they rely on peer reviewers'

criticisms, which are based on the information submitted by the

authors. "

 

Whether intentionally or not, Altman underscores the inferior

performance of U.S. journal editors compared to British journal, by

giving the last word to Dr. Richard Smith, former (longtime) editor

of the BMJ (British Medical Journal) and Dr. Richard Horton, editor

of The Lancet.

 

" Journals have devolved into information-laundering operations for the

pharmaceutical industry...journals rely on revenues from industry

advertisements. But because journals also profit handsomely by

selling drug

companies reprints of articles reporting findings from large clinical

trials

involving their products, editors may 'face a frighteningly stark

conflict

of interest' in deciding whether to publish such a study. "

 

 

 

Contact: Vera Hassner Sharav

212-595-8974

veracare

 

 

 

 

 

http://www.nytimes.com/2006/05/02/health/02docs.html?_r==1 & oref==slogin & pagewant\

ed==all

 

 

The New York Times

The Doctor's World

 

For Science's Gatekeepers, a Credibility Gap

By LAWRENCE K. ALTMAN, M.D.

Published: May 2, 2006

 

Recent disclosures of fraudulent or flawed studies in medical and

scientific journals have called into question as never before the merits of

their peer-review system.

The system is based on journals inviting independent experts to

critique submitted manuscripts. The stated aim is to weed out sloppy and bad

research, ensuring the integrity of what it has published.

 

Because findings published in peer-reviewed journals affect patient

care, public policy and the authors' academic promotions, journal editors

contend that new scientific information should be published in a peer-reviewed

journal before it is presented to doctors and the public.

That message, however, has created a widespread misimpression that

passing peer review is the scientific equivalent of the Good Housekeeping seal

of approval.

Virtually every major scientific and medical journal has been humbled

recently by publishing findings that are later discredited. The

flurry of episodes has led many people to ask why authors, editors and

independent expert reviewers all failed to detect the problems before

publication.

 

The publication process is complex. Many factors can allow error, even

fraud, to slip through. They include economic pressures for journals

to avoid investigating suspected errors; the desire to avoid displeasing the

authors and the experts who review manuscripts; and the fear that angry

scientists will withhold the manuscripts that are the lifeline of the journals,

putting them out of business.By promoting the sanctity of peer review and using

it to justify a number of their actions in recent years, journals have added to

their enormous power.

 

The release of news about scientific and medical findings is among

the most tightly managed in country. Journals control when the public learns

about findings from taxpayer-supported research by setting dates when the

research can be published. They also impose severe restrictions on what authors

can say publicly, even before they submit a manuscript, and they have penalized

authors for infractions by refusing to publish their papers.

Exceptions are made for scientific meetings and health emergencies.

 

But many authors have still withheld information for fear that

journals would pull their papers for an infraction. Increasingly, journals and

authors' institutions also send out news releases ahead of time about a

peer-reviewed discovery so that reports from news organizations coincide with a

journal's date of issue.

 

A barrage of news reports can follow. But often the news release is

sent without the full paper, so reports may be based only on the spin

created by a journal or an institution.

Journal editors say publicity about corrections and retractions

distorts and erodes confidence in science, which is an honorable business.

Editors also say they are gatekeepers, not detectives, and that even though peer

review is not intended to detect fraud, it catches flawed research and improves

the quality of the thousands of published papers.

 

However, even the system's most ardent supporters acknowledge that

peer review does not eliminate mediocre and inferior papers and has never passed

the very test for which it is used. Studies have found that journals publish

findings based on sloppy statistics. If peer review were a drug, it would never

be marketed, say critics, including journal editors.

None of the recent flawed studies have been as humiliating as an

article in 1972 in the journal Pediatrics that labeled sudden infant death

syndrome a hereditary disorder, when, in the case examined, the real cause was

murder.

 

Twenty-three years later, the mother was convicted of smothering her

five children. Scientific naïveté surely contributed to the false

conclusion, but a forensic pathologist was not one of the reviewers. The faulty

research in part prompted the National Institutes of Health to spend millions of

dollars on a wrong line of research.

 

Fraud, flawed articles and corrections have haunted general interest

news organizations. But such problems are far more embarrassing for

scientific journals because of their claims for the superiority of their system

of editing.

 

A widespread belief among nonscientists is that journal editors and

their reviewers check authors' research firsthand and even repeat the

research. In fact, journal editors do not routinely examine authors' scientific

notebooks. Instead, they rely on peer reviewers' criticisms, which are based on

the information submitted by the authors.

While editors and reviewers may ask authors for more information,

journals and their invited experts examine raw data only under the most unusual

circumstances.

 

In that respect, journal editors are like newspaper editors, who

check the content of reporters' copy for facts and internal inconsistencies but

generally not their notes. Still, journal editors have refused to call peer

review what many others say it is - a form of vetting or technical editing.

 

In spot checks, many scientists and nonscientists said they believed

that editors decided what to publish by counting reviewers' votes. But

journal editors say that they are not tally clerks and that decisions to publish

are theirs, not the reviewers'.

 

Editors say they have accepted a number of papers that reviewers have

harshly criticized as unworthy of publication and have rejected many

that received high plaudits.

Many nonscientists perceive reviewers to be impartial. But the

reviewers, called independent experts, in fact are often competitors of the

authors of the papers they scrutinize, raising potential conflicts of interest.

 

Except when gaffes are publicized, there is little scrutiny of the

quality of what journals publish.

 

Journals have rejected calls to make the process scientific by

conducting random audits like those used to monitor quality control in medicine.

The costs and the potential for creating distrust are the most commonly cited

reasons for not auditing.

 

In defending themselves, journal editors often shift blame to the

authors and excuse themselves and their peer reviewers.

Journals seldom investigate frauds that they have published,

contending that they are not investigative bodies and that they could not afford

the costs.

Instead, the journals say that the investigations are up to the

accused authors' employers and agencies that financed the research.

 

Editors also insist that science corrects its errors. But corrections

often require whistle-blowers or prodding by lawyers. Editors at The New England

Journal of Medicine said they would not have learned about a problem that led

them to publish two letters of concern about omission of data concerning the

arthritis drug Vioxx unless lawyers for the drug's manufacturer, Merck, had

asked them questions in depositions. Fraud has also slipped through in part

because editors have long been loath to question the authors.

 

" A request from an editor for primary data to support the honesty of

an author's findings in a manuscript under review would probably poison the air

and make civil discourse between authors and editors even more difficult than it

is now, " Dr. Arnold S. Relman wrote in 1983. At the time, he was editor of The

New England Journal of Medicine, and it had published a fraudulent paper.

 

Fraud is a substantial problem, and the attitude toward it has changed

little over the years, other editors say. Some journals fail to

retract known cases of fraud for fear of lawsuits.

Journals have no widely accepted way to retract papers, said Donald

Kennedy, editor in chief of Science, after the it retracted two papers by the

South Korean researcher Dr. Hwang Woo Suk , who fabricated evidence that he had

cloned human cells.

 

In the April 18 issue of Annals of Internal Medicine, its editor, Dr.

Harold C. Sox, wrote about lessons learned after the journal retracted an

article on menopause by Dr. Eric Poehlman of the University of Vermont.

 

When an author is found to have fabricated data in one paper,

scientists rarely examine all of that author's publications, so the scientific

literature may be more polluted than believed, Dr. Sox said.

Dr. Sox and other scientists have documented that invalid work is not

effectively purged from the scientific literature because the authors

of new papers continue to cite retracted ones.

 

When journals try to retract discredited papers, Dr. Sox said, the

process is slow, and the system used to inform readers faulty. Authors often use

euphemisms instead of the words " fabrication " or " research

misconduct, " and finding published retractions can be costly because some

affected journals charge readers a fee to visit their Web sites to learn about

them, Dr. Sox said.

 

Despite its flaws, scientists favor the system in part because they

need to publish or perish. The institutions where the scientists work and the

private and government agencies that pay for their grants seek

publicity in their eagerness to show financial backers results for their

efforts.

 

The public and many scientists tend to overlook the journals' economic

benefits that stem from linking their embargo policies to peer

review. Some journals are owned by private for-profit companies, while others

are owned by professional societies that rely on income from the journals. The

costs of running journals are low because authors and reviewers are generally

not paid.

 

A few journals that not long ago measured profits in the tens of

thousands of dollars a year now make millions, according to at least three

editors who agreed to discuss finances only if granted anonymity, because they

were not authorized to speak about finances.

 

Any influential system that profits from taxpayer-financed research

should be held publicly accountable for how the revenues are spent. Journals

generally decline to disclose such data.

Although editors of some journals say they demand statements from

their editing staff members that they have no financial conflicts of

interest, there is no way to be sure. At least one editor of a leading American

journal had to resign because of conflicts of interest with industry.

 

Journals have devolved into information-laundering operations for the

pharmaceutical industry, say Dr. Richard Smith, the former editor of

BMJ, the British medical journal, and Dr. Richard Horton, the editor of The

Lancet, also based in Britain.

 

The journals rely on revenues from industry advertisements. But

because journals also profit handsomely by selling drug companies reprints of

articles reporting findings from large clinical trials involving their products,

editors may " face a frighteningly stark conflict of interest " in deciding

whether to publish such a study, Dr. Smith said.

 

 

FAIR USE NOTICE: This may contain copyrighted (© ) material the use

of which has not always been specifically authorized by the copyright owner.

Such material is made available for educational purposes, to advance

understanding of human rights, democracy, scientific, moral, ethical,

and social justice issues, etc. It is believed that this constitutes

a 'fair use' of any such copyrighted material as provided for in Title 17 U.S.C.

section 107 of the US Copyright Law. This material is distributed without

profit.

 

 

 

 

 

 

 

 

 

 

 

 

Drug-Free School Zone? Just Say NO to Prozac for Children.