THE DEATH OF THE REFERENCE

[Guest guest]

http://www.redflagsweekly.com/kendrick/2004_feb17.html

 

 

THE DEATH OF THE REFERENCE

 

An Example From Heart Disease Research And Yet Another Reason Why The Scientific

Machine Is Close To Meltdown

 

By RFD Columnist Malcolm Kendrick MbChB, MRCGP

 

(email - malcolm )

 

References are very much a double edged sword, or perhaps a bazooka.

 

In the wrong hands they can do far more harm than good. And in the, essentially,

unchecked system that we now have, one careless reference can end up taking on a

life of its own. It gets stuck in the medical information ‘machine' replicating

itself like some malevolent computer virus, gradually infecting all data and

turning it into useless mush.

 

***

 

It is often said of statistics that scientists use them like a drunk uses a

lamp-post, for support rather than illumination.

 

I suppose the whole point of a scientific reference is that it is used,

primarily, to provide support, so I can't really complain about the lack of

illumination. However, I can complain about the fact that the abuse of

references has led to the point where I have found that, increasingly, I can't

be sure if what I read is true anymore.

 

As some readers will know I have a particular fascination with heart disease.

Some would call it an obsession, but I prefer the word, fascination. It's that

old Y chromosome thing. Anyway what I have found, as I have researched away, is

that a number of bold statements of fact, even those referenced to the gunnels,

when exposed to a bit of scrutiny, crumble to dust.

 

For example, at one stage I was interested why women, in most Western Countries,

suffer a much lower rate of heart disease than men - at least until about sixty

five or seventy. Women usually have very similar risk factors to men therefore,

according to conventional wisdom, there should be little difference in deaths

from heart disease.

 

I suspect I know what you are thinking. Women are protected against heart

disease by their sex hormones. This was what I used to think as well as well.

Probably because I had seen this fact stated so many times, in so many papers,

that I had been brainwashed into believing that it was true. This belief was

reinforced by the ‘knowledge' that female protection seemed to disappear not

long after the menopause, so I didn't really think to question it.

 

Nor, it seemed, did anyone else. Just to quote from one study, which accurately

reflected mainstream thinking a few years back:

 

'A protective effect of estrogen is the most obvious reason for the substantial

and consistent favored status of women vs. men with regard to coronary heart

disease.' Barrett-Connor E Atherosclerosis Dec 1995

 

However, I started to find out a number of facts about women and heart disease

that made me begin to question things. Firstly, it was clear that if you gave

female sex hormones to men, their rate of CHD increased dramatically. Which

doesn't prove anything for sure, but it does give pause for thought. After all,

there is no reason why a chemical that protects women shouldn't also protect

men.

 

Then I found that there were some populations where younger women suffered the

same rate of heart disease as men, such as women in Brazil. In addition to this,

the Framingham study showed that, when women developed type II diabetes their

relative protection against heart disease disappeared, even at a young age.

Diabetes doesn't wipe out sex hormones, so where did the protection go?

 

As this type of information began to pile up, I started to suspect that women

may not be protected by sex hormones after all. And so I was galvanised into

action and set out to track down the study, or studies, that had been carried

out proving that women are protected by sex hormones. In effect I went on the

quest to track down the original source of this hypothesis. Which I knew from

previous experience can be very hard work.

 

Just because it states in a recent scientific paper that female sex hormones are

protective, doesn't mean that the authors did a study. They are usually just

quoting from another paper, which has quoted from another paper, which quoted

from another paper, which quoted from… sometimes it seems ad-infinitum. Trying

to track backwards in time to find that very first study from which all other

studies sprang is not the work of a single day.

 

You can, it sometimes seems, find yourself late at night reading manuscripts

with illuminated script, penned by Monks in the early fourteenth century. ‘ And

it has been rightly noted by the good physics of this borough that the clutching

disease of the breast is more common in the male than the female, and that the

protection of fair lady folk be due to the most strange substance of eastrogenne

found in the most fecund of ye womanne .'

 

Well, not quite, but I once chased down a reference linking heart disease to

impotence and found the source reference from a study in Germany in 1928 (In

German).

 

Other times I have found that there is no source study at all. The whole thing

has sprung to life from thin air. On bad days I sometimes think that references

are no more than the scientific equivalent of rumour and gossip.

 

‘Ooooh, you'll never guess what, sex hormones protect against heart disease.'

 

‘Who told you that.'

 

‘The Mayo clinic did a study, I think. Or at least that's what Harvard said.'

 

‘Really?'

 

‘And I read it in the Lancet too.'

 

‘Oh well, it must be true. I'll have to go off and tell all my students.'

 

And then they'll go off and tell all their students. Some of whom will write

papers starting with the comment. ‘It is known that women are protected against

heart disease by their sex hormones.' Then other people will use their papers as

references, then…

 

After a while this rumour reaches the point where it becomes an unquestioned

truth because so many people have said it so many times, and it has been written

in hundreds of papers. In addition, experts stand up at conferences and

re-affirm it. Which means that when you write ‘Sex hormones protect women

against CHD,' you are then able to support this with hundreds and hundreds of

references from major journals?

 

But in this case, where was that original study, the source reference? What was

it all based on? Well, gentle readers, it was based on nothing at all. Because

there never was an original study. Or if there was, it is so damned well hidden

that I have never able to find it (he said covering himself from the inevitable

pedant who will no doubt triumphantly unfurl the 1936 trial done in

Serbia-Montenegro).

 

All I could find were papers referring to other papers that referred to other

papers. Yes, it is true that a number of studies were done which showed that

estrogen raised HDL levels and reduced LDL levels and protected the endothelium,

and all sorts of other ‘test-tube' effects. But in the end, if you wanted to

prove that female sex hormones protect against CHD, there are only two direct

ways to do it.

 

Remove sex hormones from younger women, and see if the rate of heart disease

goes up

Add sex hormones to women who no longer produce them, and see if the rate of

heart disease goes down.

 

Had anyone done this? No.

 

Actually I lie, the correct answer is yes. In 1963, a study was done in which

women who had hysterectomies were matched with women who had hysterectomies with

removal of both ovaries (no sex hormones). And the result of this study was…..?

 

As you might have guessed, the result of this study was that there was no

difference in the rate of heart disease between the two groups.

 

‘We found no difference in the prevalence of Coronary Heart Disease in the

oopherectomised (both ovaries removed) and hysterectomised (no ovaries removed)

women.' Ritterband et al Gonadal function and the development of CHD.

Circulation 1963 (2) pp 237 – 51

 

Okay, it's only one study, but it was totally negative, and no-one had ever done

a study that was positive. Yet, despite a complete lack of evidence, the sex

hormones theory had built to the point where millions of women around the world

were being prescribed HRT in order to prevent heart disease. All this, resting

on a piece of pure speculation that only became ‘fact' through the process of

endless repetition and cross-referencing.

 

And when, finally, researchers decided it was time to see if HRT did actually

protect against HRT, what did they find? Well, a few large studies were done,

and if I may quote from the New England Journal of Medicine on the matter:

 

‘Estrogen plus progestin does not confer cardiac protection and may increase the

risk of CHD among generally healthy postmenopausal women, especially during the

first year after the initiation of hormone use. This treatment should not be

prescribed for the prevention of cardiovascular disease.' Manson et al NEJM Aug

2003

 

Now then, at this point, some of you may have picked up on the gentle irony that

I am using references to support my argument that references are a load of

rubbish. But of course, references used properly, are a good thing. (I mean the

way I use them, or course)

 

However, references are very much a double edged sword, or perhaps a bazooka.

 

In the wrong hands they can do far more harm than good. And in the, essentially,

unchecked system that we now have, one careless reference can end up taking on a

life of its own. It gets stuck in the medical information ‘machine' replicating

itself like some malevolent computer virus, gradually infecting all data and

turning it into useless mush.

 

I don't know what the answer to this problem is. Wipe the whole database clean

and start again? Set up a system to hunt down and check all references, and

remove those that are found to be wrong. Then remove all references to those

references. The mind boggles at the size of that task.

 

In the meantime, until someone does something (he said passing the buck), I find

that when I want to know what the truth may be, or to get as close to the truth

as is possible, the only solution is to go back, get the original paper used as

a reference, and read it for myself. Which is an enormous time consuming pain.

But I believe that references are now so badly corrupted that it is virtually

impossible to trust them, or the papers based on them, anymore. In effect this

means that the entire medical research machine is close to meltdown. If it

hasn't melted down already.

 

In the meantime, remember that the truth is out there. It is just extremely

difficult to know what it is any more.

 

 

 

 

 

Finance: Get your refund fast by filing online