November 28, 2021

I, Science

The science magazine of Imperial College

Who really decides your medical treatment? The government wants it to be you, but are you really the best judge?

Lies, damned lies, and statistics. A household catchphrase referring to the deceptive nature of statistical studies. People are easily seduced by science when it is presented as logic and reason. After all, numbers shouldn’t lie. Or as Einstein put it: “Pure mathematics is, in its way, the poetry of logical ideas.”

We are constantly confronted with statistics now, from shampoo adverts (“70% of women agree”) to car insurance. But they are being overplayed and undermined. Statistics are a powerful psychological tool as well as a scientific one, but their credibility is questionable, and their motives range from pure to downright dirty.

One of the most emotive targets for evidentiary studies is medical treatment. We all seek to cure ailment, and when ill, there is often nothing people will not try as a last ditch attempt. The media is full of stories about promising new drugs, patients irrationally denied treatment, or the government letting people die because ‘treatment is too expensive’. But where do these decisions come from really? Who will get to decide in years to come when it is you facing senescence?

It’s a costly business

Drug trials are very, very expensive. On average more than £800 million is spent getting a drug to market, with no guarantee it will sell or something better won’t arrive in the mean-time. Before testing a drug, companies must be reasonably hopeful it will work. Furthermore they will only be testing certain questions. Does the drug improve the condition? Does it work better than current treatments? Is it safe?

Thus you already have two filters biasing results before a trial has even left the ground. Furthermore what happens when a drug works for many but causes unprecedented sudden death in a minority? Can it be licensed with a warning? Do you scrap it and deprive millions of a better life? Do you conduct yet another stinkingly expensive trial into investigating who is at risk?

Something like this happened with a class of NSAID (non steroidal anti-inflammatory drugs, e.g. aspirin, ibuprofen) where the aim was to seek a new NSAID that reduced the common side effect of gut ulcers, and the proffered drug, Vioxx, was hailed as a wonder. Incredibly tragic for the victims of the heart attacks that followed. How could something like this happen? Drug licensing was committed by the FDA, to whom just small studies focusing on the reduced gut side effects had been sent. No studies were done to investigate the possible complications and key data was also omitted. Was this malice, complacence or idleness? Most likely the well-wishing of researchers for a favourable outcome married with the pharmaceutical company’s desire to see a return.

Drug licensing grows ever stricter and it is often now beyond both the motivational and financial scope of pharmaceuticals to bother driving forward drugs that have meagre or shadowy evidence. Independently-funded studies simply lack spending power. You can be sure we will all miss out, but actually it seems to be the drugs that reach it to trial and are subsequently scrapped that attract the most grief in the public eye. Why are we denying cancer victims their last hope? You can be sure after spending millions nannying the drug to trial, its demise wouldn’t be undertaken lightly. For every angry patient there’s a boardroom representative with his head in his hands.
Increasingly in research, results are hailed as the next big thing if they are considered significant. Significant, being a whimsical term, is both a tool and a weapon. It is also one of the key criteria used in accepting new treatments.

In NICE we trust?

As mentioned previously, the way the maths works means that we always assume there might be an element of chance, and that the results could in fact be wrong. Generally speaking, studies use a 95% confidence interval. We accept that the result in a trial sample may differ slightly from the result were we to test every available candidate, past, present and future, but this is what we have to go on. 95% is pretty high, though mistakes will slip through the net. Particularly with any genetic studies, where variability is so wide the manipulation of statistics has to be delicately managed.

But even if we are nearly completely confident our results are real, a significant finding may still be completely useless. One study found that daily aspirin significantly reduced the risk of heart attacks in elderly men. However it also significantly raises the risk of bleeding. For every five heart attacks saved by aspirin, three haemorrhages occur. The net effect is two lives, but do we still trust this ratio when there’s a 5% error? What about in people who metabolise aspirin oddly, liver and kidney patients, what about genetic differences in blood clotting?

Fortunately there are quite a few systems in place that weigh up the pros and cons of different treatments, whilst also considering the limitations of a financial budget, in order to save the most lives overall. One such is called NICE – the National Institute for Clinical Excellence, and they receive a lot of flak. They are being increasingly marginalised in the current government, much like the rest of healthcare associates. Fortunately NICE only provide guidelines (i.e. strong suggestions). Doctors may act outside them if they are ready to stand up and defend why they are doing so. This happens more frequently in cases where drugs are awaiting NICE approval. NICE will also only recommend drugs where a certain amount of ‘gold standard’ studies have been accomplished. I think this is noble and discriminating – we wouldn’t buy a second hand car without an MOT, why on earth would we trust our lives and tax-money to a group of people who weren’t thorough in their vetting procedures. Unfortunately it does mean innovative new treatments may never see the light of day.

NICE recently lost its power to ban wildly expensive (and ambiguously effective) treatments. This has been particularly relevant to the palliative care groups and patient lobbies who understandably have a fierce interest in NICE’s licensing efforts. (Sometimes these groups are funded by the pharmaceutical company themselves – go figure). Quite a few last stage cancer treatments have been in the media recently, and now it is up to clinicians whether to prescribe them. This has been promoted as handing power back to the doctors on the frontline, but in reality it is a mess.

Seduction not empowerment

Clinicians do not have time to sit and read hundreds of papers, cross-correlate the maths, work out the risks and benefits of each treatment as pertains to the individual patient subgroup, assess the effects on longevity and quality of life on both patient and carers – as well maintaining an experienced grip on the state of the current economy and where else in the health service the same money could be spent. Do you really want a computer like this sat in front of you? Would you really want our palliative doctors to hand out undeniably expensive drugs that extend a painful life by a few more weeks because of the personal, emotional ricochet that would occur between doctor and patient if the doctor says no? I’ve heard cases where doctors have been emotionally blackmailed for antibiotics when the patient simply has flu.

No. It’s outrageous and has the potential to crack the doctor-patient relationship where it matters most. When the patient is the most vulnerable, emotions are high, and doctors are quite literally playing God.

Handing decision-making like this straight to clinicians isn’t empowering people. Politicians are simply seducing them.

There is an unapologetic need for national guidelines set out by a rational, apolitical committee. As guidelines, they streamline healthcare in a way that wouldn’t occur were decision making left to a few hundred thousand individuals. Furthermore doctors may set aside guidelines in hours of need when a new drug is breaking through. Lastly, when all else has failed and treatment is futile, it can be NICE, and not the doctor, the human, that becomes the villain. At the end of the day, we all need NICE to be bad.

Statistics, like politicians, should be handled with care.

More > Silence Isn’t Golden, In Trials We Trust, Podcast: Science in Parliament.