Treatment decisions based on QALYs are seriously flawed and must not be used in Europe, say researchers

European project shows QALY approach to funding medical treatment should be abandoned

Patients denied access to life-saving treatments due to flawed reasoning

The Quality Adjusted Life Years approach to deciding what medical treatments are available on health services – used by the UK, Canada and Australia and generating much interest across Europe – is dangerously flawed and should be abandoned, according a European Commission funded research project.

The results of the project, announced today (25th January) at a Brussels conference on health outcomes in Europe, highlights huge failings in the way some Health Technology Assessment (HTA) agencies notably the UK s National Institute for Health and Clinical Excellence (NICE), recommend which treatments be made available on health services, and warns against this system being adopted across Europe.

HTA agencies are charged with recommending whether new treatments should be publicly funded. NICE, perhaps Europe s most well known HTA agencies, uses Quality Adjusted Life Years (QALY). QALY is an economic theory which mathematically weighs the number of life years and the improvement to quality of life provided by different treatments. On the basis of this calculation, recommendations are made about whether treatments are offered by the UK s National Health Service (NHS). In the UK, if the incremental cost per QALY (cost of one additional year in perfect health) is below £30,000 then the treatment is usually recommended to be accessible to patients. Whilst Germany has explicitly rejected it, many European countries are currently considering replicating the NICE model.

he QALY method is based on four key assumptions about how patients rate additional life and different health states. ECHOUTCOME, a three year, €1m, European Commission-funded project, brought together six universities and research agencies to investigate these four assumptions, finding them all to be invalid.

The research surveyed 1,300 respondents in Belgium, France, Italy and the UK, and is the largest investigation into QALYs ever undertaken. It used a questionnaire designed by medical and economics experts and carried out by trained interviewers to test the assumptions.

The responses varied hugely across the population showing that the way people rate different medical outcomes is personal, and cannot be summed up by a neat formula. The project concludes that QALY is therefore an invalid way to make medical decisions and should be replaced by a more flexible, case by case approach.

The four assumptions invalidated by the research are:

Assumption 1: Time and quality of life can be measured in consistent intervals. This is true for life years, as two years is twice as long as one year. But quality of life is personal and different people rank different conditions as having a greater or worse impact. The QALY system which grades different conditions and assigns them a number cannot be said to reflect how real people view these conditions.

Assumption 2: Life years and quality of life are linked. This assumes that someone who prefers 10 years of healthy life to five years, will also prefer 10 years in a wheelchair to five years. The results showed that this does not reflect how real people feel.

Assumption 3: People are neutral about risk. The results showed most people are either risk averse or willing to take risk. It was very rare to find someone was neutral. The assumption that people are neutral can lead to decisions which completely distort the desires of patients.

Assumption 4: Willingness to sacrifice life years is constant over time. This assumption suggests an individual willing to trade off five years in a remaining 25 years (20% of life) for better health, will also give up two years in 10. The research showed that willingness to sacrifice time for life quality over different periods of time varied enormously.

Gerard Duru, Emeritus Research Director in Mathematics at the French National Centre of Scientific research (CNRS) said: “The underlying assumptions of the QALY outcome are very theoretical and are not verified in a real population. The QALY indicator is not a valid scientific scale. It is impossible to know what we are measuring, and therefore impossible to base a formula upon it. To be able to trust this formula, all four of these assumptions must be validated. If they aren t we don t have the right to use it.”

The researchers point to examples where NICE has made negative appraisals of innovative medicines based on this flawed assessment method. NICE had assessed that some biologic drugs could be used for the management of rheumatoid arthritis, but not a new one – abatacept. This was based on one QALY assessment that said it was over the threshold, whilst other QALY assessments, and other cost-effectiveness approaches, had shown this new treatment to be cost-effective. This means that should conventional anti-rheumatic drugs stop working or cause major side effects, as drugs may sometimes do, some patients may not be switched to this innovative medicine although it had been shown to provide a clinically safe, effective and cost-effective alternative.

Ariel Beresniak, CEO of Data Mining International and ECHOUTCOME Project Leader said: “The research provides robust scientific evidence that QALYs produce hugely inconsistent, wrong results, on which important decisions are being made.”

“Agencies such as NICE should abandon QALY in favour of other approaches. European HTAs currently looking to adopt the NICE model must seriously reconsider”.

Alastair Kent, Director of Genetic Alliance UK, a group working to improve the lives of people affected by genetic conditions, says: “We recognise that there is a limited budget and tough decisions to be made. But the QALY system in its current form is an inadequate, incomplete measure which neglects important issues. Any system for allocating medical resources must carry confidence of clinicians, patients and society at large, and the current system does not.”

The consortium recommends that QALY should never be used in healthcare decisions. Beresniak says “Each case is different and each should use adequate evaluation tools. There should be list of adequate validated tools to make these decisions with guidance on when to use each, instead of one non-validated tool.”

For example a cost per remission approach would be welcome for drugs targeting Rheumatoid Arthritis, which could then be issued along with guidance as to when a patient can be granted the drug. This is a clear, scientifically robust assessment as remission rate can be defined and cost calculated simply and quickly.

In other cases a cost-benefit approach is appropriate, ie the cost of disease avoided by investing in wide scale vaccinations.

Beresniak says “The pro-QALY lobby will respond that such a system doesn t compare different treatments for different diseases. But we never need to do this in real life medical situations.”

About the consortium
ECHOUTCOME (European Consortium in Healthcare Outcomes and Cost-Benefit research) is made up of experts in healthcare, mathematics and economics from Data Mining International (Switzerland – Project leader), the University of Bocconi (Italy), the Université Libre de Bruxelles (Belgium), the French Society of Health Economics (SFES), Cyklad Group (France), Lyon Ingénierie Projets and the Claude Bernard University (France).

About QALY
The QALY is a formula calculated from additional life expectancy and a measure of the quality of the remaining life-years that would be added by the medical intervention.

Each year in perfect health is assigned the value of 1.0, down to a value of 0.0 for being dead. If the extra years would not be lived in full health then the extra life-years are given a value between 0 and 1 to account for this.

There are many different ways to calculate quality of life. A number of factors are considered including the level of pain the person is in, their mobility and their general mood.

A value is then assigned which is multiplied by the number of additional life years to reach the QALY value. . If the incremental cost per QALY (= cost of one additional life year in perfect health) is below £30,000 then the treatment is usually recommended by NICE to be reimbursable from the NHS.

Most of the cases, data to assess quality of life is mapped from published studies of how people rank different life situations, rather than individual tests for each drug. Much of the process is speculative, and the different methods will produce different results.

Results from pharma companies tend to come in below £30,000 per QALY. Results from NICE studies, where they feel the need to reappraise pharma results, tend to be over the threshold.

How a QALY is calculated (from the NICE website):

Patient x has a serious, life-threatening condition.

If he continues receiving standard treatment he will live for 1 year and his quality of life will be 0.4 (0 or below = worst possible health, 1= best possible health)

If he receives the new drug he will live for 1 year 3 months (1.25 years), with a quality of life of 0.6.
The new treatment is compared with standard care in terms of the QALYs gained:
Standard treatment: 1 (year s extra life) x 0.4 = 0.4 QALY
New treatment: 1.25 (1 year, 3 months extra life) x 0.6 = 0.75 QALY

Therefore, the new treatment leads to 0.35 additional QALYs (that is: 0.75 -0.4 QALY = 0.35 QALYs)
The cost of the new drug is assumed to be £10,000, standard treatment costs £3000.

The difference in treatment costs (£7000) is divided by the QALYs gained (0.35) to calculate the cost per QALY. So the new treatment would cost £20,000 per QALY.

Back to Press Releases