What is an Economic Evaluation?

Economic evaluation is the comparative analysis of one or more technologies to determine which technology provides more value. In the context of health economic evaluation, the value is determined through analysing the costs of the technology against the benefits it provides to patients.

Traditionally, cost-effectiveness evaluations are performed after late-stage clinical trials to assess both the economic impact and clinical effectiveness of a technology. However, assessing the economic benefits earlier can help inform manufacturers on whether the technology will be commercially viable later down the line.

Economic Evaluations typically consist of several phases:

1. Defining the Scope and Objective:

• Define the population, intervention, comparator and outcomes, timing & setting (PICOTS) for the technology
• Understand the current “as-is” state without the technology
• Map the “to-be” state following introduction of the technology

2. Data Collection & Model Conceptualisation

• Gather data on costs (development, implementation, maintenance) and outcomes (clinical effectiveness, user satisfaction).
• Form a concept for the model based on preferred modelling techniques for the individual situation

3. Modelling:

• Develop the health economic model
• Compare predicted costs and benefits using standardised comparative metrics

4. Analysis:

• Use statistical modelling techniques to predict long-term costs and benefits.
• Perform quantitative analysis on the costs and benefits of the technology; including sensitivity analysis, scenario analysis, costed risk analysis and optimism bias.

5. Reporting

• Write up of the results of the analysis
• Evaluate health outcomes using data generated using the model

What are the Key Differences with Digital Technologies compared to Pharmaceuticals and Medical Devices

Digital health technologies, including digital technologies such as telemedicine and mobile health apps, differ in several ways from pharmaceuticals and medical devices.

For example, DHTs often have lower initial costs but require ongoing updates and maintenance.

Pharmaceuticals typically have fixed dosing schedules, whereas DHTs may offer scalable solutions that adapt over time.

Additionally, DHTs often involve continuous user interaction through apps or platforms, while drugs and devices usually have more passive roles.

Understanding these differences is crucial for a robust evaluation:

1. Initial Costs: Lower for DHTs but higher for pharmaceuticals.

2. Maintenance: Ongoing for DHTs; minimal for most pharmaceuticals.

3. Scalability: High for DHTs due to software updates; limited for physical devices.

4. User interaction: High for DHTs, as needed for pharmaceuticals and devices.

Key Features of Economic Evaluations for Digital Health Technologies (DHTs)

Within the context of the economic modelling, there are several key considerations for digital health interventions vs pharmaceuticals and medical devices (See Figure 1 below):

• The comparator for a digital health intervention is not always well defined and there can be a combination of alternative treatment options

• The product often evolves fast with user feedback and requires frequent updates

• Active user input (patient or doctor) is always required for DHT to be used as intended

• DHT is often provided at scale through subscription based pricing models. The unit price is the marginal cost, which tends to zero

• DHTs typically lead to diffused and less clearly defined health and non-health benefits

• Non health impacts are often significant, such as productivity impacts, irrespective of the setting

• The standard cost per QALY metric is unlikely to reflect the broad range of health and non-health impacts of the DHT

Key differences between pharmaceuticals or medical devices and digital health interventions (DHIs). Gomes et al. (2022).

Recommendations when Evaluating Digital Health Technologies

• Take a broader costing perspective. Most DHT evaluations are done with the payer/provider costing perspective in mind (69.4%), however a growing number take the societal perspective (52.4%). To read more about the different types of costing perspective that can be taken, see our guide to measuring the costs of DHTs.

• Consider both digital and non-digital comparators and whether DHI replaces or complements existing technology

• Account for the rapid evolution of DHT and its impacts on costs and benefits, and the timing of the analysis

• Consider user time (costs) and user experience (benefits)

• Development costs are not always included in cost analysis. Mean cost per user should be based on the eligible population and expected uptake rates

• Include non-health benefits, both to patients and other parties (e.g. health professionals, carers)

• Consider all relevant impacts outside the health care sector as part of an ‘impact inventory’.

• Cost-consequence analysis is likely to be most suitable and in line with an impact inventory

What are the 5 Types of Economic Evaluation for Digital Health Technologies (DHTs)?

Cost-Effectiveness Analysis (CEA)

Compares the relative costs and outcomes of different interventions. For example, CEA might compare two diabetes management apps to determine which provides better glucose control per dollar spent.

One of the key advantages of cost-effectiveness analysis (CEA) is that clinical outcomes are relatively straightforward to measure, especially when the economic evaluation is linked to a clinical trial. Additionally, CEA provides an assessment of alternative options based on disease-specific measures of health effects, making it particularly useful for evaluating interventions within a specific medical context.

See Figure 2 below for an example of how cost and outcomes would be presented in a cost effectiveness analysis.

A cost-effectiveness analysis (CEA) should be used when you aim to assess the value for money of your digital product compared to alternative options using the same unit of effect, typically related to a specific disease. This method is particularly useful when the benefits of using your product are primarily health-related and clinical measures are the most appropriate way to capture these health benefits. The incremental cost effectiveness ratio (ICER) can be calculated using the difference in costs and outcomes between the intervention and comparator, providing a standardised approach to facilitate comparison between different interventions.

Cost-Utility Analysis (CUA)

Measures outcomes based on quality-adjusted life years (QALYs). This analysis is valuable when assessing interventions that aim to improve quality of life, such as mental health apps.

CUA has two main advantages over other forms of economic evaluation:

1. It uses measures that combine length and quality-of-life.

2. The use of a standardised, generic measure enables economic comparisons across different disease areas with very different clinical outcomes.

Consequently, decision-makers, such as NICE in the UK, prefer this method because it facilitates choices about budget allocation across the whole spectrum of disease areas.

In a systematic review of 35 published economic evaluations for digital health interventions, it was found that only 22.9% of economic evaluations had conducted a CUA.

In the same review, several questionnaires and surveys were used to estimate QALYs, including two key generic measures:

• European Quality of Life 5-Dimensions (EQ-5D)

• ShortForm-6 Dimensions

See Figure 3 below for an example of how costs (blue) and outcomes (green) may be presented in a CUA. The cost effectiveness ratio (CER) (orange) can be calculated for both the intervention and comparator (light orange) and the incremental cost effectiveness ratio (ICER) (dark orange) can be calculated using the difference.

ICER threshold levels are used by health technology assessment (HTA) bodies and payers globally to determine whether to reimburse a new intervention or not.

A cost-utility analysis (CUA) should be used when you need to assess the value for money of a digital product that might be funded by the NHS. This method is particularly important when establishing the cost-effectiveness of a product that requires a significant financial commitment from the payer. Additionally, CUA is ideal when both the quantity and quality of life are important dimensions of the health benefits derived from using your product.

Cost-Benefit Analysis (CBA)

Cost-benefit analysis (CBA) involves comparing interventions by expressing both costs and benefits, including health outcomes, in monetary terms. This allows for the evaluation of treatment alternatives through the net monetary benefit (NMB), which is derived by subtracting the costs from the monetary value of the benefits for each treatment.

Monetary valuations of benefits are typically obtained through willingness to pay (WTP) surveys or discrete choice experiments (DCEs). However, CBA is not commonly applied in HTA due to the inherent difficulty in assigning monetary values to health outcomes such as increased survival.

Instead, CBAs are most often employed for evaluating large capital projects, like new hospital facilities, or interventions aimed at improving waiting times or access to services.

Cost-benefit analysis (CBA) presents both cost and outcome data in monetary terms within a descriptive table, facilitating easy comparison between a new health technology and its alternatives (see Figures 4 & 5). By converting all effects to monetary values, CBA can account for non-health benefits alongside the health effects of the digital product.

CBA studies often include non-health benefits such as:

• Financial benefits from cost savings

• Indirect benefits from productivity gains

• Intangible benefits related to wellbeing and convenience

When performing a CBA over a particular time horizon, the total cost and benefit differences over the horizon can be plotted graphically to determine the point at which you will break even. In breakeven analysis performed on health technologies, this will show you which year the interventions’ benefits outweigh the costs.

Consider a cost-benefit analysis when you need to evaluate whether your digital product is worth the investment compared to alternative products, especially in scenarios where non-health benefits constitute a significant component of the total effects of using your digital product.

Cost Consequence Analysis (CCA)

One tool used for economic evaluation is cost-consequence analysis (CCA). This method compares the costs, such as treatment and hospital care, and the consequences, such as health outcomes, of a test or treatment with a suitable alternative.

Unlike cost-benefit analysis (CBA) or cost-effectiveness analysis (CEA), CCA does not summarise outcomes in a single measure like quality-adjusted life years (QALYs) or monetary terms.

Instead, outcomes are presented in their natural units, which may include monetary values. Decision-makers then determine whether the overall treatment is worth carrying out based on these detailed comparisons.

The CCA approach offers several key benefits:

• Helps refine economic methods

• Identifies relevant costs and outcomes

• Generates hypotheses for definitive cost-effectiveness studies

• Provides a broader and richer source of economic information increasingly needed by NHS decision makers

CCA presents cost and outcome data alongside each other in a descriptive table that allows easy comparison of a new health technology and its comparator(s) (see Figure 6).

This is especially useful in situations where complexity in the research design might otherwise be pervasive, such as comparing the costs and consequences of different models of care across a care pathway in an observational study.

CCAs may also be particularly useful in feasibility or pilot studies when it is not clear which costs and outcomes will be most relevant to future definitive trials. Given the limited funding available for feasibility studies and the scarcity of health economists, CCA can provide a less resource intensive alternative if interventions have important economic consequences or a full comparative analysis is premature, but still provide an opportunity to pilot instruments used to collect economic data such as resource use and health-related quality of life.

Budget Impact Analysis (BIA)

Budget Impact Analysis (BIA) provides an understanding of both the costs incurred and the savings achieved by implementing a technology, thereby allowing for an estimation of the financial impact on the decision maker’s budget.

While BIAs are a go-to model for digital health technologies (DHTs), they do not measure the effectiveness of a technology in terms of health outcomes, so are typically performed in conjunction with cost-effectiveness analyses (CEAs) or cost-utility analyses (CUAs), often once a technology is already deemed cost-effective.

BIAs estimate the likely change in expenditure for a specific budget holder, typically calculating the net budget impact over a period of 3 to 5 years, either at a national level or for local healthcare payers and providers.

Unlike CEAs, which estimate value for money, BIAs assess affordability by comparing two scenarios:

1. One where the new intervention or policy is implemented
2. One where it is not

Each scenario considers factors such as:

• Population size
• Patient eligibility (eligible population)
• Patients that receive intervention (effective population)
• Speed of uptake
• Market share of the intervention
• Costs (direct, indirect, intangible)

BIA presents cost data for both scenarios in a descriptive table that allows easy comparison of the impact of implementing a health technology vs not implementing the technology (see Figure 7).

A budget impact analysis (BIA) should be used when you need to assess the potential financial impact of your product before its implementation. This method is also essential for determining whether your product will be affordable within the decision maker’s budget constraints if it is recommended for us.

How to Choose the Most Appropriate Evaluation Method

What is the Importance of Economic Evaluation for Digital Health Technologies

The implications of economic evaluations extend beyond cost savings. They can influence policy decisions, reimbursement rates, and investment opportunities. Proper evaluations can also identify the effectiveness of digital health interventions, influencing policy decisions and investment opportunities. These evaluations can highlight potential areas for improvement in digital health solutions.

For instance:

1. Policy Decisions: Governments can prioritise funding for cost-effective technologies.

2. Reimbursement Rates: Insurers may adjust coverage based on demonstrated value.

3. Investment Opportunities: Investors may prefer technologies with proven economic benefits.

Choosing the right type of economic evaluation is crucial to accurately capture the value of digital health technologies and support informed decision-making.

Performing economic evaluations for DHTs can highlight cost-saving opportunities and justify further investment, including further service evaluations, clinical trials and business cases

Selecting the Method that is Right for You

Selecting the appropriate evaluation method is crucial and depends on the specific context and objectives of your assessment. Below are detailed explanations of each method to help digital health CEOs determine the best approach for evaluating their technology:

1. Cost-Effectiveness Analysis (CEA):

   • Use When: Comparing similar interventions that have clear clinical outcomes.

   • Details: CEA measures the cost per unit of health outcome, such as cost per life year gained or cost per case prevented. This method is particularly useful when decision-makers need to choose between multiple healthcare interventions targeting the same disease or condition.

2. Cost-Utility Analysis (CUA):

   • Use When: Evaluating quality of life improvements across different health conditions.

   • Details: CUA uses quality-adjusted life years (QALYs) or disability-adjusted life years (DALYs) to measure outcomes. This method is ideal when the health benefits of an intervention extend beyond simple clinical measures and include broader impacts on patients’ quality of life.

3. Cost-Benefit Analysis (CBA):

   • Use When: Converting all outcomes to monetary values for broad comparisons.

   • Details: CBA compares the costs and benefits of an intervention, both expressed in monetary terms, allowing for a straightforward comparison across various sectors. This method is useful when non-health benefits, such as financial savings or productivity gains, are significant considerations.

4. Cost-Consequence Analysis (CCA):

   • Use When: Conducting early economic evaluations to decide whether to pursue a technology.

   • Details: CCA presents both costs and outcomes in their natural units without aggregating them into a single measure. This allows decision-makers to see a detailed breakdown of benefits and costs, making it easier to understand the trade-offs involved in adopting a new technology.

5. Budget Impact Analysis (BIA):

   • Use When: Providing a robust comparison of scenarios with and without the technology.

   • Details: BIA estimates the financial impact on a specific budget holder, such as a healthcare provider or payer, resulting from the adoption of a new technology. It typically involves modelling over a period of 3 to 5 years and considers factors like population size, patient eligibility, and market share. BIA is essential for assessing affordability and planning purposes at both local and national levels.

How Healthonomix can support you in your decision

If you would like to speak about which economic evaluation method is best tailored to your situation then please contact us