Conducting a high-quality economic analysis and evaluation is vital for digital health companies; well-documented economic benefits can differentiate your technology from competitors and prove its value to payers. There are three important scenarios where an economic evaluation must be watertight:
- Demonstrating early economic viability of a technology. From a commercial point of view, this can help secure funding, showcase potential returns, and build investor confidence.
- Performing a comprehensive evaluation for regulatory approval. Thorough economic evaluations ensure compliance with regulatory requirements, which facilitates smoother market access.
- Identifying cost savings is crucial for adoption within healthcare systems. Providers want to provide the best care for the lowest price, and economic evaluations can highlight efficiencies, building trust among healthcare providers, physicians, and patients.
However, with a number of different evidence frameworks used to assess the value of DHTs globally, selecting the best framework for your technology can be difficult and is often dependent on your target market.
I have spent countless hours researching how to perform the best economic evaluation for a Digital Health Technology (DHT) and in this article I have summarised the most useful findings from my research so far.
How to Measure Quality: The CHEERS Checklist
The Consolidated Health Economic Evaluation Reporting Standards (CHEERS) checklist (2022) provides a set of guidelines for reporting economic evaluations in health care. This checklist aims to improve the transparency and quality of published health economic evaluations. It includes 28 items grouped into several categories:
- Title and Abstract: Clearly state that the study is an economic evaluation and provide a concise summary.
- Introduction: Describe the context, objectives, and rationale for the economic evaluation.
- Methods: Provide detailed information on study design, data sources, analytic methods, and perspectives taken. The methods section should detail the type of economic evaluation conducted, such as cost-utility analysis, to ensure comprehensive reporting.
- Study Parameters: Explain key parameters, including costs, outcomes, discount rates, and time horizons.
- Results: Present findings clearly, including uncertainty analyses and incremental cost-effectiveness ratios.
- Discussion: Interpret the results, discuss limitations, and compare findings with previous studies.
- Other Relevant Information: Include ethical considerations, funding sources, and conflicts of interest.
The checklist was developed to ensure that economic evaluations are comprehensive, transparent, and reproducible. But while generic to all health economic evaluations, the checklist is commonly applied to DHTs due to the lack of standardised evidence frameworks****in the field. See Figure 1 below for the full checklist.
What does Quality look like? Applying the Checklist to DHTs
A systematic review of cost effectiveness evaluations for digital health interventions, published by Gentili et al. in the Frontiers in Public Health Journal in 2022, assessed the quality of 35 different economic evaluations for DHTs. Applying the CHEERS checklist to digital health technologies involves various types of economic evaluations, including cost-benefit analysis, to determine the overall net welfare gain of interventions.
The studies were graded on the basis of the number of items achieved, classified as follows:
- Excellent, if all items were present in the study
- Good, if at least 80% of the items were satisfied
- Fair, if at least 60% of the items are satisfied.
What Items are found in all Economic Evaluations?
Seven CHEERS checklist items were fulfilled by all studies (100%):
- Title
- Background and objectives
- Comparators
- Time horizon
- Choice of health outcomes
- Analytical methods
- Study parameters
Title
The title is the first point of contact for readers and should clearly indicate that the study is an economic evaluation, including information about the interventions being compared. A clear, descriptive title helps in categorising and indexing studies, facilitating easier access and reference.
Background and Objectives
Providing a well-defined background and clear objectives sets the context for the study, explaining the health problem being addressed, the rationale for the study, and its aims. This helps stakeholders understand the unmet needs being addressed, guiding more informed interpretations of the study findings.
Comparators
Identifying and describing comparators is critical as it frames what the new intervention is being measured against. Accurate selection ensures that the economic evaluation is meaningful, influencing cost-effectiveness outcomes and subsequent decision-making.
Time Horizon
The time horizon specifies the period over which costs and outcomes are measured, significantly affecting the results of an economic evaluation. A suitable time horizon ensures that all relevant costs and benefits are captured, crucial for chronic diseases and when assessing the long-term benefits of digital health interventions.
Choice of health outcomes
Selecting appropriate health outcomes, including health-related quality, ensures that the benefits (and harms) of an intervention are accurately captured and relevant to your target stakeholders. Different interventions may have varied impacts on health outcomes, affecting how benefits are quantified in economic terms.
Analytical methods
The analytical methods describe how data are processed and interpreted, including statistical techniques, model choices, and approaches to handle uncertainty. Transparent reporting ensures that others can replicate or critique the study’s validity, crucial for credible results.
Study parameters
Study parameters include all input values used in the analysis such as costs, probabilities, discount rates, and utility values. Detailed reporting allows others to understand the assumptions behind an economic evaluation, vital for assessing robustness and generalisability.
What Items do Most Economic Evaluations Contain?
The 12 items that were included in a majority (>80%) of the studies were (%):
- Incremental costs and outcomes (97%)
- Study findings, limitations, generalisability, and current knowledge (97%)
- Conflicts of interest (97%)
- Assumptions (94%)
- Study perspective (91%)
- Measurement and valuation of preference-based outcomes (91%)
- Estimating resources and costs (91%)
- Source of funding (91%)
- Characterising uncertainty (86%)
- Setting and location (83%)
- Measurement of effectiveness (80%)
- Currency, price date, and conversion (80%)
Incremental Costs and Outcomes
Economic evaluations hinge on comparing new interventions with existing alternatives, often using the incremental cost-effectiveness ratio (ICER) to show the added value of a new treatment or digital health solution. This comparison is crucial. It shows the added value of a new treatment or digital health solution. Reporting incremental costs and outcomes gives stakeholders the right tools to make informed decisions. This transparency is key when weighing up new options against standard of care.
Study findings, limitations, generalisability, and current knowledge
Comprehensive reporting is vital. It ensures results are interpreted correctly within the context of what we already know. This approach helps other researchers replicate studies and conduct meta-analyses. It also gives policymakers a clear picture of how broadly the findings can be applied. Importantly, it shines a light on potential biases or limitations that could sway decision-making.
Conflicts of interest
Transparency is non-negotiable in scientific research. Disclosing conflicts of interest maintains trust. It allows readers to critically assess a study’s credibility. This practice ensures that financial or personal interests don’t unduly influence results. It’s about maintaining the integrity of economic evaluations.
Assumptions
All economic models and analyses are built on assumptions. These assumptions shape how we interpret data and make projections. By clearly stating these assumptions, we open the door for critical assessment. It allows others to validate the model’s robustness. This transparency enables sensitivity analyses, showing how changes in assumptions affect outcomes.
Study perspective
The perspective of a study, whether it’s from a healthcare system, societal viewpoint, or public health interventions, is crucial. It determines which costs and benefits are considered in the analysis. This choice significantly impacts results and their relevance to different stakeholders. Clearly stating the perspective helps readers understand the evaluation’s scope and allows for fair comparisons across studies.
Measurement and valuation of preference-based outcomes
Preference-based outcomes, such as Quality-Adjusted Life Years (QALYs) or Disability-Adjusted Life Years (DALYs), are essential for quantifying benefits across different health conditions. They allow us to compare the health outcomes and costs of competing alternative interventions, providing both quantitative effects (life extension) and qualitative effects (health-related quality of life). This standardisation facilitates comparisons between interventions with diverse health effects. It’s particularly useful in health technology assessments for guiding resource allocation.
Estimating resources and costs
Accurate estimation of resources and costs is fundamental. It determines an intervention’s true economic impact. This includes direct medical costs and indirect costs like lost productivity. Detailed cost estimation provides a comprehensive view of financial implications. It supports budget impact analyses and helps policymakers allocate resources efficiently.
Source of funding
Disclosing funding sources is about more than just ticking a box. It’s crucial for transparency. Funding can influence study design, conduct, and reporting. By reporting funding sources, we allow readers to consider potential biases related to financial support. This practice enhances trust in the research by demonstrating accountability.
Characterising uncertainty
Uncertainty is inherent in all economic evaluations. It stems from variability in data and assumptions. By characterising this uncertainty, we quantify the range of possible outcomes. This approach enables more robust decision-making. Instead of relying on a single point estimate, stakeholders can understand the risks associated with different interventions.
Setting and location
Context matters in economic evaluations. The setting and location of a study can significantly affect outcomes. Different healthcare systems, populations, and local practices all play a role. Reporting these details helps determine how generalisable the findings are to other contexts. It’s crucial for understanding how local factors influence cost-effectiveness.
Measurement of effectiveness
Measuring effectiveness is the foundation of assessing an intervention’s impact. Without solid effectiveness data, we can’t accurately determine cost-effectiveness. Detailed measurement supports robust comparisons between interventions. It ensures that economic evaluations are grounded in reliable evidence about health benefits.
Currency, price date, and conversion
Economic evaluations often deal with data from different time periods or countries. This necessitates currency conversions and adjustments for inflation. Reporting these details ensures that cost estimates are accurate and comparable across time and regions. It’s essential for enhancing the reliability of economic conclusions.
What Items are Most Likely to be Missing?
The five items most likely to not be reported (%) were:
- Characterising heterogeneity (31%)
- Target population and subgroups (37%)
- Abstract (63%)
- Choice of model (69%)
- Discount rate (71%)
Characterising heterogeneity
Heterogeneity shows how different subgroups respond to interventions. It’s vital for both medicine and digital health, where patient characteristics vary widely.
Why it’s often skipped:
- Lack of detailed subgroup data
- Requires complex statistical methods
- Smaller trials lack statistical power to detect differences
Benefits of including it:
- Enhances study relevance across diverse populations
- Guides targeted interventions for specific groups
- Improves resource allocation by identifying who benefits most
Heterogeneity analysis can reveal that a treatment works better for older patients or that a digital health app is more effective for tech-savvy users. This insight is gold for decision-makers.
Target population and subgroups
Defining who the study applies to is fundamental, especially for health care interventions, to ensure findings are relevant to the intended audience. It ensures findings are relevant to the intended audience, be it clinicians, public health officials, or healthcare systems.
Reasons for omission:
- Limited demographic data in the study design
- Privacy concerns about detailed patient information
- Study focuses on a homogeneous population
Value of reporting:
- Enables more accurate cost-effectiveness analyses
- Supports external validation and study replication
- Enhances credibility by showing clear applicability
Knowing a study focused on urban diabetics aged 40-60 helps readers understand its scope and limitations.
Abstract
The abstract is your study’s elevator pitch. It provides a quick overview of objectives, methods, results, and conclusions. It’s often the first (and sometimes only) part readers see.
Why it might lack detail:
- Word count limitations imposed by journals
- Editorial guidelines restricting content
- Authors underestimating its importance
A well-crafted abstract:
- Increases transparency of the study’s key points
- Improves accessibility for busy stakeholders
- Allows rapid assessment of relevance and quality
A clear abstract can mean the difference between your study being read or overlooked.
Choice of model
The model you choose shapes how data is interpreted and predictions are made. Different models can lead to different outcomes, so transparency is key.
Reasons for non-reporting:
- Authors assume it’s standard practice
- Use of proprietary models that can’t be disclosed
- Space constraints in academic journals
Benefits of reporting:
- Allows critical assessment of the model’s appropriateness
- Facilitates comparisons with other studies
- Strengthens the overall evidence base in healthcare economics
Knowing whether a study used a Markov model or a discrete event simulation helps readers understand its approach and limitations.
Discount rate
The discount rate affects how we value future costs and benefits in present terms. It’s crucial for long-term evaluations where impacts stretch over years.
Often omitted because:
- Authors overlook its importance
- They assume a standard rate is understood
- It seems irrelevant for short-term studies
Importance of inclusion:
- Ensures transparency in handling future values
- Enables accurate comparisons across different studies
- Supports sensitivity analyses to test different scenarios
A study using a 3.5% discount rate will value future outcomes differently than one using 5%. This can significantly affect conclusions about long-term cost-effectiveness.
The Key Take Home
Including key often-missed elements enhances study quality, transparency, and real-world applicability. It’s not just about ticking boxes – it’s about conducting evaluations that truly inform decision-making in healthcare.
How Healthonomix can help?
At Healthonomix, we understand the challenges of conducting comprehensive economic evaluations for Digital Health Technologies. Our team of expert health economists and data scientists specializes in bridging the gap between cutting-edge digital health solutions and robust economic evidence.
Our Tailored Approach
- Customised Modelling: Our experts select and justify the most appropriate economic models for your specific DHT, enhancing the credibility and applicability of your results.
- Clear Communication: From crafting compelling abstracts to detailing complex methodologies, we ensure your economic evaluation is both comprehensive and accessible to diverse stakeholders.
Why Choose Healthonomix?
- Expertise in Digital Health: Our team has a deep understanding of the unique challenges and opportunities in the digital health landscape.
- Regulatory Insight: We stay informed about evolving regulatory requirements, ensuring your evaluation meets current and future standards.
- Tailored Reporting: We tailor our reports to your target audience, whether it’s investors, payers, or regulatory bodies.
Don’t let gaps in your economic evaluation hold back your digital health innovation. Partner with Healthonomix to unlock the full potential of your technology. Contact us today to speak to one of the team.
