To infinity and beyond—three things international donors should consider as health economic evaluation goes global

July 11, 2017
Imagine you have a budget of £150m (US$193m) to spend in a country where the prevalence of malaria and teenage pregnancy is high, as are the corresponding infant, child and maternal mortality rates. You have three options to spend your money: a programme to distribute bed nets, protecting children against malaria; a programme that expands access to family planning, offering protection to young women; or a salary reform initiative, enabling the Ministry of Health to hire enough staff. You haven’t got enough money to do it all and you need "quick and attributable wins" that avoid negative media on the wastage of money. What would you choose?

Increasingly, in health and elsewhere, the principles of economic evaluation are being drawn on for help. Based on the premise that any expenditure involves a trade-off, economic evaluation provides a lens through which to ask whether money spent on one programme could be better spent on another. It commonly does this by comparing the costs and benefits of that programme with those of alternative options.

Using old tools in new places…

The various methods available have grown into a sub-discipline of their own over the last 20-30 years. However, this growth has been driven largely by the needs of stakeholders in higher-income countries to guide domestic public policy (often in health). International donors work in a different arena: they often have limited direct accountability to the beneficiaries; and health systems in lower-income countries operate under more severe resource constraints. This means fewer health workers, equipped facilities, drugs and medical consumables per person. Or, in the language of economic evaluation, the opportunity cost of expenditure becomes much higher.

Furthermore, some systemic differences are also apparent. Health systems in lower-income countries often rely more on out-of-pocket expenditure, which increases the risk of pushing households into poverty. There is often a wider variety of provider types (such as public, private, faith-based, unregulated and traditional providers). The global market for health workers makes it hard to retain expensively trained staff. And, of course, the prevalent diseases differ (communicable diseases remain major issues in lower-income countries, but the burden of non-communicable diseases is growing).

What does this all mean for the use of economic evaluation methods to guide health-sector decisions in such contexts? Here are some recommendations.

1.       Keep it local

There is no "one-size-fits-all" model, estimate or guideline. As the methods for health economic evaluation have gone global, there has been a corresponding shift towards top-down analysis using increasingly universal measures of value—applied to everything from discount rates and cost-effectiveness thresholds right through to fundamental ideas about what "good" health means. Two high-profile examples of this are the (including ), and the World Bank’s publication of up-to-date information on intervention efficacy and programme effectiveness (now in its third edition).

There is a danger, however, that, without sufficient tailoring, evaluations based on generalised metrics lose their meaning at the local level. A regional estimate plugged into a generic model may produce results that do not reflect local realities. An overreliance on impressive-looking models risks undermining the need to understand local epidemics and to generate reliable, context-sensitive evidence on relative effectiveness and costs, ultimately affecting the credibility of the analysis.

 2.       Decide in advance what good value looks like (know your thresholds)

 Insufficient clarity on what good value is decreases the transparency and accountability of the process. For example, it is becoming common for donors to conduct routine retrospective "Value for Money" (VfM) analyses on programme expenditure, constructing a measure of cost-effectiveness out of economy, effectiveness and efficiency (and sometimes equity). However, much of the current guidance does not comment on the point at which something should be considered cost-effective or good VfM.

 As such, analysts are asked to make a "reasoned judgement" about the VfM with the information they have. They are told to think about economy, efficiency, effectiveness and equity, but not what levels are actually considered acceptable. This creates space for inconsistency across programmes, where projects with similar costs and effects are treated differently owing to the personal perspective of the analyst.

 3.       Be realistic

 Analysts often find themselves asked to comment on broad, system-wide interventions (such as the blanket removal of user fees or a new national information system). Furthermore, the data available in lower-income settings are often scarce and of insufficient quality. Put these two things together and what do you get? The principles of economic evaluation are applied, with sub-optimal data, to options so broad that they touch on the nature of the health system itself. How much trust would you put in such answers?

 Of course all questions should be considered, but when limited resources are available to spend on analysis, the methodological approach to answering the question should be both valid and feasible. There are plenty of important questions that can be answered with the limited data that are available.

Go forth and multiply

I absolutely encourage international donors to make use of economic evaluation. But consider some simple steps to make sure you get what you want. Inaccurate analyses, without a clear conception of good value, based on inadequate data, are of no use and can actually do more harm than good. Instead try to use the local data that are available, decide in advance what good value looks like, and be realistic about what you can answer.

The issues addressed in this blog post are currently being discussed at the .

Alex Jones

Alex Jones is a health economist at Oxford Policy Management. His interests are in the use of evidence in policymaking, particularly the use of economic evaluation.

Receive forward-looking perspectives from our editors - Sign up for our Weekly Digest