When approaching a public policy evaluation, one of the first commonly accepted methodological steps is to analyse the logic of the intervention: How was the policy designed? Does a clear relationship exist between your objectives (explicit or implicit) and your results, activities and outcomes, and among all the interlinking elements?

To address this complex situation, governments have adopted extremely different measures that affect very diverse sectors of society: health, the labour market, housing, mobility, taxation, and so on.

But what are their specific, general, medium, short, long-term objectives? In principle, we might think that the main objective would be to save as many human lives as possible, to prevent the collapse of the health system, to enable the population to keep its rights and purchasing power. However, to make a rigorous, exhaustive assessment, we need to establish clear goals against which to measure progress, achievements and failures. Against which criteria can the adopted measures be evaluated to determine their effectiveness, that is to say, against their capacity to achieve the proposed goals?

“Against which criteria can the adopted measures be evaluated to determine their effectiveness?”

 Of course, this leads to complex and politically and socially sensitive questions: What is the acceptable goal in terms of loss of human life? How many (non-temporary) unemployed people are acceptable? How many small and medium-sized companies can be expected to disappear? These are just a few examples.

In order to carry out an evaluation of the effectiveness of the adopted measures, we therefore need out governments to:

-Clearly define existing and potential social problems.

– Based on these, establish a Contingency Plan, ideally, inserted into the Decree on the Declaration of the State of Alarm (so that it forms part of the Legal System), with a set of approved measures that address the objectives in a clear manner, also defining process and result indicators for monitoring their evolution.

-Establish new measures to build a coherent master plan to achieve the objectives defined.

-Design a thorough monitoring system with flexibility to modify measures based on the results obtained and the evidence collected.

A mature society, the vast majority of which is behaving impeccably and following the recommendations deserves to know the parameters used by institutions to manage the situation for subsequent evaluation.

If we consider an impact analysis in the strict sense, the difficulty lies in the absence of a counterfactual argument: What would have happened if no measures had been adopted rather than taking this course of action? What would have happened if other measures had been taken or had been taken earlier? Which of the possible approaches would be most successful to deal with the crisis?

Every day and every hour that goes by, data and figures are being generated for drawing graphs and compiling statistics that evolve rapidly and are ever changing as they are analysed by country, by region and by continent. They provide valuable information and reveal developments on which to build hypotheses to test.

In view of the data and the context, countries have adopted different solutions, but the existence of too many variables that are difficult to control (strength of health systems, baseline citizen health, population pyramid, cultural habits, life expectancy, etc.), are preventing a comparative analysis with the necessary rigour to draw robust conclusions.  However, we do have similar previous experiences that can give us some analysis guidelines.

“The existence of too many variables that are difficult to control is preventing a comparative analysis with the rigour necessary to draw robust conclusions”

 Experts in the field have mentioned that during the SARS (2003) and influenza A (2009) crises, there was a breakthrough in research and the fight against these diseases. It is also evident that as the outbreaks were controlled and the emergency ended, the funds to continue research and development of different drugs either dried up or were drastically reduced, and these lines of work stopped. The work was unfinished. Unfortunately, we did not use those earlier crisis to slow the current pandemic earlier and better.

One of the benefits of evaluating public policies can be the lessons learned. Hopefully an ex post evaluation of the current crisis will teach us lessons that were previously lacking and we will use them to build strengths that, without being aware of it, we are all building these days.

The EVALÚA Project is attending to the new priorities of partner institutions to create a baseline that will serve as a reference point for  subsequent evaluation of current actions, with a comparative approach at the regional level.

The views and opinions expressed in this blog are the sole responsibility of the person who write them.