Bringing clarity to impact assessment for public policy initiatives
Crowe's Consulting team is an advisor to Government, public bodies and not-for-profit organisations for the evaluation of the impact and success of funded initiatives and programmes for improving social cohesion and for delivering major public policy goals.
There can be a large gap between a large-scale public policy objective and the reality of implementing that change to affect society. Imagine, for example, what it actually takes in terms of funding, planning, time and commitment to implement large-scale initiatives such as the roll-out of Sláintecare, or the Anti-poverty and Social Inclusion Measures as set out in the Programme for Government 2022, and how many hundreds of people and organisations are involved. The delivery of lofty policy objectives is typically composed of hundreds, thousands, or perhaps hundreds of thousands of interventions by agencies, institutions and non-governmental organisations spending time and money to target groups, households and individuals.
How do you assess the effectiveness of public policy initiatives?
When it comes to determining the effective use of public resources, an important question to ask is, "Is this work meaningfully contributing to the big picture?" This could mean confirming that money spent is having the intended effect, determining if a programme is leading to positive change, or even if a process is functioning correctly. This question poses a significant challenge to organisations managing or implementing large-scale initiatives, for three reasons:
- It can be difficult to "see the forest for the trees" – when a project or initiative forms part of a wider ecosystem, measuring the project's effect on that wider ecosystem is tricky.
- Many state-funded or charitable interventions are intended to have effects that are difficult to financially or quantitatively measure in the short-to-medium term – they are often about changing attitudes, removing barriers, or influencing behaviour, which may only be noticed or measurable in the longer term.
- Much like marking your own homework, objectively determining your own success is difficult – particularly when it comes to the use of public funds, impartial expert assistance is recommended or may be required in order to achieve the necessary level of rigorous assurance.
Good practice involves appointing an external advisor to assist with assessing impact and to provide assurance about the effective use of resources in the delivery of these kinds of initiatives. It is also often a funding requirement that an external evaluator is in place.
What we do
Our Consulting team work as external evaluators for Government departments, state agencies and not-for-profit organisations – looking in detail at the operational effectiveness of public policy initiatives and providing insight into their impact in the real world. We provide clarity and insight into, and understanding of, a complex strategic, administrative, political, operational and political landscape, and determine whether and to what extent projects, programmes and initiatives are effective.
The audience for our reports is usually a wide range of civil and public service organisations, sponsors and funders, and members of the public who want to understand in clear and unambiguous terms the value achieved.
Each project is, of course, unique – meaning that there is no single approach to an impact assessment or effectiveness review. However, the way we go about this work is grounded in sociological research methods and internal audit principles, and is informed by our years of experience. The high-level approach to the generation of evidence and creation of a useful and informative report is illustrated below.
|
Here is a short selection from the large range of evaluative assignments we have undertaken in recent years examining public policy initiatives and their impact:
|
Our objective is to work in partnership with our clients to bring clarity, to articulate the reality of the situation in a way that the audience will easily understand, and in a way that allows improvements to be made that are implementable in the real world.
How we bring clarity
During assignments, data-collection activities typically generate a large volume of quantitative and qualitative data. Much of the qualitative data is in the form of stakeholder interviews and recorded perspectives, and by its nature is tightly bound to and influenced by the individuals' outlook and the challenges of their own roles and responsibilities.
Creating useful information from a large volume of qualitative and quantitative data is the key value-adding activity involved in bringing clarity to impact assessment projects.
We use a proven and extensive suite of consulting tools and analytical techniques to ensure that we work with this data in a reliable, robust and ethical way. This starts at the very beginning with determining our research focus and data collection strategy and designing our engagements with stakeholders. One such example of our approach to ensuring that our work generates clear findings and recommendations is our stakeholder feedback analysis lifecycle, shown in the diagram below. Our experience has shown us that our engagement with key stakeholders must be grounded in the essential research questions, and in order to have clear, evidence-informed findings, our analysis must close this loop and link back to those research questions.
Our clients have confidence in the validity and coherence of our work because our focus is on building a clear evidence basis for our findings and recommendations, distilling the noise and complexity of a multifaceted initiative into comprehensible answers to important questions. Is the initiative working well? Is this programme a good use of public funds? What could be improved?
Our experience, expertise, and toolkits provide confidence in the evidence we generate and the findings and recommendations we provide. We also bring an external and high-level perspective, helping our clients to "see the entire forest" as well as the individual trees.
Crowe's Consulting team is an advisor to Government, public bodies and not-for-profit organisations for the evaluation of the impact and success of funded initiatives and programmes for improving social cohesion and for delivering major public policy goals. Should you have any questions about the issues raised above, our experienced team will be happy to share their insights with you.
