How we measure our impact for New Zealand learners.
At Springboard Trust, we work with more than 300 volunteers and principals every single year. Each of these individuals has their own unique experience, bringing their own expertise and background to courses like High Performing Leaders and the Strategic Leadership for Principals Programme.
Throughout their journey with Springboard, and for some time afterwards, we gather information on the impact that Springboard’s work has had through surveys, interviews and specially designed assessment rubrics. This gives us a significant breadth of qualitative and quantitative data about how our portfolio impacts principals, volunteers, organisations, schools, senior leaders and New Zealand students.
Every year, we publish this data – along with substantial analysis of it - in our Impact Report. It includes our work on a new evaluation framework, reflexive thematic analysis, qualitative and quantitative findings that link the work we do with principals to positive outcomes for students.
With the 2019 edition now available for you to read, it’s a good time to break down some of the ways we use this data to measure our impact.
How we are measuring our impact: Reflexive thematic analysis
In the past, we have presented impact data qualitatively, as stand-alone case studies or supporting evidence. By utilising reflexive thematic analysis and a dedicated statistician, we have been able to turn this wealth of information into statistically significant findings around our impacts on schools and learners.
Thematic analysis (TA) is an overarching term for a set of practices in psychology, that have applications well beyond this field. In TA, researchers analyse qualitative data (like interviews, surveys or other expressive, open-ended responses) and identify statistically significant themes and outcomes. In short, it’s a more objective way of demonstrating results from data sets that can be highly subjective.
Reflexive thematic analysis (RTA) is a subset of this, and was originally developed by the University of Auckland’s Virginia Braun and Victoria Clarke in 2006. Since then, it has become a hugely popular approach which has seen it being used as a methodological approach in hundreds of academic journals internationally.
It is particularly well-suited to data sets that relate to people’s personal experiences or perceptions, and so forms a useful basis for analysing Springboard Trust’s impact.
RTA consists of the following six key steps which are recursive - meaning that the researcher might move back and forth between these steps several times:
Familiarising yourself with the data or information
Giving each element of the data a name or label (coding)
Developing the high-level themes or patterns within the data
Reviewing these themes against the entire dataset
Detailing and analysing each theme
Writing up the findings
We have adopted this approach, together with quantitative methods for the 2019 Portfolio Impact Report with the assistance of a dedicated Research and Evaluation professional, resulting in the most in-depth analysis of Springboard Trust’s impact to date.