Crafting compelling stories that communicate the objectives, methods, results, and recommendations of an evaluation in a way that is easy to understand and memorable is a crucial aspect of evaluation reporting. It often involves the use of real-life examples, anecdotes, and visuals to make the evaluation more relatable and impactful. Traditional research/evaluation approaches often fall short in capturing the complexities, nuances, and intricacies of programs. Complex programs with multiple components often present complex evaluation challenges.
This was the case with the Indiana STEM Teacher Residency project, which aimed to recruit and train culturally competent and highly qualified teachers in STEM subject matters for the diverse context of the Indianapolis Public Schools (IPS) district.
However, our team had to adopt a nimble and flexible approach to evaluate the project due to numerous unanticipated challenges such as the pandemic, low participant recruitment rate, personnel turnover, and the complexity of the context itself. Guided by Crane et al (2018) principles of pragmatism, we utilized theoretical flexibility, methodological comprehensiveness, and operational practicality. Theoretical flexibility enabled us to utilize formative, summative, and developmental techniques, facilitating the use of participatory methodologies that allowed us to account for not only the process and impact measures but also the contextual factors. Methodological comprehensiveness involved the use of multiple methods in both the quantitative and qualitative realms, allowing us to examine different facets and levels of program implementation and beyond meta-narratives, making every voice count. Operational practicality facilitated our ability to communicate real-time and actionable feedback to stakeholders, make considerations for real-life limitations and constraints, and manage the complexity of the project.
In summary, our evaluation effort blended diverse methodologies, arrived at a comprehensive understanding of program intricacies, and effectively communicated findings. It also aided in managing complexity, adapting to unforeseen challenges, and offering real-time feedback, turning obstacles into opportunities. This approach served as the linchpin that enabled us to weave our methodologies and enhance our narrative's relevance and value to stakeholders and the broader educational community.