Building GSGC Capacity with Meaningful Evaluation and Learning

By Kate Wilson, GSGC volunteer

Adding numbers and context to success stories

This year, Mentors and a team in the US focused on making our approach to evaluating GSGC more systematic. We want to better understand how our work improves the lives of Mentors, club members and communities across our network. We agreed on common goals and ways to measure progress, revised our Mentor model, and identified data we have and data we need.

By the end of 2022, we will be able to report trends, including educational achievement, using numbers collected from each club. We also will gather personal stories from Mentors to better understand the range of experiences, successes and challenges with GSGC. The stories will provide context for and meaning to the numbers.

Centering mentors in the evaluation

Mentors are central to GSGC’s approach to evaluation. Mentors collaborate with the evaluation team throughout the process, providing guidance about what questions to ask and what data to use. We call our approach Meaningful Evaluation and Learning (MEL), which emphasizes that this process must be meaningful to Mentors and prioritizes time for learning and reflection.

Working questions and plan

Our evaluation plan has key questions on two levels: the role of GSGC with Mentors and the role of Mentors with club members. Here are the questions:

1) Are Mentors getting the support they need through GSGC to lead their clubs successfully? If not, what are the challenges?

2) What should be changed about the GSGC approach to better support Mentors?

3) Do we see improvements in club members’ self-confidence, educational achievement and health thanks to being in the clubs? If not, why not? What are the challenges?

4) What should be changed about the clubs to show more improvements among members?

Identifying common goals and measures

This year’s main challenge to building our MEL capacity was identifying a common set of goals and measures. Clubs are tailored to specific needs and interests of club members, so Mentors have a range of goals and activities. These include building self-confidence, developing artistic skills, improving educational achievement, improving gender relations, and engaging boys to support girls’ empowerment. Through group discussions and review of quarterly reports, we identified at least three goals common to all clubs, and we began tracking them in 2022. They are:

• Increasing self-confidence

• Improving educational achievement (e.g. complete secondary school, get all ‘As’)

• Increasing beliefs and behaviors in support of gender equity

Mentors know what indicators to look for and which sources can provide data about these goals, and we are confirming the ones all Mentors can report. For example, most Mentors have access to school grades and attendance to record educational achievement. Measuring self-confidence is more challenging. One option is to track how many club members give school or public presentations.

Our planned evaluation activities and data

Impact of GSGC support on Mentors’ leadership and technical skills: To understand how GSGC supports Mentors’ leadership skills and growth, we conducted in-depth interviews with each Mentor from April through June 2022 (Questions 1 & 2). Interviews focused on what works, what does not work, and what GSGC can do to improve how it supports Mentors.

GSGC impact on club members and communities: Our evaluation will build on activities we already do and use data we already have. Mentors report on their activities and outcomes every quarter, and those quarterly reports will be our main data source regarding questions of impact on club members (Questions 3 & 4). We also will add common goals and indicators and open-ended questions about community engagement.

Working Conceptual Model of GSGC

We have developed a working evaluation plan and GSGC program ‘theory of change.’ (Figure 1.) This theory shows a cyclical process: GSGC supports Mentors, who in turn empower club members and engage communities. Feedback loops indicate that Mentors’ experiences leading clubs inform their work and GSGC’s approach. This graphic guides our evaluation questions and activities and is a living document that we update as we learn from our results.

Figure 1: Our theory of change shows a cyclical process

Next steps

After starting with the revised quarterly reports and Mentor interviews, we will consider appropriate comparison groups, especially for trends in pregnancy and educational achievement. We will use our MEL-focused Mentor Meet-Ups to learn from and reflect on our evaluation data and offer individual and small group MEL coaching for Mentors who request support.