In 2017, over 15,000 new women enrolled in WfWI’s core program to gain critical knowledge, skills, and resources to change their lives. But how can we know these women have benefitted from the program? Ask WfWI’s Monitoring and Evaluation (M&E) team.
The M&E team regularly collects data from program participants to help ensure that program objectives are being met. To make any assumptions about the success of our program, however, we need high quality data – data that are reliable, realistic, and useful. We introduced in the first MERL blog the types of changes recently made to our standard survey to increase the quality of data received from participants. Preliminary new survey results in hand, we can share some insight on what the new data is telling us.
One of the key pathways through which we hope to improve women’s lives is increased household decision-making power. To measure this, we ask women to self-report whether they are involved in a variety of household decisions, such as family planning decisions or decisions about household finances. Previously, our survey question only allowed women to respond “yes” or “no” as to her involvement in a decision. The question also did not define what we meant by “involvement,” so the question’s interpretation was left to each woman to determine. This type of vague wording could result in different women reporting decision-making power according to their own, unique definitions. Without a common understanding of the question, it’s not easy for us to precisely interpret women’s responses.
To improve the usefulness of this data, we revised the decision-making questions to be less ambiguous and more specific. In place of our old question on family planning decisions, we now ask “Who in your household has the final say in decisions about having more children?” Now, a woman can respond “my husband” or “me and my husband jointly” or a number of other possibilities. Another key difference is that we now define the type of decision involvement we want to learn about: having the final say. Further, we include a specific decision scenario, such as deciding to have more children, rather than asking about the broad category of family planning. This specificity helps guide women to respond with greater reliability than before.
Using the old type of survey question, 35% of our 2016 Afghanistan graduates told us they were involved in household family planning decisions at the beginning of their enrollment in the program. By the end of the 12-month program, 95% of these graduates reported involvement in family planning decisions. This is an enormous increase in women’s reported decision-making power, and fantastic feedback to hear from our graduates. But if we want to gain a more realistic reflection of household decision-making dynamics, we examine the results of our improved question.
Using our revised survey question, only 21% of 2017 Afghanistan graduates tell us at enrollment that they are included in the final say of decisions about having more children. By graduation, this increases to 65% of women. When asked a more specific question with stricter definitions, we see, on average, a lower share of women reporting involvement in family planning decisions than from the old question. The new data gives us a richer sense of household decision-making power balance than before. It also tells us that despite revising the question to yield more realistic results, we still see a 210% increase in family planning decision-making. We believe the results of our new survey provide a clearer picture of the reality in which our participants live, one in which a woman’s effort to govern decisions on her own wellbeing can be a daily struggle.
The more we improve our data quality, the more we can use this data to improve our program. By scrutinizing the data from our revised survey, we are better able to pinpoint opportunities to strengthen our program and yield improved outcomes for our women. For example, while the majority of Afghan graduates report involvement in critical household decisions by the time they graduate, their responses to “Who in your household has the final say in decisions about having more children?” may also show that we can do more to help women negotiate greater decision-making power at home.
Women’s direct feedback on the core program is of utmost importance to WfWI, and the M&E team strives to ensure that WfWI uses participants’ experiences to drive key programmatic and organizational decisions. Ultimately, improved data helps us build a stronger, more focused, and more relevant program for the women we serve.