7.8: Clarifying the evaluation question(s)

As you think about the questions that your evaluation will answer, you may need to break larger questions into sub-questions. The point is to be as clear as possible about what you REALLY want to know so that you can better collect the information needed.

We often see evaluation questions that are broad and vague. When asked, these questions yield broad and vague responses that are difficult to interpret and of little use for program decision making. It is worth your time and effort NOW to bring clarity to your evaluation question(s).

Let’s consider “Get Checking” – a program aimed at high school students who lack basic financial literacy (budgeting, saving, borrowing and investing) with an emphasis on increasing skill in money management using a savings and checking account. An example of a broad evaluation question might be: Did teens benefit from attending the “Get Checking” program? What might be some possible sub-questions that would provide more focus to this question? What might you really want to know?

Are the Questions Appropriate?

Can the questions be answered given the program?

To determine what questions are appropriate based on the program is one of the main reasons for doing a logic model. By describing what the program is, the logic model helps determine what is appropriate to evaluate. For example, it may be inappropriate to ask if smokers in the county quit smoking when the program was focused only on building awareness and knowledge of local tobacco policies. Or, it would be inappropriate to survey all business owners about changes resulting from a program when the program was targeted to businesses employing fifty or fewer individuals.

Are the questions key, of high priority, practical?

As we’ve said before, you can’t and don’t need to evaluate everything. In most cases, it will be necessary to prioritize the evaluation questions. Try to distinguish between what you need to know and what might merely be nice to know. What are the key, most important questions?

Consider time, resources, and the availability of assistance needed to answer the questions. As appropriate, bring stakeholders together and negotiate a practical set of questions. Remember, it is better to answer a few questions thoroughly and well. Given the current interest in and demand for outcomes, often our evaluation questions focus on outcomes. Remember, however, that to attribute your program or your role to outcomes you also need to ask questions about the process that contributed to those outcomes.

Are the questions understandable?

Finally, ensure that your evaluation question(s) are understandable. Avoid the use of jargon or vague words that can have multiple meaning. Always define key terms so that everyone understands the meaning.