Route One Evaluation

Think Practice Impact


3 Comments

Why Conduct a Program Evaluation Thesis or Dissertation?

In my first blog post, I introduced a framework for conducting a program evaluation thesis or dissertation. I’m an advocate for these studies, particularly for students in professional practice degree programs—the very students who can use program evaluation to benefit their workplaces. Many will be expected to do so.

There are three main challenges to conducting a program evaluation thesis or dissertation. On the bright side, these challenges provide opportunities to move the discipline of program evaluation forward and impact positive change.

Challenges

Lack of understanding of program evaluation among faculty and evaluation clients/stakeholders: Although faculty know how to conduct research, many have limited understanding of program evaluation. Similarly, the client and/or key program contacts for whom the evaluation is being conducted may lack understanding of program evaluation.

Lack of understanding of program evaluation among students: Students who aren’t in program evaluation degree programs typically have limited, if any, coursework in program evaluation. They take courses in research methods where program evaluation may be briefly covered, often mistakenly, as a type of research. Some will take a dedicated course in program evaluation, or maybe even two, if available and encouraged.

Acceptability of a program evaluation dissertation: Some programs and faculty don’t agree that conducting an applied study, such as a program evaluation, is appropriate for a dissertation. This is often related to the first listed challenge and misunderstandings about program evaluation. Truth is, a quality program evaluation is often more difficult to pull off than a quality research study. It requires strong technical skills AND strong non-technical skills. It also requires additional sections in the dissertation…for example, to discuss stakeholder involvement and standards of quality program evaluation.

Opportunities

Building evaluation capacity: When students conduct program evaluation thesis and dissertation studies, students, faculty, and evaluation clients and stakeholders learn about program evaluation and its applicability to their fields.

Contributing to local and academic knowledge: Most agree that a quality program evaluation contributes local knowledge that can be applied directly to decisions about program improvements, expansion, etc. If evaluation findings are interpreted in the context of other relevant studies, a program evaluation thesis or dissertation can also contribute to academic knowledge. Further, students can investigate an aspect of evaluation methodology while conducting the study; thus, also contributing academic knowledge about program evaluation.

Promoting program evaluation: By building evaluation capacity and contributing to local and academic knowledge through program evaluation thesis and dissertation studies, we promote the discipline. Perhaps best said by Michael Morris (1994) in his article on the single course in program evaluation…

Although a little knowledge can be a dangerous thing, program evaluation is a field in which total ignorance is much worse. Evaluation is most likely to achieve its dual goals of demonstrating scientific credibility and bettering the human condition in an environment where it is not just the professional evaluation community that has access to relevant knowledge and skills (p. 57).

Additional Reading…

Framework for Conducting a Program Evaluation Thesis or Dissertation

An article Mike Trevisan and I wrote about conducting evaluability assessment thesis and dissertation studies—although the focus is on evaluability assessment, the implications are relevant to program evaluation theses and dissertations in general.

 


1 Comment

Developing a Framework for a Program Evaluation Thesis or Dissertation

I often chair or serve on committees of students conducting a program evaluation thesis or dissertation and am developing a framework to support faculty and students in this work. I introduced the framework in a presentation during the Graduate Student and New Evaluator business meeting at the 2015 American Evaluation Association conference in Chicago. The evolving framework includes six key points:

  • Consider the evaluation problem
  • Understand the program and program context
  • Involve stakeholders
  • Choose an evaluation approach
  • Establish evaluation quality
  • Get the evaluation used

The specifics of how to apply these when conducting a program evaluation thesis or dissertation study are available as resources on my website, Route One Evaluation. I welcome questions and feedback on the framework. My next blog post will focus on challenges and opportunities associated with program evaluation thesis and dissertation work.