Principles and Practices for Federal Program Evaluation Proceedings of a Workshop (2017) / Chapter Skim
Currently Skimming:
5 Putting Principles into Practice: A Balancing Act
5 Putting Principles into Practice: A Balancing Act
Pages 25-30
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 25... ...
The keys to protecting scientific quality, she said, are having a strong evaluation team, a strong design that addresses the appropriate questions, and review procedures that protect against false claims and reinforce credibility. Rebecca Maynard (member, steering committee)
|
From page 26... ...
Howard Rolston (member, steering committee) , referring to Gueron's comment about the importance of design, said that there is tension between performance management and evaluation: performance management typically pays less attention to design and makes causal claims without any explicit identification of a counterfactual, which is a real issue in the use of administrative data and would not be tolerated in high-quality evaluation.
|
From page 27... ...
She asked the participants to weigh in on the pressures of timing when balanced against a desire to release complete results and whether or not either of those factors could threaten future funding or lead to undue political interference. Whitehurst emphasized the need for schedules for each component -- contractors, peer reviewers, professional staff, and so on -- that are appropriate to the context.
|
From page 28... ...
Barnes reminded participants that unlike larger agencies with standalone evaluation agencies or offices, smaller agencies may have to handle evaluations for their specific program area. That difference in structure has important implications for how a program manages independence and works to ensure scientific integrity.
|
From page 29... ...
Maynard commented that if an agency is proposing to use a method other than a randomized controlled trial to answer questions about impact or effectiveness, it should have a compelling argument as to why randomization cannot or should not be used. She said that the first thing one should do is gather information about what stakeholders believe would be challenging or unethical about randomization and what they view as the preferred alternative -- which Gueron noted is often the hardest issue to counter -- and then to systematically address the concerns, including the evaluation threats associated with the alternative.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.