Your answer will be discussed in the open session of the Round Table.
The deadline for submitting comment is May 13th at 8:30 a.m - Central Europe Summer Time
For more information about this Online discussion on Impact Evaluation, please go to the right hand side of this weblog – General Information Box, or visit http://www.irc.nl/page/40054
I would very much like to see – right uprfront – a good discussion of the cost/benefit of impact evaluation – and the different approaches to it. There’s lots of talk around at the moment about ‘gold standards’- but gold is expensive and sometimes baser metals will do …. so – guidance on what/when/where based on how much it costs and how it can help will be very useful
Comment by Patrick Moriarty — 9 May 2008 @ 17:51
Re question 1 posed on the preceding web page:
1. “Why do we do impact evaluation? Who is the consumer of impact evaluation results and for what purpose do they need the information?”
Impact evaluation, and M&E more generally, should help produce knowledge that goes into the public domain and enables others to replicate the same type of impacts. It should generate a form of usable knowledge that is a “public good”, in contrast to the privatised knowledge that is subject to patents and copyright, covering production and business processes.
Only when these public goods are created can aid investments hope to have a large scale impact, in proportion to the scale of the problems being addressed.
For more on the same theme, see http://mandenews.blogspot.com/2008/03/aid-organisations-as-self-interested.html
Comment by rick davies — 12 May 2008 @ 20:58
I would like to see some discussion of what is meant by impact evaluation – whether it only refers to evaluations that use counterfactual approaches, or to all evaluations that gather evidence about impacts and analyse the contribution of particular interventions to them. I would particularly like to see any examples of impact evaluations of complex projects, programs and policies which do not suit experimental and quasi-experimental approaches.
Comment by Patricia Rogers — 13 May 2008 @ 13:07
Anura Widana: comment on Round Table on Impact Evaluation:meeting and on-line
I wish to address few neglected issues relating to impact evaluation of WATSAN projects for comments, discussions and possibly to be included into the round table being planned. Some critical issues are listed below:
a) The first question is that many projects are not subject to impact evaluation itself. This may be due to the obvious reason of being fear if the project has not delivered the expected outcome. Project design should explicitly state that it will be subject to impact evaluation after completion
b) Information for impact evaluation usually comes from short missions which do not have the time for extended field work utilizing participatory techniques. Hence, very little of achievements by the project is utilized for the impact evaluation whenever it is undertaken. It is necessary to avoid this situation by adopting other appropriate strategies. What are these strategies? How could the strategies be built into the project design?
c) Many WATSAN projects do not have a participatory / social development specialist as part of the team. This is partly because the TOR for the impact evaluation does not adequately include participatory, community and social sustainability issues as it is done with little or no inputs from a social specialists. It is critical that projects should have developed a well balanced TOR specifically stating participatory and social dimensions. In many situations, project teams do NOT include skilled and professional participatory and social specialists. More often than not, the participatory specialist attached whenever it is done does not have the appropriate skills and experiences to handle all relevant issues. In many cases, the participatory specialist is less comparable in relating to skills and qualification of other technical specialists such as engineers, sanitation specialists, etc. The end result being the social and community aspects are inadequately completed.
d) Many projects do not have on-going activities aimed at completing in-depth studies as part of the implementation programme. When the project is about to terminate, the day-to-day mundane and admin activities take over the remaining time of staff leaving little or no time for writing the findings of the project. The project design should be such that it has a programme to undertake and publish case studies that may focus on various aspects inclusive of sustainability issues. The case studies should be undertaken by specialists with proven track record inclusive of social aspects.
Finally, I wish to suggest that the panel for the round table itself SHOULD include 1-2 social and community development specialists possessing a track record of skills and qualifications in these areas.
Thank you
Anura W Ph D
Comment by Wash manager — 13 May 2008 @ 13:51
In the summary of block 1 of the ongoing discussion in Delft, you can find a summary of the general expectations defined. The summary of the whole day, including the comments from the online discussion will be published at the end of the day.
Comment by ircuser — 13 May 2008 @ 14:41
Maybe a question that has been clearly answered, if so let me know:
Is attributable IMPACT monitoring / evaluation possible at all? I was initiated by my former colleague Kathy Shordt:
1. don’t even try to assess impact;
2. monitor efficiency;
3. monitor effectiveness;
4. make sure monitoring system is affordable and owned;
5. make sure monitoring is integrated and provides action oriented management follow up.
So… I think: lets not waste limited resources to try to assess impact!
Peter
Comment by Peter J. Bury (IRC) — 13 May 2008 @ 16:39