Assessing Your Impact

Learning from Experience: Hamilton and Winnipeg

The Winnipeg Foundation

Groundwork - First Steps
Guiding Principles
Evaluation Approach
Evaluation Activities
Reporting

Background: Centennial Neighbourhood Project

In 2004, The Winnipeg Foundation undertook a five year comprehensive community development/community capacity building initiative that operates within an identified geographic neighbourhood - Centennial Neighbourhood - in the north end of the city.

The Centennial Neighbourhood Project (CNP) begins with the view that poverty reduction and community change can be addressed by improving the educational prospects and outcomes of children in the community. The overall vision of the project is to: "demonstrate that with appropriate school and community supports, the prospects of students in one of Winnipeg's least advantaged neighbourhoods can be significantly improved." The project recognizes the range of realities and challenges within the neighbourhood and is actively engaged with the community in addressing those factors. It focuses largely on one elementary public school called Dufferin School.

The project has four key objectives against which impact and progress are measured:

To improve the prospects of students living in Centennial Neighbourhood through a family literacy initiative, pre-school or in-school programming, and broader community supports;
To assist Dufferin School, where appropriate, to develop enhanced programming for its students and families;
To support local residents and community service agencies with their efforts to influence the quality of life in Centennial Neighbourhood; and
To identify the most effective practices that might be extended through public policy to other inner city neighbourhoods

The Winnipeg Foundation has worked very closely with Leskiw and Associates and Chris Leskiw on its evaluation of the Centennial Neighbourhood Project.

Groundwork- First Steps	The evaluation of The Winnipeg Foundation project was designed and conducted in such a way that it walks alongside the project and provides a continual opportunity to assess, support and understand changes taking place in the community and the impact of the interventions of the project. A key first step in the evaluation process, conducted in 2003, was to prepare background on the neighbourhood including: a baseline demographic data base, a summary of perceptions and commentary from those who work or live in the area; and, a visual description including pictures. An initial process for identifying and prioritizing outcomes was conducted with resident leaders and committees were established. These committees identified the detailed changes in the neighbourhood that were expected, indicators and evidence of such changes, desired outputs, and the relationship of the CNP and its partners to those changes and to the capacity of the project to support the prospects of children and youth. Back to top

Groundwork- First Steps

The evaluation of The Winnipeg Foundation project was designed and conducted in such a way that it walks alongside the project and provides a continual opportunity to assess, support and understand changes taking place in the community and the impact of the interventions of the project.

A key first step in the evaluation process, conducted in 2003, was to prepare background on the neighbourhood including: a baseline demographic data base, a summary of perceptions and commentary from those who work or live in the area; and, a visual description including pictures. An initial process for identifying and prioritizing outcomes was conducted with resident leaders and committees were established. These committees identified the detailed changes in the neighbourhood that were expected, indicators and evidence of such changes, desired outputs, and the relationship of the CNP and its partners to those changes and to the capacity of the project to support the prospects of children and youth.

Guiding Principles	From the beginning the evaluation process of the Centennial Neighbourhood Project has been guided by a number of principles: Evaluation will be driven through the residents and community representatives. It will serve to support a growth in their understanding and knowledge to continue to be involved and direct the work of the CNP. Research and evaluation efforts will be focused and non-intrusive. It is very important that assessment tasks not turn the neighbourhood into a laboratory Evaluation will support the development of mutual understanding between the residents and the 'systems', including government departments, agencies, community service organizations, funding bodies, etc., that can impact on changes within the neighbourhood; and, Evaluation will remain flexible to accommodate both opportunities and limitations that arise through the work of the CNP, serving to track the changes that occur, as they occur. Changes to plans of action will be accommodated within the evaluation framework. Back to top

Guiding Principles

From the beginning the evaluation process of the Centennial Neighbourhood Project has been guided by a number of principles:

Evaluation will be driven through the residents and community representatives. It will serve to support a growth in their understanding and knowledge to continue to be involved and direct the work of the CNP.
Research and evaluation efforts will be focused and non-intrusive. It is very important that assessment tasks not turn the neighbourhood into a laboratory
Evaluation will support the development of mutual understanding between the residents and the 'systems', including government departments, agencies, community service organizations, funding bodies, etc., that can impact on changes within the neighbourhood; and,
Evaluation will remain flexible to accommodate both opportunities and limitations that arise through the work of the CNP, serving to track the changes that occur, as they occur. Changes to plans of action will be accommodated within the evaluation framework.

Evaluation Approach	Working with Leskiw and Associates, The Winnipeg Foundation's evaluation has focused on participatory approaches that encourage the collection and dissemination of knowledge to help the project, the neighbourhood leadership, and the support systems better understand their roles in bringing about positive change. Gathering and assessing information takes place on an on-going basis with a focus on two evaluation events and two specific report-backs each year: a year-end report that provides an overview of progress across all activities, and a topic-specific report that focuses on identified areas of concentration within the Project. Back to top

Evaluation Approach

Working with Leskiw and Associates, The Winnipeg Foundation's evaluation has focused on participatory approaches that encourage the collection and dissemination of knowledge to help the project, the neighbourhood leadership, and the support systems better understand their roles in bringing about positive change.

Gathering and assessing information takes place on an on-going basis with a focus on two evaluation events and two specific report-backs each year: a year-end report that provides an overview of progress across all activities, and a topic-specific report that focuses on identified areas of concentration within the Project.

Evaluation Activities	A wide variety of sources and methods are used to gather information, both qualitative and quantitative. Sources of quantitative data vary, depending on the topic, but include things like: Health indicators extrapolated from Winnipeg Regional Health Authority Data; Safety indicators including block activity statistics from the Winnipeg Police Service and neighbourhood level statistics from the National Crime Prevention office; Housing activity and physical condition of housing from the City of Winnipeg Investment levels and program expenditures from The Winnipeg Foundation (within each priority area) and all formal and informal partners and funding sources; Recreational and social supports via local area agencies, and their respective government/private sectors partners. Qualitative data focuses on residents, agency and partner perceptions of changes in areas like: Overall and specific quality of life issues Perception of safety, area cleanliness and neighbourhood pride factors Opportunities for children and the resulting implications for their development Family and community strength Stability within the neighbourhood Changes in the capacity of residents to direct and build their own opportunities Reactions by governments and agencies Knowledge of the neighbourhood by the 'systems' Preliminary information was established through: Working with the committees A community survey developed and conducted with approximately 100-150 households (reaching 250-400 people) Interviews with a select number of government, funder and agency representatives More specifically and for example, a mid-year evaluation report completed in January 2006 focused on whether the educational prospects of children in the neighbourhood have been enhanced. It drew on the following sources of qualitative information: Available written reports from CNP, the neighbourhood newsletter, and the school's year-end report; A discussion circle with teachers from the school; Interviews with school administrators A discussion circle with parents The results of a parent survey Back to top

Evaluation Activities

A wide variety of sources and methods are used to gather information, both qualitative and quantitative. Sources of quantitative data vary, depending on the topic, but include things like:

Health indicators extrapolated from Winnipeg Regional Health Authority Data;
Safety indicators including block activity statistics from the Winnipeg Police Service and neighbourhood level statistics from the National Crime Prevention office;
Housing activity and physical condition of housing from the City of Winnipeg
Investment levels and program expenditures from The Winnipeg Foundation (within each priority area) and all formal and informal partners and funding sources;
Recreational and social supports via local area agencies, and their respective government/private sectors partners.
Qualitative data focuses on residents, agency and partner perceptions of changes in areas like:
- Overall and specific quality of life issues
- Perception of safety, area cleanliness and neighbourhood pride factors
- Opportunities for children and the resulting implications for their development
- Family and community strength
- Stability within the neighbourhood
- Changes in the capacity of residents to direct and build their own opportunities
- Reactions by governments and agencies
- Knowledge of the neighbourhood by the 'systems'
Preliminary information was established through:
- Working with the committees
- A community survey developed and conducted with approximately 100-150 households (reaching 250-400 people)
- Interviews with a select number of government, funder and agency representatives

More specifically and for example, a mid-year evaluation report completed in January 2006 focused on whether the educational prospects of children in the neighbourhood have been enhanced. It drew on the following sources of qualitative information:

Available written reports from CNP, the neighbourhood newsletter, and the school's year-end report;
A discussion circle with teachers from the school;
Interviews with school administrators
A discussion circle with parents
The results of a parent survey

Reporting	The Centennial Neighbourhood Project has reported out with documents structured around four themes: What primary changes have occurred: Findings What actions and efforts have generated those changes: Contributing factors What is the cumulative effect to date of the work of CNP on the prospects of the children in the neighbourhood: Impact and Implications What has been learned to maintain positive changes in the long term: Lessons Learned: Sustaining Change An evaluation committee was established to oversee the evaluation work. That committee was composed of individuals representing partners that are most involved with the priority elements of the neighbourhood at any given time. The general functions of the committee are to: Review the progress of the evaluation and provide assistance in information gathering; Serve as a conduit with their respective organizations/agencies/departments and the neighbourhood in order to facilitate information sharing; Support the learning of residents about the functions of their respective organizations/agencies/departments; Problem solve as needed Back to top

Reporting

The Centennial Neighbourhood Project has reported out with documents structured around four themes:

What primary changes have occurred: Findings
What actions and efforts have generated those changes: Contributing factors
What is the cumulative effect to date of the work of CNP on the prospects of the children in the neighbourhood: Impact and Implications
What has been learned to maintain positive changes in the long term: Lessons Learned: Sustaining Change

An evaluation committee was established to oversee the evaluation work. That committee was composed of individuals representing partners that are most involved with the priority elements of the neighbourhood at any given time. The general functions of the committee are to:

Review the progress of the evaluation and provide assistance in information gathering;
Serve as a conduit with their respective organizations/agencies/departments and the neighbourhood in order to facilitate information sharing;
Support the learning of residents about the functions of their respective organizations/agencies/departments;
Problem solve as needed