home
the team
newsletters
about
cape town
stnews
downloads
links
 
   

This article is taken from the December 2002 Phatlalatsa newsletter

 

Challenges in evaluating ANTI-POVERTY PROGRAMMES

Challenges in evaluating ANTI-POVERTY PROGRAMMES imageS&T partner Ross Jennings has considerable experience in evaluating small development projects and large development programmes. In this article he outlines some of the key challenges he has encountered.

Tackling poverty and measuring impact

South Africa faced massive challenges in the 1990s; tackling the poverty in which many citizens live was among the most important. Using a basic needs approach, the African National Congress provided a national blueprint (The Reconstruction and Development Programme) for integrated development. Since 1994, significant policy and regulatory reviews have been undertaken. The RDP was superceded by the Growth, Employment and Redistribution (GEAR) financial strategy; commentators now tell us GEAR is dead, and we wait for the next acronym to arrive.

We have a battery of tools and terms: integrated development and sustainable development, poverty that is chronic or transitory, verifiable and key performance indicators, and toolkits that claim to measure everything.

How effectively and efficiently has government tackled poverty? How useful have these tools been? This article tries to shed some light on these questions by looking at some of the challenges of evaluating government's anti-poverty programmes.

What is poverty?

One of the major challenges in evaluating government's anti-poverty programmes relates to definitional issues. A recent study of the monitoring and evaluation practises of national departments involved in anti-poverty programmes (commissioned by the Department of Provincial and Local Government) found multiple and varying definitions of poverty. Different interpretations of the "poorest of the poor" resulted in programmes targeting different geographical areas and groups of people. What do anti-poverty programmes do?

The problem of definition is further compounded by the differing objectives and goals of anti-poverty programmes. These objectives and goals commonly mix deliverables and concepts - capacity building, empowerment, community participation, sustainability, high social impact, economic growth, job creation and so on. Many are differently understood and defined across different programmes.

Definitional differences:do they matter?

So what is the problem with these different definitions, in the context of evaluation? We need to remember that evaluations are meant to measure programmes against their objectives, identify strengths and weaknesses, and produce recommendations that are realistic and lead to an improvement in delivery. One could argue that there is no problem with differing definitions as long as each individual programme clearly states its objectives at the outset against which it is then evaluated.

What to measure?

The problem, however, manifests at two levels. At the level of each individual programme, the absence of shared definitions has hampered the development of measurable indicators. This, in turn, restricts the ability of any evaluation to comprehensively measure the programme against its objectives.

Aggregation

The second level is to be found in government as a whole. The different definitions across programmes do not allow for consolidation of information and findings. It becomes methodologically problematic (if not impossible) to evaluate delivery by government as a whole.

What to do?

Given South Africa's history, the pressure on government to deliver (especially through its anti-poverty programmes) is considerable. Recognising this pressure, the government is looking to develop a comprehensive system for performance and progress monitoring across its departments, directorates and programmes. In the poverty arena, the Integrated Sustainable Rural Development Programme (ISRDP) has been designed to provide an integrated response to poverty.

The ISRDP, however, is not a programme but a strategy for co-ordinating delivery of existing programmes around priorities set at local level through Integrated Development Plans (IDPs). It is imperative that goals and objectives are clearly defined and measurable indicators developed for each of the key areas within these definitions. The various anti-poverty programmes then need to ensure that their own goals and objectives are in line with government's overall policy thrust.

Integration

It is important to remember that integrated delivery demands integrated evaluation. Evaluations of individual programmes remain important. However, given that many of the programmes work in the same areas often delivering very similar services, some form of cross-over in evaluating delivery will limit duplication and be more cost-effective.

This cross-over will also help address the fact that programmes do not operate in a vacuum - methodologically, evaluations of individual programmes are unable to conclusively tie any noted effect/impact to the actual programme under scrutiny.

Furthermore, cross-over should allow resources to be allocated to longitudinal studies, rather than the standard approach of only conducting summative evaluations at the end of each funding cycle. This would allow richer, more accurate data to be gathered on issues such as empowerment, sustainability and impact, which can only be effectively measured over time.

 

[top] [to ] [Previous page]
     
home
the team
newsletters
about
contact
stnews
downloads
links