Project Cycle Management and the art of Log-frame Analysis The widely-used strategic planning tool, Project Cycle Management, along with the Logical Framework approach, can ensure quality design for projects, and is simple to apply and monitor. S&T has been using it in the design of M and E systems for a number of government departments. Matthew Smith explains how it works
Strategy and Tactics is presently running workshops across
South Africa on behalf of the Department of Labour to assist Labour Centres
and provincial offices in preparing implementation plans for the Labour Market
Skills Development Programme. These plans are developed using the Project
Cycle Management (PCM) approach.
We are also helping to design a monitoring system for the
LMSDP based on the log-frames.
PCM and log-frames may appear to be new buzzwords in strategic
planning, but are neither new nor just another planning fad. Introduced by
the European Union a decade ago, PCM was developed to ensure the quality of
project design and to strengthen the quality of management on the project.
Anyone who has been involved in PCM will know that its value
relies on its simplicity. As the name suggests, PCM is a cycle which follows
a logical path. Each step relies on the success of the previous step.
Most projects are conceived as part of a programme, often
at a national level. As part of the programming phase, national problems are
identified and the objectives of the programme specified. Next comes the identification
phase, where projects (components of the programme) and other forms of intervention
are identified. This is followed by the formulation phase, in which ideas
for projects are turned into operational plans, and consultation takes place.
Once the proposals have been drawn up, financing is sought
and/or apportioned to the projects. At this point the project can be implemented.
Embedded in any coherent project design is evaluation, which can begin as
soon as the cycle begins.
Each stage of PCM is a mini-cycle that should be monitored
and evaluated. After implementation begins, evaluation becomes critical.
The results of the evaluation will feed back into the cycle
at the identification stage. This allows the management of the programme and
the project to assess whether the particular project is achieving its objectives.
If not, it may be decided that another project be introduced, or the existing
project be modified. S&T's preferred design sees evaluation as an ongoing
tool, utilised to judge and improve performance in all phases. It also
ensures that all players are open to evaluation, not just implementing
At the centre of the PCM cycle - in the version S&T utilises
- is ongoing monitoring, evaluation and reporting (M,E & R). All steps along
the PCM path are cycles in themselves, and each is open to measurement, assessment
and dissemination of the results (the M,E & R sequence). The danger of the
standard PCM design is that evaluation is left as a summative-only activity,
and reporting is frequently forgotten.
The core tool for PCM is the logical framework (often simply
referred to as log-frame), the construction of which is typically done in
two phases. In the first phase, the analysis phase, key problems and opportunities
are identified, from which objectives that will address these problems are
clarified. The underlying principle is that projects are designed to address
the problems faced by beneficiaries. In the second phase, the construction
of the log-frame is completed. Once the log-frame has been designed, activities
can be scheduled and resources can be allocated.
The log-frame matrix provides no magical solution, as the
official EU PCM handbook warns: 'if you put garbage in you will get garbage
out'. However, advocates of PCM argue that the log-frame is extremely useful
in that it assists with making logical connections between different levels
of the project. Moreover, it provides a useful aid to thinking strategically
about the project.The key is to remember that all plans, proposals, and log-frames
should be seen as dynamic tools that should be revisited and revised regularly.
The actual log-frame is a 4x4 matrix. Running down the matrix,
in order, is the overall objective of the programme, the purpose of the project,
the results of the project and the activities that the project will deliver.
The other three columns provide the basis for measuring the project's performance.
OVIs are indicators that provide a means for measuring the
objectives of the project and also lay the foundations for monitoring. The
SOVs are the sources of information and the methodology to be used to measure
the indicators. Assumptions are those factors external to the project which
could affect the implementation and sustainability of the project. Be warned
about the "killer assumption" as it kills the project dead.
The completed log-frame thus provides a concise summary of the objectives
of the project, the indicators by which the project will be assessed and the
key risks and assumptions which may affect the achievement of the objectives
of the project. From this log-frame a detailed workplan can be drawn up.
S&T: a year on Since our first newsletter the bulk of our work has been in the monitoring and evaluation (M&E) arena. S&T has worked with various national departments - including Labour, Public Works, Welfare, Constitutional Development and others - assisting both in the design of M&E systems and their implementation.
In numerous evaluation projects we have been actively involved
in shifting the mindsets of clients - including government and NGOs - to avoid
situations where we are called in only at the end of a programme to provide
final judgement on its worth and impact, too late to make key decisions based
on sound research findings. We believe that evaluation, in all its forms,
is a fundamental component of any social programme, interacting with regular
monitoring to provide important information concerning the direction and impact
of such pro- grammes.
S&T keeps in touch with theoretical developments in the academic
arena through conferences and workshops where we have presented papers for
critique and review, and submitting articles to various refereed journals.
We will also be hosting an international summer school on youth, aimed at
bringing together academics, policy-makers and youth workers, to bridge the
gap between theory and practice.
The focus of our work remains the ability to translate
theory into practice, and all our work - from involvement in government task
teams to evaluation of poverty relief initiatives - aims to provide the tools
and techniques of research and facilitation for decision-makers in order to
better inform their decisions.
Committed to the transformation of South Africa into a powerful
democracy and the development of Africa as a continent, we look forward to
the new millennium as we continue to contribute in our small way to this transformation
In Cape Town
The Cape Town office has played a significant role in several
major national education and health initiatives, including participating in
the research team that is measuring transformation in institutions of higher
education in South Africa, and the National Business Initiative's study of
further education and training in the Western Cape.
S&T worked on the Labour Market Skills Development Programme,
facilitating Project Cycle Management and Log Frame analysis workshops with
labour centres throughout the country. These will continue running for the
early part of the year, and this workshop has also been run for a sub-division
of the National Productivity Institute.
The Cape Town office had a busy year producing research reports and Sihaam
also produced a baby girl.
The hidden side of research: high quality fieldworkThe S&T approach to fieldwork is a developmental one, recruiting unemployed local matriculants training and deploying them in the field. The reasons? Enhanced quality, and a developmental investment in local human capital. The problems? Considerable, as Nobayethi Dube explains.
Having been involved in fieldwork training for research projects
for the past three years, one begins to understand that each project has its
own dynamics and challenges.
In planning a project, concepts and methodology are what
specialists normally worry about. A proposal puts forward the methodology
for the study, and once the client has accepted the project a brainstorm is
arranged to design the questionnaire or instruments to use in field.
Designing a questionnaire could take weeks, especially if
there are many interested parties involved. S&T normally goes through five
to six drafts before anything is shown to the client. We then pilot the questionnaire
to evaluate what works and what doesn't, once a final instrument is agreed,
the research team is ready to go out to field.
Unemployed local matriculants are used to administer questionnaires,
having been trained and sent out as teams to the various areas. Because of
the unemployment 'epidemic', more fieldworkers have tertiary qualifications,
but even so, a synergy of theory and practice must be achieved.
When planning for fieldwork teams, one has a set number of
days within which to complete the sample. A fieldwork manager will always
add days for 'unforeseen circumstances' which could affect the smooth running
of data collection and which will vary from one project to the other. It could
be violence, natural causes such as rain, areas which appear on the maps but
are not necessarily known on the ground and (visa versa), and so on. In some
instances, access may prove to be a problem where communities feel threatened
by strangers asking for household information.
Using local people known in the area may avoid some problems,
and should also lead to improved quality of data since there are no barriers
of language or dialect or the sense of an 'outsider' asking questions. Locals
know the area and are familiar with the culture.
Having chosen fieldworkers and supervisors (team-leaders)
one is now set to train teams on sampling procedures (i.e. the selection of
households and respondents) and administering the questionnaire.
During training it is imperative that the researcher who
designed the questionnaire is part of the training team. Some clients also
want to be present during training in order to introduce themselves and the
project. We encourage this, and fieldworkers appreciate the respect it shows
for their work.
In every training session, fieldworkers raise issues which
may not be obvious to researchers. These may relate to the phrasing of questions,
or perhaps to double-barrelled questions which may prove problematic during
coding. Language is an issue, it is very rare that questionnaires are translated
into vernacular languages because this is time-consuming and expensive. It
is therefore important to get consensus during training on how certain questions
will be asked. This is particularly true in metropolitan townships where people
grow up speaking up to four and five languages.
The first week of fieldwork is crucial. It is important for
the training team to be available for the issues and logistical problems that
may come up, because of the unpredictable nature of issues on the ground.
If fieldworkers are experiencing access problems, the researcher
may need to negotiate with local structures, amakhosi (traditional
leaders) or others. During data collection a co-ordinator may be stationed
in an area to act as a trouble-shooter. The co-ordinator is also responsible
for check-backs (S&T normally checks 25% - 30% of all interviews) and meeting
with teams during the survey.
Questionnaires don't always come back from field correctly
completed, questionnaires must be checked before coding takes place. Again,
coding poses a challenge because it is not only transferring numbers - coders
must understand the questionnaire and pick up minor mistakes. Once coding
is completed, questionnaires are sent for punching and then researchers can
plan for write-up.
It is only at this point that top-line results can be shared
with clients. Their first response is normally, "What took you so long?"
If only they knew!
Development programmes and the challenges of partnershipMuch has been said and written about partnerships in development initiatives , between government, civil society and business. There is space to enhance such initiatives. However, experience shows that many partnerships are not thought through carefully. Partners discover too late that they do not all share a common vision regarding roles and deliverables.
Entering into partnerships
In most tendering processes, companies are encouraged to
enter into partnerships. The argument is that the client can access a broader
range of expertise and services.
However, people from different sectors have different perspectives.
Private sector companies judge performance by time and money spent in delivery.
People from the development sector use these measures but alongside, local
participation, sustainability and so on. These are not merely questions of
style or surface: they are fundamental.
Some companies that enter into partnership are either an
emerging empowerment or new company, or well-established mostly white-owned
firms. In the development arena one encounters a scenario where engineering
firms want to collaborate with an emerging black company to maximise PDI scores.
(PDI scores are the points given to 'previously disadvantaged individual'
status by government departments and tender boards.)
Unfortunately, however, in many cases companies such as these
have difficulty in accepting a slower, uneven and more developmental approach.
These issues may be discussed while tender documents are being prepared, and
all sides may claim to be fully committed to empowerment. Once implementation
begins, however, many partnerships begin to struggle.
Faced with what amount to ideological differences - not merely
questions of style, as often suggested - partners suddenly find themselves
in a legalistic relationship, governed by Terms of Reference which become
guiding principles. Budgetary concerns, if not absolutely clear beforehand,
can become the central issue.
One side of the partnership expects the key concern to be
delivery within deadlines and budget, regardless of the extent to which communities
are on board. They voice concerns about foot-dragging and an over-concentration
on consultation. The other side points out that consultation is critical to
sustainability, and that delivering within time and budget is irrelevant if
the assets are left to disintegrate because the community does not feel part
of the project. Both are right, and the challenge is to find a balance of
both views within a single partnership.
Evaluating the Gauteng Youth Development InitiativesS&T has been commissioned by the Joint Enrichment Project (JEP) to undertake a formative evaluation of the Youth Development Initiative (YDI) of the Department of Welfare (DoW). The DoW set up the YDIs via its poverty alleviation programme. The YDIs, located in Bekkersdal, Cullinan, Daveyton, Dobsonville, Deep South, and Lanseria, are youth businesses that pay a social wage and re-direct profits to community development.
Each business is operated by a collective of young people
aged 18 to 35. The group's number varies from site to site with a minimum
of ten participants and a maximum of fifteen, who all receive business and
life-skills training from CBE&T, FEBDEV and JEP.
The business training should enable them to design Business
Plans and then run their own businesses. Life skills training offered by JEP
provides assistance with interpersonal problems and working together as a
The YDIs aim to generate income and create job opportunities
for other youth, or to help support local community initiatives. There is
a real challenge in linking entrepreneurship with collectivism.
Our methodology will combine qualitative and quantitative
measures. A focus group with participants from all six sites will be used
as a basis for designing a questionnaire, to be completed by participants
from all the sites. This will be complemented by in-depth interviews with
training providers and others. Analysis and reporting take place in early
Quantitative training for ANC officialsS&T was commissioned by the ANC to design, conduct and analyse 24 focus groups across South Africa, as part of a broader capacity-building initiative, in which David and Ross ran a two-day workshop training ANC provincial staff in quantitative analysis, complemented by the focus groups.
The training workshop looked at technical issues - sampling,
weighting, questionnaire design, question 'stacking' and so on. The samples
used for much of the public polling during the 1999 election were analysed,
and severe skews and biases became evident to all participants - and helped
explain why the public pollsters got the results so badly wrong.
On the second day, participants (who came from all nine provinces
as well as some from HQ) used existing data to assess their strengths and
weaknesses as the local elections approach.
Community profiles in the three poorest provincesNobayethi Dube tells us more about S&T's involvement in this project.
The Department of Public Works, Monitoring and Evaluation
Section, commissioned S&T to conduct community profile surveys in three provinces:
Eastern Cape, KwaZulu-Natal and the Northern Province. These will assist the
Department to plan better for future projects and measure the success of past
We used project lists provided by the Department, with project
number, District Council, project category, name of the project, project type
and budget size. We then stratified our sampling using the project number,
District Council, project category and budget size.
To establish starting points we used maps provided by Davies,
Lynn & Partners, with project numbers and areas, selected the identified projects
and drew a 10 km radius to identify the sample area on the maps. In each of
25 starting points, fieldworkers had to visit four households.
Our training, done by S&T and the Department of Public Works,
focused on sampling procedures and administration of the questionnaire. Our
fieldwork team in KwaZulu-Natal comprised social facilitators used by the
Department to work with communities, chosen because of their long-standing
relationships with communities and with the amakhosi. A co-ordinator
was based in the province for the duration of the survey.
However, using social facilitators did not prevent the problems
that normally accompany field research. Access was a problem in some areas
where the amakhosi felt that there was insufficient consultation and
they would not allow strangers to ask 'their people' questions. Extensive
negotiation was necessary, with District Councils in some instances intervening
to meet with all stakeholders. Team leaders were called in to explain at meetings
what the study was about and who would benefit. This meant that fieldwork
stopped for days whilst access was negotiated.
Fieldwork has been completed in KwaZulu-Natal, work in the
Eastern Cape is still in progress and Northern Province will be surveyed in
2000. The results will provide a fascinating look at the benefits of small-
and large-scale anti-poverty programmes.
Gauteng CBPWP Directorate workshopThe Gauteng Directorate of the Community Based Public Works Programme (CBPWP) is in the process of putting in place new systems and new staff. S&T assisted the Directorate to review its performance to date, focusing on issues relating to personnel, expectations, challenges and problems and agreeing on a proposed strategy.
Under discussion were programme issues within the Directorate that relate to implementation, the need to devise a targeting strategy for the province, project identification and selection criteria, and the extent to which the strategies used by the province are in line with the national strategy of the CBPWP. The following immediate key areas were identified:
- The province should undertake its own targeting strategy and spend its budget where it is most needed, focusing on poverty pockets and the best strategies to tackle these.
- It is important to realign the programme within the province, and it was agreed that the province should undertake a brief review in the form of a desktop study.
- A financial audit of projects implemented in the last 3 years must be linked to the design of a new implementation strategy for the directorate.
- Finally, where the need for capacity development is greatest, this must be implemented as a matter of urgency.
National Enterprise Survey This is part of a study of investment in South Africa commissioned by the Cabinet Investment Cluster, a Cabinet sub-committee chaired by Minister Alec Erwin. The study has been designed to understand the views of business - large and small - about the current business climate and its effects on investment.
S&T has been commissioned to undertake the survey among 1000 smaller firms (those with 50 or fewer employees). Face-to-face interviews are used with these smaller firms, and the results will be synthesised with those from the larger firm survey to provide a comprehensive picture of the current state of affairs.
The M&E system for the LMSDPDavid has been involved in the second phase of our work in the Department of Labour, specifically the Labour Market Skills Development Programme (LMSDP). Parts of the inception workshop were facilitated by David and Matthew who then assisted two of the six EU-funded projects through Programme Cycle Management and log-frame exercises; the next step was to design a monitoring and evaluation (M&E) system for the LMSDP as a whole.
David has been involved in the second phase of our work in the Department of Labour, specifically the Labour Market Skills Development Programme (LMSDP). Parts of the inception workshop were facilitated by David and Matthew who then assisted two of the six EU-funded projects through Programme Cycle Management and log-frame exercises; the next step was to design a monitoring and evaluation (M&E) system for the LMSDP as a whole.
David, working with British consultant Mike Felton of ITAD, completed the log-frames for the EU-funded projects (which comprise about half of the LMSDP). This was a necessary precursor to designing a monitoring system that will effectively utilise the OVIs and MOVs in a systematic and useful monitoring system. One area of difficulty is the great detail required in each log-frame, resulting in a great number of OVIs (also called Key Performance Indicators or KPIs) and a potentially massive quarterly monitoring report. The initial design was completed in December and work on refining the project continues.