YOU ARE HERE:
Access chapter 5: Learning from and improving HIV-related treatment work
   Last updated: 19.07.02
Summary
Chapter 5 of this handbook helps NGOs/CBOs/PLHA groups to develop a basic understanding of:

  • The importance of keeping track of their work and managing change.

  • Monitoring what is happening and making use of the information gathered.

  • Evaluating how the work has affected the needs of people with HIV.

  • Tracking the organisation's response to change within and outside the work.

  • Working out how to respond to changing situations.

  • Exchanging lessons with others involved in treatment work.

  • Deciding whether to increase or decrease their work, or keep it the same.


5.1.A. Keeping records of your work
A group involved in HIV-related treatment work needs records for managing and improving their work and responding to change. The aim of record keeping is to provide useful information to track what has happened in your work.

Different types of records are needed to tell the complete story about your work and the people you are helping. These include:
  • Quantitative records - showing facts and figures about what work has been done.

  • Qualitative records - showing information about human relationships, experiences and developments.


Both types of records can tell you about:

1. The people using your services.
  • Quantitative records - such as statistics about the personal details of people (age, gender etc.), their health problems, types and quantities of care received, treatments given, referral and follow-up appointments made and kept.

  • Qualitative records - such as needs assessment interviews, meetings involving clients, case studies and experiences in using your services


2. Your organisation.
  • Quantitative records - such as statistics about the management of the work, including details and numbers of staff, and records of drugs and other supplies.

  • Qualitative records - such as notes about discussions, meetings, interviews, decisions taken and policies.


It can be difficult to keep good records - as workers usually prefer to get on with their jobs and not spend time recording what they have done. They may not understand why the records are important. If records are kept but never used, people may stop bothering with them. There may also be a lack of relevant skills. It is important that:

  • Staff know why the records are important and how they will be used.

  • The process of making records is as quick and simple as possible.

  • Staff have the necessary skills for making the records you need.


Finally, a group needs to ensure that its records are accessible to anyone who needs to use them, while also ensuring confidentiality. It is important to think about who needs to hear about your experiences and to find interesting ways to communicate with them, such as using photo diaries or wall charts.

5.1.B. Monitoring your work
It is important to monitor the quantity and quality of work as an HIV-related treatment project progresses. Monitoring is an important management tool that can provide ideas and activities to:

  • Check on progress and report on it to staff, supporters and donors.

  • Help in day-to-day management and use of resources.

  • Share experiences and lessons learned with others.

  • Change what is not working and avoid duplicating what others are doing.

  • Plan for the future.


Monitoring is an on-going process, not a one-off event. It involves using records to regularly review work. The records that are needed for monitoring must provide useful measures of progress and change. For example, quantitative information about people receiving treatment can help to show:

  • The numbers of people seen and their health problems.

  • Their gender and age.

  • The numbers and types of drugs dispensed.

  • The number of community health education and awareness talks given.


You will also need to use qualitative methods of monitoring - which depend on observation, careful listening and questioning of the people involved in the work and people receiving treatment. Together, quantitative and qualitative monitoring will give a much broader and clearer picture of how work is progressing and if the people with HIV are getting what they need.

Key questions to ask when monitoring HIV-related treatment work include:
  • What type of quantitative and qualitative information do you need to know about your treatment work?

  • What records should be kept about the treatment work?

  • Who should keep the records and how should they be kept?

  • What should the records be used for and who should use them?


Case study: managing clinic queues
In a busy clinic, records showed that prescriptions were being dispensed very quickly. Patients received their drugs within one minute of handing the doctor's prescription to the person dispensing the drugs. Quantitatively, this seemed to show that things were going well. The queues were short, dispensing was quite accurate and the number of patients was high. But qualitatively, things were going badly. The patients did not remember being told anything about how to take their drugs, and the dispensers felt under pressure to keep the queues short or face criticism from their supervisor.

Understanding the whole story helped the manager of the clinic to work with the dispensers and their supervisor to improve the quality of their work. Together, they found new ways of managing queues while also taking time to check that patients understood their medication.

5.1.C. Evaluating your work
While monitoring is on-going, evaluation only needs to take place from time to time during the life of HIV-related treatment work. Evaluating work involves looking at the results of monitoring, asking probing questions about the information that has been gathered, and assessing how treatment work has gone. It can help to answer questions such as:

  • What has been achieved?

  • What difference has the work made to the treatment of people with HIV?

  • How do the achievements match with the organisation's goals?

  • What has been the cost - in human and money terms - for the organisation, people with HIV, and the community?


The key reasons to evaluate the work include:

  • To improve the effectiveness and efficiency of the work.

  • To help with future choices and decisions.

  • To learn lessons that can be shared within the organisation and with others.

  • To increase accountability to whomever has an interest in the work.


The reasons for evaluating your work will depend on who has encouraged the evaluation. For example, a donor may have asked for it in order to decide whether to carry on funding, or people with HIV and the community may have asked for it to see if the project is meeting their needs. Each group will have different interests and requirements. It is important to be clear about the expectations of those requesting an evaluation before starting it.

The process of evaluation can involve many different people, including:

  • Staff and volunteers of the organisation

  • People with HIV and other community members

  • Other NGOs, CBOs and PLHA groups

  • Donors and decision-makers


Participatory methodologies - such as mapping and priority ranking - are useful tools for evaluation because they help people to communicate more freely. Many of the tools included in Section 3 of this handbook can be adapted for evaluation, alongside other methods - such as reviewing records and holding focus group discussions.

It is also important to relate the evaluation of the treatment work to the assessment of the needs and resources of the group [see chapter 3.2.C.].

Evaluating people's experiences, organisation's activities and project management
Ways to evaluate people's experiences include:
  • Analysing facts and figures from monitoring activities.

  • Asking questions, using interviews and holding group discussions.

  • Using written questionnaires.

  • Analysing reports, case studies and people's stories about themselves

  • Observing activities and meetings.


Ways to evaluate an organisation's activities include:
  • Reviewing written information - such as reports, accounts and records.

  • Analysing the areas covered, numbers of people served and treatments prescribed.

  • Using verbal or written questions in questionnaires, interviews and surveys.

  • Observing activities and procedures.


Ways to evaluate the management of a project:
  • Reviewing records, reports, budgets, minutes of meetings and previous evaluations.

  • Directing verbal or written questions to individuals.

  • Interviewing individuals or teams.

  • Observing the work and practices.

  • Holding discussions with staff and other interested parties.


What to cover in an evaluation
An evaluation of an HIV-related treatment project should cover areas such as:

  • The people involved in the project

  • Treatment activities

  • Management activities

  • Linkages with others


Many evaluations focus on particular aspects of a project, but the following questions are common to most:

  • Effectiveness: To what extent is your work in HIV-related treatment achieving its objectives?

  • Efficiency: What has been the cost of achieving your objectives - in terms of the people involved and the funding?

  • Relevance: Is the work appropriate to the needs of the people you are trying to serve? Are there other problems that should take priority?

  • Sustainability: How long can the work continue and how can it find the money, people and other resources to do so?

  • Progress: Is the work achieving its original objectives, or have they changed?

  • Impact: What are the effects of the work on people with HIV, the community, the HIV epidemic, and your organisation? Has your work been helpful or made little difference? What effects has it had on economic, social and cultural factors? Has the impact remained local or spread wider? How does this affect your plans for the future?


5.2.A. Making the most of new resources and linkages
The overall aim of most organisations in HIV-related treatment work is to help communities to meet their treatment needs. However, these needs keep changing. It is important to be responsive and to assess needs on an on-going basis. No group working in treatment can expect to respond to everything, so it is important to keep in touch with new, emerging needs. Even if a group cannot do much about them, they may be able to encourage others to help.

New resources may become available. It is important to look continually and actively for new people, materials and information that will make a difference. However, new resources also bring challenges. They will provide more effective ways of getting things done and keeping up to date, but they may also require new skills, people and materials to use them well.

New linkages will also come along. Others may be starting to do work that is closely connected with your own objectives. The key to finding out how their work will affect your own will be communication - getting to know them and their priorities and making solid connections with them to allow the sharing of information or resources.

New resources and linkages for HIV-related treatment work
New resources and linkages might include things such as:
  • Accessing new treatment guidelines to help with choosing the most appropriate drugs.

  • Meeting people who can help you by supplying their skills or training your team.

  • Finding new sources of drugs at reduced prices.

  • Getting more support from the political and business communities.

  • Having increased access to e-mail and the internet - which widens the network of support and brings a lot more information.

  • Receiving extra funding - which allows expansion of work or keep your projects going for longer.


5.2.B. Coping with changes to your work
Change is a constant factor in the HIV epidemic and people's lives. An organisation must respond to changes in the community it serves. From time to time, it will also need to assess if change is necessary within its own organisation. Some unexpected changes may happen, but some can be predicted and plans can be made to deal with them.

As changes occur, new assessments of needs and resources should be made - to check that the treatment needs of the community are still being met. It will be helpful to return to the original assessments of needs and resources that were made before work began [see Chapter 3] - to see the progress and assess if the questions that were asked and the methods that were used are still appropriate or should be updated.

Responding to change is not a "one off" event. Instead, it can occur at any time during the cycle of work:

Identifying needs - setting aims and objectives - planning - implementing - reviewing and evaluating - identifying needs, etc.

Changes should be seen as opportunities to improve treatment access for people with HIV. The effects of change on your organisation must also be managed - as change that is too much, too fast or not enough can be confusing and stressful for staff and service users alike.

Changes that can affect HIV-related treatment work
Changes that can affect HIV-related treatment work include:
  • Changes in the pattern of the HIV epidemic, for example the numbers and type of people affected.

  • Changes resulting from treatment work. For example, some successful programmes have seen a reduction in problems such as diarrhoea and oral thrush, but also, because more people are now living longer with HIV, more cases of unusual opportunistic infections that require specialist treatment.

  • Changes in relationships within your group and with other individuals and organisations.

  • Changes in key material resources for treatment - such as new ways of using old drugs, the introduction of new drugs, and changes in drug prices.

  • Changes in other resources - such as funding, referral systems and human resources.

  • Political and economic change - whether local, national or global.


5.2.C. Sharing your experiences with others
This handbook is based on the experiences of many local organisations working in challenging conditions with limited resources in many countries. One of its key aims is to share the lessons that these organisations have learned through their practical work on HIV-related treatment.

Too often, efforts in treatment work are duplicated and mistakes are repeated just because one group's experiences are not passed on to others. Therefore, an important question to ask is: What can your organisation do to share the lessons from its work? What can you tell others about your successes, about things that did not work well, and changes you had to make? Sharing these experiences openly means that you and others can reflect and see how things might be done better in future.

Examples of how a treatment project might share its experiences with others include:

  • Linking up with other groups for experience sharing at meetings or workshops.

  • Documenting experiences in reports and briefing papers and distributing them widely.

  • Meeting with people with HIV and other community members to discuss the results of evaluations and to develop plans for the project.

  • Communicating key lessons and messages through the mass media, including radio and television.

  • Contributing articles to publications - whether your own organisation's newsletter or a national journal on HIV/AIDS.

  • Contributing to national and international debates, for example through the internet.

  • Sending representatives to national and international meetings and conferences to speak about your experiences.


The same methods can be used to help your organisation to learn from other people's experiences. This might involve building linkages with a variety of different organisations involved in treatment work, including:

  • Other NGOs and CBOs

  • Groups of people with HIV

  • Government and policy makers

  • Pharmacists and drug suppliers

  • Community health workers

  • Donors

  • Traditional healers

  • Religious groups

  • Media

  • Businesses


5.2.D. Scaling-up, scaling-down or staying the same
The results of monitoring and evaluating your HIV-related treatment work might tell you that your work is going well, you are reaching your goals and you do not need to alter anything. It is likely that results of monitoring and evaluation will highlight areas that you need to change. These results can be reviewed and assessed on a regular basis as a part of the annual review and re-planning process.

Key questions to ask yourself to assess if your work should change include:

  • Does the work still need doing?

  • Have we been doing it in the best possible way? Should we do a different kind of work to achieve our goals?

  • Should we be doing more work? What will happen to our group if we do more work?

  • If all or some of our work has to increase: How should it increase? When should it increase? Where should it increase?

  • Should we be doing less work? What will happen to our group if we do less work?

  • If all or some of our work has to stop: How should it stop? When should it stop? Will it be continued by someone else?


If your work has been successful, you may want to reach out to more people. But work often starts to expand in response to demand and organisations do not plan what their response should be. This can lead to management problems and staff "burn-out". There are a number of ways in which you can respond to this:

  • Scaling-up means finding ways to do more work - either by doing it yourselves or encouraging others to do it. There are many possible ways to do this - including expanding your organisational size, building partnerships with others and integrating treatment into other areas of work. It will be important to consider whether you have adequate resources to scale-up and can maintain the quality of your work.


  • Scaling-down means finding ways to do less work - as a response to limited or reduced resources and/or needs. This may be a wise decision if your group is being stretched beyond its capacity or if other organisations are responding more effectively to the needs you aimed to address. It can help to ensure that the quality of your work is not compromised by trying to do too much.


  • Staying the same may be possible, although it should be remembered that change is part of life. Even to continue with the same type and amount of work, you will need to stay in touch with developments affecting people with HIV and your organisation. Technical skills must be kept up to date, staff replaced as they move on, and team-building encouraged.


5.3. Want to know more?
An Evaluation of the Ministry of Health/NGO Home Care Programme for People with HIV/AIDS in Cambodia. International HIV/AIDS Alliance, June 2000.
-an evaluation report of an NGO and government run home care programme in Cambodia

Expanding community action on HIV/AIDS - NGO/CBO Strategies for Scaling-Up. International HIV/AIDS Alliance, June 2001.
- a report sharing results of workshops for NGOs and people with HIV to exchange successes and lessons learned in scaling-up successful community based work

Toolkits - A Practical Guide to Assessment, Monitoring, Review and Evaluation. Save the Children Development Manual 5. Save the Children Fund. 1995.
- a collection of principles, practice and tools for NGOs, including who to involve, uses of information; applies to all kinds of development projects

Comfort and Hope: Six Case Studies on Mobilising Family and Community Care for and by People Living with HIV/AIDS - Best Practice Collection. UNAIDS. June 1999.
- a collection of experience (case studies, good practice, lessons learned) from six care and support projects.

Source: Access to Treatment Handbook
This is an extract from Mobilising NGOs, CBOs and PLHA groups for improving access to HIV/AIDS-related treatment: a handbook of information, tools and other resources, developed by the International HIV/AIDS Alliance with support from the World Health Organization (WHO) and the Joint United Nations Programme on HIV/AIDS (UNAIDS), published in this form in June 2002.

To view the whole publication follow this link.

To download this section, complete with graphics, in pdf format (which requires Adobe Acrobat software to read it) follow this link (file size 517 Kbytes).