YOU ARE HERE:
Home Care - 2.6 Limitations of the evaluation
2.6 Limitations of the evaluation
The absence both of baseline data and a comparable control group imposed methodological difficulties in evaluating the impact of the programme. The logistical and ethical difficulties associated with obtaining a suitable control group have been outlined elsewhere in the report.
The measurement of impact therefore relied extensively on the perspectives of the PLHA who are the primary stakeholders in the home care programme. As the patient interviews were conducted by the home care team co-ordinators, there was the possibility of interviewer bias. An effort was made to reduce bias by not allowing team co-ordinators to interview their own patients, and by close supervision of approximately one third of the interviews by the evaluation team.
There was some variation in the quality of the interviews, due largely to the lack of privacy, the sensitivity of the subject matter and the health status of the interviewees. Nevertheless, data collected in these interviews was triangulated with data from other sources, and found to be consistent.
The presence of the evaluation team members during observations of home care visits may have affected the quality of the interactions between the home care team and the patients.
The quality of the monthly statistics on patient numbers and visits was sometimes inconsistent. On checking a sample of the home care record sheets with the monthly statistical summaries, some arithmetical errors were found. One of the home care teams counted every person they interacted with professionally (either a PLHA or merely someone in the community requesting information or medicines) as a “patient”. Another team counted any professional interaction as a “visit”. The evaluation team attempted to correct these anomalies, in collaboration with the Khana staff and home care co-ordinators. When the number of patients and visits are averaged over 12 months for the 10 teams it is estimated that these errors will amount to less than 5%.
Finally, although the study explored impact related to improvements in a number of quality of life indicators for PLHA and their families, no attempt was made to measure lifespan or health status of PLHA visited by home care teams compared with those who did not receive visits.
The absence both of baseline data and a comparable control group imposed methodological difficulties in evaluating the impact of the programme. The logistical and ethical difficulties associated with obtaining a suitable control group have been outlined elsewhere in the report.
The measurement of impact therefore relied extensively on the perspectives of the PLHA who are the primary stakeholders in the home care programme. As the patient interviews were conducted by the home care team co-ordinators, there was the possibility of interviewer bias. An effort was made to reduce bias by not allowing team co-ordinators to interview their own patients, and by close supervision of approximately one third of the interviews by the evaluation team.
There was some variation in the quality of the interviews, due largely to the lack of privacy, the sensitivity of the subject matter and the health status of the interviewees. Nevertheless, data collected in these interviews was triangulated with data from other sources, and found to be consistent.
The presence of the evaluation team members during observations of home care visits may have affected the quality of the interactions between the home care team and the patients.
The quality of the monthly statistics on patient numbers and visits was sometimes inconsistent. On checking a sample of the home care record sheets with the monthly statistical summaries, some arithmetical errors were found. One of the home care teams counted every person they interacted with professionally (either a PLHA or merely someone in the community requesting information or medicines) as a “patient”. Another team counted any professional interaction as a “visit”. The evaluation team attempted to correct these anomalies, in collaboration with the Khana staff and home care co-ordinators. When the number of patients and visits are averaged over 12 months for the 10 teams it is estimated that these errors will amount to less than 5%.
Finally, although the study explored impact related to improvements in a number of quality of life indicators for PLHA and their families, no attempt was made to measure lifespan or health status of PLHA visited by home care teams compared with those who did not receive visits.