OR/15/019 Lessons learnt from applying the toolbox

Bonsor H C1, Oates N2, Chilton P J1, Carter R C3, Casey V3, MacDonald A M1, Calow R2, Alowo R4, Wilson P1, Tumutungire M5, Bennie M5. 2015. A Hidden Crisis: strengthening the evidence base on the sustainability of rural groundwater supplies – results from a pilot study in Uganda. British Geological Survey Internal Report, OR/15/019. 1BGS Edinburgh, 2ODI London, 3Wateraid UK, 4Wateraid Uganda, 5Makerere University

The overall aim of this UPGro catalyst grant was to develop a robust methodology which could be replicated by wider research to develop a systematic evidence base for understanding the underlying root causes of failures in groundwater‐based water supply in sub‐Saharan Africa. To test the toolbox methodology developed, a pilot study was conducted in Eastern Uganda as has been described in this report. An important part of this process has been to assess and review the toolbox approach for future improvement. A critique of the approach is now provided below based on the experience of implementing the methodology in the pilot study, and the analysis which was facilitated by the data generated.

Overview

Overall, the approach and methods used in the catalyst project have been shown to make a significant step to developing a replicable and robust methodology which can be used to generate a systematic evidence base for water supply failure. The work has:

• gone a significant way to encapsulating the complexity of the interlinked aspects of the problem, balancing the natural science and engineering ('technical') aspects of the research with those concerning the ability of communities to manage and maintain their water points (the 'social' aspects). The multiplicity of interlinked causes of water point failure was explicitly acknowledged and taken into account through the use of multi‐disciplinary field and analytical methods within the toolbox and research team.
• the study was firmly rooted in the realities on the ground, in communities and (literally) in the boreholes on which communities depend for their water point. The methods of investigation used were highly practical and appropriate to the information sought. This was not a theoretical study, nor one which was highly reliant on modelling, but was based on detailed observational science.

Despite the improvements to be made to the methodology of the catalyst project discussed below, the data collated by applying the toolbox and the diagnostic framework approach in the pilot study, have enabled a clear insight to the importance of corrosion, poor siting and poor supervision of contractors to water point failures in the area, against a backdrop of weak institutional support and inappropriate practices of implementers.

The key lessons learnt from the study approach and methods will need to be addressed in future research on this topic, and are discussed below. Some of these were due to the limited time and scope of the catalyst project, and were known from the outset of the research; others are more fundamental and have been realised through the work and the results of the catalyst grant.

Key lessons

Scope of the study
Due to the limited time and scope given by the one‐year catalyst grant, the study focused on information gathered from and about the community, and on the attributes of the water point itself. Therefore, the study was subject to several limitations which would need to be addressed in future studies.

1. The methodology did not explicitly investigate the role and actions of local or national Government, although many members of the research team have sound understanding of this important aspect. The limited ability of local Government to act in support of community management is well‐known, as the GoU Community‐Based Maintenance System (CBMS) permits only very meagre resources to be devoted to this task.
2. The surveys did not explicitly include the perspectives of private sector handpump mechanics (HPMs) in the study, and our understanding of their role and actions were based primarily on information provided by the communities (and to some extent, prior knowledge of the researchers and local partners). The HPM are the first line of support to WUCs when assistance is needed with handpump repairs.
   Sampling bias was generated in the pilot study results, by the study explicitly examining only borehole‐handpump water points which had been abandoned for several months. Those which were temporarily out of action pending repair were not included. While this approach was taken for practical reasons, it has limited the insights our data can provide regarding the full range of ‘failure types’. For example, community management capacity and other non‐technical factors may have been found to be more significant determinants of non‐functionality in a sample characterised by ‘repeated’ or ‘temporary’ failure rather than in the one‐off catastrophic terminal failures examined in the pilot study that had been abandoned for several months.
   As some of the water points were quite old, siting reports, construction records, test pumping or water quality data were not readily accessible. As a consequence the information gathered from communities about aspects such as siting, WUC training and supervision of construction could not be cross‐checked.
3. Importantly, the community surveys were focused heavily on examining the effectiveness and capacity of WUCs, and it did not include other, wider community management arrangements which may exist. There was also no higher‐level analysis of wider institutional arrangements. Recent work by Pahl‐Wostl et al. (2011) and Cleaver (2012) have found that polycentric community management arrangements often exist, and are highly significant to the capacity of communities to sustain water point functionality. There is, therefore, a need to widen the analytical scope of the methodology beyond the WUC.
4. The study involved a single ‘snapshot’ at the end of the dry season. Seasonal changes in water levels, yields and water quality and temporal variations in the significance of different causes of failure could not be examined in the study. The community surveys also relied heavily on a single visit to each village of 2–3 hours, so there were limited opportunities to follow up outstanding issues, or speak to different sub‐groups within the community to establish whether the data were representative.

For logistical reasons the work took place in two neighbouring districts of one country. A greater diversity and number of field sites would have been preferable. This would be possible with the same methodology, within a wider research programme, and is not a limitation per se of the project method.

Field methods
The field investigations were conducted in two discrete phases: one undertaking the community surveys, which involved a 2–3 hour group discussion with community members and the water user committee; the second phase undertaking the technical investigations at each water point. This was found to be an effective and efficient means of carrying out the fieldwork, allowing sufficient time for the project team to review the data collected in each phase and make decisions against agreed criteria on where the subsequent phase would focus. Working with local communities through the local NGOs partners and the WaterAid programme was very effective in facilitating the fieldwork and gaining trust from the communities to examine the water points.

The methods of the technical investigations were successful in generating a comprehensive post‐ construction audit. There were, however, some mixed results in the effectiveness of some of the individual methods of the downhole investigations. Some form of yield test is essential, and our experience was that bailer tests were generally more useful than conventional pumping tests, although where the latter were possible, they should be carried out. In the low productivity crystalline basement aquifer, which characterised the pilot study area, the aquifer permeability was often too low to support a conventional pumping test — and the pump available to the field team had a relatively high fixed discharge rate (0.5 l/s).

Bailer tests were found to be very useful in these conditions and provided a reliable proxy estimate of aquifer permeability. CCTV inspection was found to be best used at the end of a pumping test of purging of the boreholes, so that clear images of the borehole construction and any inflows could be obtained. The use of dissolved anthropogenic gas was found to be very useful technique, in conjunction with the CCTV surveys, to gain a better understanding of the main sources of shallow groundwater (inflows from the regolith versus deeper underlying bedrock) into the boreholes, based on their design.

The range of investigations and number of organisations involved in the fieldwork meant that in both phases of fieldwork the field teams were relatively large (e.g. in phase 1, translators, surveyors, NGO local partners were all required; in phase 2, hydrogeologists, handpump mechanic, translators, NGO partners and generator operators were demanded. A more efficient approach in future could be achieved by smaller teams (3–4 people), who are skilled in several aspects of the investigations.

The absence for many of the sites of construction records precluded comparison of some of the findings of the technical investigations (borehole depths, yields, groundwater levels, regolith thickness) with equivalent data at the time of drilling and any deterioration in these over time could not be assessed.

Conceptual diagnostic framework
The study set out to investigate methods to determine why water points (specifically boreholes with handpumps) fail. The science, philosophy and logic of causation is a major field of research in itself, and at the outset it was not clear which approach or approaches would be most suitable for the study. We considered several different root cause analytical approaches which are used in medical diagnosis, and others which are used in accident investigations and management studies.

The shape and structure of the relationship between cause(s) and effect (the failure of a water point) is not yet firmly established. Is there a chain of contributory factors, the failure of any one of which means that the chain breaks? Or is the analogy of a bicycle or a motor car more pertinent — in which forward motion is possible as long as certain conditions are fulfilled (tyres inflated, motive power working), but other appurtenances (e.g. bicycle bell, car windscreen wipers) are less critical?

As discussed above, there is a known bias in our results due to the site selection process which shifts us towards terminal failures rather than sources that are brought back into service. A positive outcome here is that there has been an in depth analysis of the drivers of terminal failure which in this case would seem to indicate that some of the user related social aspects are less influential whereas the actions (or inactions) of implementers and institutions are more influential. This is a significant result, and one which the DWD of MWE of Uganda has also found significant.

Future research
The pilot study and methodology recognize the inter‐disciplinary nature of the research issues, and that many contributory factors are interlinked in complex ways. However, the methodology of the pilot study remains multi‐disciplinary, and does not undertake truly inter‐disciplinary analysis of the research data. Future research will need to employ inter‐disciplinary analysis methods, and also widen the field data collation to a wider array of the institutional, financial and governance arrangements which determine service provision.

Future research also needs to reflect the roles of all players — national Government, local Government, development partners, private sector, civil society and communities — in defining the human and institutional dimensions of the problem. Long‐term trends in the natural groundwater resource, through changes in recharge, or demand also need to be examined, to fully understanding long‐term sustainability of rural water supplies.