OR/15/019 Development of a Toolbox – examining causes of supply failure

Bonsor H C1, Oates N2, Chilton P J1, Carter R C3, Casey V3, MacDonald A M1, Calow R2, Alowo R4, Wilson P1, Tumutungire M5, Bennie M5. 2015. A Hidden Crisis: strengthening the evidence base on the sustainability of rural groundwater supplies – results from a pilot study in Uganda. British Geological Survey Internal Report, OR/15/019. 1BGS Edinburgh, 2ODI London, 3Wateraid UK, 4Wateraid Uganda, 5Makerere University

One of the main aims of the catalyst grant was to develop a toolbox — a set of robust and repeatable methodological approaches — which can be used to investigate the causes of water supply failure in SSA. The toolbox provides a basis for researchers and practitioners to begin to develop systematic data on functionality. This section describes the development of the toolbox, and an outline of the methodologies contained within it. Section 5 of the report describes the application of the toolbox within the pilot study in Uganda, and Section 9 at the end of the report provides a detailed review of the toolbox and the lessons learnt on the methodological approach from the pilot study.

A borehole‐handpump supply in the Amuria pilot study area

Defintion of ‘failure’

A working definition of ‘failure’ of a water point was developed at the outset of this work, so that the toolbox could be developed to be comprehensive, applicable to a range of failure ‘types’. One of the key difficulties in examining functionality data is that different studies use different definitions of functionality or failure, and as a result, different data are collected, or it is not possible to compare the data meaningfully. For example, studies often only consider a few aspects of failure, and are biased towards new boreholes, excluding older boreholes which have gone through cycles of failure and repair, seasonal failure (e.g. low yielding) or which provide inadequate yield or water quality, but continue to be used in the absence of other water sources. As a result, a generous definition of failure and current functionality is often portrayed. ‘Failure’ is defined by this study as:

“the inability to supply sufficient quantity or quality of water for domestic drinking needs, year‐round”.

This encompasses supplies which have failed once catastrophically sometime after construction and initial use, and supplies which fail repeatedly and go through cycles of failure and repair, or seasonal failure.

Toolbox approach

The toolbox uses a variety of methodological approaches (e.g. community surveys, deconstruction of handpumps, pumping tests, water chemistry analyses) to collate a wide range of social science, engineering and hydrogeological data from a failed water point to provide a detailed post construction audit of the water point and the local governance arrangements. Equal emphasis is given to the social science and technical and engineering investigations within the toolbox. The data collated using the toolbox provides a detailed evidence base which can be used to assess the immediate causes of failure, as well unravel the deeper underlying root causes.

The toolbox is designed primarily as a field‐based tool, with the different data collation methods being employed in a rapid phased approach; an initial phase of reconnaissance work to identity the total number and different immediate symptoms of water point failure in an area; then focused research at a selection of the sites involving community surveys to ascertain community capacity and management arrangements; and finally technical investigations examining the possible engineering and hydrogeological aspects of failure — Figure 7. Field methodologies and techniques are described in full in the following sections.

Figure 7 A conceptual diagram illustrating the general strategy of the field methodology, with a phased approach combined community survey methods and technical investigations.

Toolbox methods

Reconnaissance surveys — identification of failed supplies

The aim of the reconnaissance survey is to identify the total number, and different symptoms of failed water points in a region. It is also essential to ascertain which communities are willing for more detailed investigations of the water point to be undertaken, some of which involve the removal of the hand‐pump and pump rods, and downhole measurements and tests. Key secondary data (e.g. borehole log, drillers log, construction/completion log, and any water quality analyses and pumping test report completed) are collated in the reconnaissance phase if accessible. These are vital information to be able to accurately interpret data collated by some of the later investigations contained within the toolbox.

Local practitioners and NGOs play a key role in the reconnaissance survey work, based on their knowledge of the communities and location of failed water points, from their regular travels throughout the districts, and discussions with communities. They also are able to provide information to the typical frequency and types of water point failures in an area.

The information collated in the reconnaissance survey gives a sufficient platform to select a representative sub‐sample of failed water points for future focused field research. Using a sub‐ sample of sites is often necessary due to the time and cost intensive nature of focused field research methods.

Community surveys — collection of data on local governance arrangements

The community survey is designed to collect basic information on community water use, reconstruct the history of the failed water point and, most importantly, explore the local socio‐institutional factors that may have contributed to non‐functionality. Key topics included:

• Community engagement in planning and construction of the water point
• Access to the water point (and alternative water sources)
• Water quality and yield
• Mechanical failures and repairs
• Water point management, rules and enforcement
• Fees and finances.

The survey methodology developed for the toolbox partly drew on the Water Economy for Livelihoods Systems (WELS) analysis approach developed and tested under the RiPPLE programme in Ethiopia (Coulter et al. 2010, Calow et al. 2013) and adapted by others (e.g. Dessalegn et al. 2013; Lapworth et al. 2011). The survey collects information on community water needs and access to different sources, as well as more detailed information on the non‐functional water point, its history and management.

A survey template was generated to enable accurate repetition of the survey within different communities. The template includes sections relating to:

• Planning and construction processes
• Breakdowns and repairs
• The Water User Committee roles and rules
• Capacities for management and enforcement, and finances.

Table 2 below provides more detailed information to the different components of the community survey and the information to be collated, and the survey template form is shown in Appendix 1.

Where two sample water points were located in one village, the water point‐specific sections of the survey were completed for each, whereas general questions regarding community attributes and available water sources were asked only once.

Table 2 Data collected through the community survey

Community survey section	Information to be collected
Community attributes (general)	Village name and location Community size (households, total numbers, population growth since installation of the water point) Community representatives Register of all participants
Local water sources (general)	Water sources available to the community (all types) Which sources are used and when (e.g. seasonality) and for what (domestic and productive uses) Key changes seen over the last 5‐10 years relating to water availability and use (e.g. droughts or floods, increasing demand)
Pre‐construction narrative	Extent to which intervention was demand‐driven Planning process and extent/nature of community engagement, including siting and design Existence of an MoU with the community Construction process (when, by who, whether there was any supervision) Community contributions (e.g. labour, materials, finance) Level of community understanding of the process Any perceived problems with construction/installation process
Post‐construction narrative	Water quality perception (taste, appearance, odour, change over time) Water quantity perception (sufficiency, seasonality, queuing times, pumping difficulties) Persistence of poor quality or yield Pump performance, breakdowns, repairs over time, length of time broken, frequency of mechanical failure
Community management and capacity	Presence of a WUC Whether the WUC was established before or after construction, their roles, training received Existence of by‐laws governing use and maintenance of the water point, and ability of the WUC to enforce these rules and regulations Any restrictions to on use of the water point (e.g. tariffs, opening times) Capacity of WUC to undertake or arrange for routine repairs and maintenance – skills, tools and spares availability Access to a handpump mechanic (HPM) Access to higher‐level support (sub‐county, district water officer, NGO)
Finance	Fees collected by WUC Expenditure (particularly costs of repairs) Perceived transparency of financial management Ability of community to raise necessary finance to maintain and repair the water point Willingness of community to pay for repairs of the water point

The design of the community survey reflects a trade‐off between depth of analysis and the time taken to complete the survey. Detailed community surveys and WELs analysis take place over several days and require several visits to the communities (Coulter et al. 2010). The aim of the toolbox is to provide a relatively rapid survey method, and technical investigations, which take no more than 1 day — to enable a larger sample size to be surveyed, and a statistically significant evidence base on water point failure generated. The community survey template was designed to take two to three hours, allowing a survey team to cover two communities a day (including travel to sites) in sufficient depth to identify key causes of failure. Multiple survey teams can be working in parallel in an area, enables more sites to be sampled in a day.

The survey template was designed to be used flexibly, as whilst it was important to ensure that key information was obtained for each site (to enable comparative analysis) the complex nature of water point failure meant that there had to be opportunities to delve into site‐specific issues as they arose in discussion, and to adapt questions to best draw out the relevant information, particularly around sensitive issues such as WUC performance or finances.

The primary survey data should always be triangulated with the information gathered in the reconnaissance visits wherever possible, as well as in the later technical investigations, which give additional opportunities for discussion with local community representatives alongside investigation of the technical causes of failure.

Dis‐assembly of a water point in Katakwi District; the fieldwork relied on strong support from and engagement with the local communities.

Technical investigations — collation of engineering and hydrogeological data

A suite of engineering and hydrogeological methods are used by the toolbox to develop a detailed post construction audit of failed water points. The technical investigations are designed to provide a comprehensive dataset on the hydrogeological and engineering factors contributing to water point functionality — including pump installation and condition, borehole construction, the surrounding environment and groundwater resource — see Table 3. None of the individual techniques used within the toolbox are new, but the use of all these techniques together with the detailed and careful deconstruction, measurement and photography of the failed pumps and boreholes, has not been done on a wide scale, or systematically, before by previous work in water point functionality, and is a major novel aspect of the toolbox. The different categories of investigations, and information and/or data collated within each of these sections are summarised in Table 3. Table 4 provides more detailed information on the different investigative techniques used.

The techniques in the toolbox include:

• Deconstruction of a water point to examine the conditions and performance of the pump, and rising main, as well as inspection of the construction and condition of the borehole using downhole CCTV.
• Local physical hydrogeological investigations — pumping tests, and bailer tests, to assess permeability and borehole performance
• Groundwater chemistry and residence time sampling using robust field sampling techniques for inorganic chemistry, stable isotopes and dissolved anthropogenic gases (CFC, SF6, and tritium) to investigate local water quality and also groundwater residence times to help gain insight to recharge rates and sources.

Field data collated from the investigations should be compared, wherever possible, with secondary data (e.g. drilling records, borehole logs) collated in the reconnaissance survey phase.

Table 3 The investigative techniques used to assess different possible technical causes of water point failure.

Possible (technical) causes of to water point failure	Investigations performed to establish status of different factors
Non, or partly functional pump operation	Time to fill rising main Discharge measurement Water taste, smell and aesthetic appearance General mechanical condition
Poor pump condition	Condition of head, handle, bearings, chain, flanges Number and condition (visual, length, weight) of pump rods and rising mains Condition of cylinder, cup seals, foot valve Diagnosis of faults if present
Inappropriate borehole design for aquifer resource, and/or low quality borehole construction	Total depth Rest water‐level Bottom of borehole (hard/soft) Verticality test (plumb line) CCTV survey inspection (casing joints, depth of screen horizon)
Limited groundwater resource potential and/or groundwater quality	Aquifer testing (either by bailer test and/or pump test) Well head chemistry measurements (pH, Eh, SEC, DO and temperature) Detailed inorganic groundwater chemistry sampling Dissolved gas sampling (groundwater residence time indicators)

Table 4 Field observations and techniques within the toolbox

Methods	Equipment	Equipment
Hydrological setting	Observation of position in the landscape: relative relief and proximity to drainage lines.	By eye
Location	Global positioning system, lat/long decimal degrees	GPS
Pump stand verticality	Spirit level placed in various positions on pump stand	Magnetic spirit level with verticality bubble
Dismantling of pump	Lifting of all below ground components by Hand Pump Mechanic (HPM) with help from team and community members	Full India Mark II tool kit including lifting spanners and vices, and camera
Pump apron information	Handwritten recording and photography of any markings on apron	Camera
Pump head manufacturer’s plate	Reading and recording, photography.	Camera
RWL and Borehole depth	Dipping of water level, plumbing of total depth	Graduated electronic dip tape with extra weight for use as plumb line.
Pump condition	Observation and description of condition of rod and pipe surfaces, colouration, thread condition, perforations. Measurement of pipe and rod lengths and weights of selected components	By eye. Camera. 5m steel measuring tape. Weighing scale
Borehole verticality	Centred plumb line cord in top of casing, carefully lowered to 50m depth, observed and estimated horizontal drift of cord from centre line.	Camera tripod and plumb line on 4mm nylon cord
Groundwater resource potential/aquifer permeability	Pumping test and/or Bailer test	Generator, submersible electric pump/ bucket or bailer and rope, timer, notebook
Groundwater chemistry	Well head measurements of groundwater chemistry (pH, Eh, SEC, DO and Temperature) Laboratory analysis of inorganic chemistry, stable isotope	Hand‐held analytical probes (well head measurements) Laboratory
Groundwater residence time	Measurement of dissolved anthropogenic gases (CFC, SF6, tritium)	Laboratory (BGS, IAEA or Flinders University Australia)
Additional information	Discussion with members of community, WUC, Local Council office holders.	Notebook

The quality of the data collated from the technical investigations is optimised if the investigations are performed in a set sequential order in applying the toolbox, as follows:

1. Deconstruction of the pump, rising main and inspection of the borehole occurring first;
2. The bailer test and latter pumping test occurring second (the pumping test only conducted if sufficient aquifer permeability to sustain an electric submersible pump discharge rate)
3. Groundwater chemistry sampling (both well head measurements, and collection of laboratory samples)
4. A repeat CCTV survey downhole survey to inspect the borehole construction and repair status, whilst the water‐level in the borehole is lowered as a result of the pumping test — clear images of the borehole structure, and main water inflows, can be obtained for a significant length of the borehole above the water‐level (below the water‐level the disturbed water is typically too murky to obtain clear camera images).
   Bailer tests form a very useful investigative technique in areas of low permeability aquifers. A bailer test is comprised of removing a set volume of water from the borehole using a manual bailer (or bucket) on a rope, and then timing the rate of recovery of the water‐level in the borehole, following the removal of the displaced water volume (MacDonald et al. 2005). Often in low permeability aquifers the local transmissivity is insufficient to conduct a full pumping test using a submersible electric pump. Carrying out the bailer test first gives a quick and reliable indication of transmissivity and whether it is worth undertaking a pumping test. In more productive aquifers the results of the bailer test can be used to select an appropriate pumping rate in the pumping test. If a pumping test is conducted after a bailer test, sufficient time should be left for the water‐level to fully recover between the tests.
   
   The groundwater chemistry samples and well head measurements should be taken after a suitable period of pumping (during the pumping test if conducted), to ensure the samples are of groundwater from the aquifer, and not water which has been residing in the borehole and in contact with air for a significant time.

Analysis approach

Existing studies and water point functionality surveys form an important knowledge base regarding the different symptoms and consequences of water supply failure. To understand why water supply services fail requires a systems approach which recognises the many different components of the service delivery supply chain, including the water point itself, and variety of factors (technical, social, institutional) contributing to failure, which interact and/or are interlinked. A systematic, diagnostic framework that allows for comparative analysis is essential to understand causes of water supply failure in different socio‐economic and environmental settings. With this aim, a root cause analysis approach was used to begin to develop a diagnostic framework for understanding why the water points examined in this pilot study had failed.

Root Cause Analysis (RCA) has traditionally been developed in a health and safety context — examining the conditions and decisions made which ultimately led to an accident or emergency situation (Ragen 1999; Livingston et al. 2010). However, the approach is also useful for framing the issues of borehole and handpump failure, and identifying the underlying root causes of the failure, rather than just the immediate symptoms and causes. There are various approaches to RCA, of which we took two to develop a diagnostic framework approach for examining causes of failure in groundwater‐based water supply systems.

The first RCA approach used was the The 5 whys approach. In this, a problem is stated and then five ‘why?’ questions are asked to delve into the underlying causes of water supply failure. For example:

Problem statement: borehole handpump has failed;
1 Why has the water supply service failed? — Because the yield was too low.
2 Why was the yield too low? — Because productive aquifer horizons were cased out of the borehole.
3 Why were the productive horizons cased out? — An inappropriate borehole design was used.
4 Why was the borehole designed inappropriately? — Because of inadequate understanding of aquifers and lack of application of good borehole design principles by implementing organisations
5 Why was there this lack of understanding and application? — Because of the absence of hydrogeological or engineering expertise in the design of programmes and contracts and a lack of supervision by someone competent to ensure the design was followed or to modify standard borehole designs on‐site if necessary.

The first ‘Why?’ statement represents the primary symptom of failure (e.g. insufficient yield) which is generally captured by existing functionality surveys (e.g. UNICEF 2012; RWSN 2009 , MWE 2010). The second, third and fourth ‘Why?’ statements all identify different factors contributing to water point failure, whilst the fifth ‘Why?’ begins to undercover the underlying root causes of the failure — Figure 8. In the example above, the absence of supervision by a hydrogeologist or trained engineer points to the underlying conditions that might underpin the water point failure — ‘why wasn’t a competent technical person present on site to ensure appropriate water point design and construction?’ This suggests either: the importance of having a competent individual present was not recognised by implementing agencies and local governments or there was not a sufficiently sized pool of competent individuals to draw from. Moreover, when implementing agencies and local governments outsource all aspects of drilling (including supervision) to contractors they may create an environment in which there is little interest in competent supervision; or implementing agencies and local governments are not willing to invest in the cost of technical supervision on site. Behind these answers there are further ‘whys’ linking to the deeper institutional capacities and arrangements which can make water point failure more or less likely.

A second RCA approach — Causal Link Diagrams — was then used to begin to examine how these different factors might interlink (Rooney and Vanden Heuvel 2004) — Figure 9. This was very useful in identifying the number and complexity of inter‐linked factors, and to begin to frame water point failure as a system. One symptom of failure can arise from different causes. For example, ‘insufficient yield’ (a symptom of failure) can arise due to low groundwater resource potential; inappropriate design of the borehole in relation to the aquifer resource; or, inappropriate siting and borehole depth; or, high demand pressures. As another example, mechanical failure can arise from: poor installation of the pump and inappropriate construction materials; or the inability of the community to finance and facilitate repairs. These two sets of causal factors may be linked — the community may not be willing to raise the finance for repairs if the handpump construction is poor and repeatedly fails. The ability of a water user committee to repair the handpump is dependent on the underlying borehole construction and siting as well as access to spare parts and available finance.

Figure 8 Hierarchy of symptoms, causal factors and underlying causes of water source failure.

The factors underpinning failure of a water supply are, therefore, numerous and inter‐related, and the causal link diagram (Figure 9) provides a useful diagnostic framework for visualising these layered and inter‐linked sets of factors which can lead to failure. To investigate which linkages have led to supply failure at any one water point requires a systematic field methodology to collect comprehensive data on supply failure.

Figure 9 A schematic causal link diagram of water point failure. Factors which can contribute to water point failure are shown in concentric rings around the failed water point in the centre. The arrows indicate linkages between different causal factors. Underlying conditions and root causes, which underpin these causal factors, and ultimately lead to the water point failure, are shown by the outer circle of red text.