Impact evaluations (IEs) are essential for determining the effect and effectiveness of USAID programs on development outcomes. IEs depend wholly on accurate identification of comparison communities and the collection of high-quality data. To enhance both factors, ERC continues to develop and perfect new evaluation methodologies and approaches. The recently launched baseline data collection for the Land Administration to Nurture Development project (LAND) IE in Afar, Ethiopia, which is focused on pastoral areas of the region, uses two such innovative methods.
To improve the accuracy of sample selection, a community listing was conducted before the launch of baseline data collection. For community listings, the survey team visits each prospective treatment and control community and conducts a short, census-style survey with a community leader. The survey collects general community characteristics such as population, number of female-headed households, settlement patterns, and grazing routes, and GPS coordinates of key community areas. Using the listing and GPS data, Geospatial Analysts in E3/Land developed a list of indicators derived from geospatial data, including road density, market access, and the prevalence of invasive bush species. These indicators were used to create a subsample of matched treatment and control communities for the larger data collection effort of the IE. Since national organizations rarely have accurate community or micro-level data, the listing process allows the research team to select more accurate control communities, which strengthens the validity of the evaluation.
To improve the quality of the baseline data, ERC designed a rigorous mapping exercise in which facilitators lead local herders and scouts in drawing their wet and dry season grazing areas, migration routes, settlement areas, water points, and important natural landmarks. Large format base maps were prepared, then participants used color-coded markers to denote different areas and points of interest. The maps created in these exercises can be digitized and analyzed using geographic information system (GIS) software to generate quantitative data including travel times to grazing areas, spatial patterns in land and water point management, and spatial patterns in lost access to land and water points. Despite its importance to pastoral communities and the programs affecting them, this type of data is largely undocumented and represents a major knowledge gap which ERC is attempting to fill.