1
Evaluation Approach Paper
World Bank Engagement in Situations of Conflict
April 24, 2020
1. Background and Context
1.1 The World Bank Group has made a strong commitment to addressing the
development challenges associated with fragility, conflict, and violence (FCV) as part of
its corporate goals. It situates this challenge at the core of its poverty reduction focus,
especially since extreme poverty is rising in fragile countries (Cuaresma et al. 2018). By
2030, it is estimated that over 50 percent of the world’s extreme poor will live in fragile
and conflict-affected situations (FCSs). Achieving development outcomes in FCV
countries is also critical for achieving the Sustainable Development Goals. On average,
only 18 percent of FCV countries are on track to achieve unmet needs targets in goals 1
to 7 and 11 (Samman et al. 2018).
1.2 To fulfill this commitment, the World Bank has recognized that it must do
business differently. It has done this by providing conflict-affected countries with
enhanced assistance through increased and diversified financing and by updating
operational policies to enable countries experiencing emergencies to receive expedited
assistance. International Development Association (IDA) financing, in nominal terms,
has significantly increased for countries on the World Bank’s List of Fragile and Conflict-
Affected Situations (figure 1.1) that have experienced conflict.
1
Innovations to the FCS
financing tool kit include reform of IDA allocation framework, debt relief, the Crisis
Response Window, the Global Concessional Financing Facility for middle-income
countries, the Refugee Sub-Window, exceptional allocations, multidonor trust funds,
and most recently an FCV Envelope.
1.3 Notwithstanding these achievements, ensuring that the World Bank engages
effectively in these contexts requires more than increased and diversified forms of
financing. To this end, a multipronged approach to engage along the conflict spectrum
has been proposed in the World Bank Group Strategy for Fragility, Conflict, and Violence
2020–2025. This includes (i) engaging upstream to prevent violent conflict and
interpersonal violence, (ii) remaining engaged during conflict and crisis situations, (iii)
helping countries transition out of fragility, and (iv) mitigating the spillovers of FCV
through, for example, support for refugees and host communities (see figure 1.2). The
approach is grounded in the findings of the World Development Report 2011: Conflict,
Security, and Development; the United Nations–World Bank Pathways for Peace report; and
Forcibly Displaced: Toward a Development Approach Supporting Refugees, the Internally
Displaced, and Their Hosts? among other products.
2
Figure 1.1. Financing to Countries on the List of Fragile and Conflict Situations with
Medium- to High-Intensity Conflict Has Increased and Diversified*
Source: World Bank (various years); Independent Evaluation Group (2020).
Note: Countries with medium- to high-intensity conflict are defined by the World Bank Group Strategy for Fragility,
Conflict, and Violence 2020–2025 (see methodology in appendix B). Fiscal year 2020 includes the pipeline of projects. FCS
= fragile and conflict-affected situations; IBRD = International Bank for Reconstruction and Development; IDA =
International Development Association.
Figure 1.2. Multipronged Approach for Conflict Situations from the World Bank
Group’s FCV Strategy
Source: World Bank Group Strategy for Fragility, Conflict, and Violence 2020–2025.
Note: FCV = fragility, conflict, and violence.
1.4 The World Bank has also recognized that implementation of the FCV Strategy
requires more effective partnering along the humanitarian-development-peacebuilding
3
nexus. This includes various forms of cooperation and coordination with actors such as
development partners, the United Nations (UN), and humanitarian agencies, as well as
interactions with nonstate actors that can be a party to conflict. Although the World
Bank has worked with various actors in situations of conflict in the past, it has begun to
formalize relationships through memoranda of understanding with the UN and other
agencies.
2
2. Evaluation Purpose and Scope
Purpose
2.1 The purpose of the evaluation is to examine the relationship among various
modalities of World Bank engagement in situations of conflict and the achievement of
development gains. The evaluation is designed to focus on how the World Bank is
working differently in conflict-affected countries, why engagement decisions are made in
different contexts, and what contributions the World Bank has made to development
gains.
2.2 The evaluation categorizes engagement, in an instrument-neutral manner, along
the following lines:
• Approach: such as instrument selection, sequencing, prioritization, and risk
assessments;
• Temporality: activities that occur before, during, and after situations of conflict;
and
• Actors: who the World Bank interacts with in situations of conflict (for example,
state and nonstate actors, development partners, humanitarian agencies).
2.3 The evaluation will seek to surface operational lessons from experience to help
inform implementation of the FCV Strategy but will not evaluate the strategy. It uses
evaluative tools to assess past and ongoing engagements to support corporate and
operational learning needs. It will surface lessons on how World Bank engagements are
contributing to development outcomes in some of the most difficult contexts: in situations
of conflict. The evaluation is also designed to provide inputs to IDA’s FCV Special Theme.
Scope
2.4 The evaluation will focus on a set of countries that have (i) experienced medium-
or high-intensity conflict since 2014 per data obtained from both the Armed Conflict
Location and Event Data Project (ACLED) and the Uppsala Conflict Data Program
(UCDP) and (ii) been included on the World Bank’s List of Fragile and Conflict-Affected
4
Situations (figure 2.1). The cut-off for country conflict activities (2014–present) was
chosen to enable a deeper assessment of countries that have recently experienced
conflict to ensure operational relevance. The definition of high and medium intensity is
derived from the World Bank’s FCV strategy methodology.
2.5 The FCV strategy defines high conflict intensity by the number of absolute and
relative conflict deaths, and medium intensity by both the absolute and relative number
causalities as well as change over time. Per the FCV Strategy definition, high-intensity
conflict countries have (i) an absolute number of conflict deaths of more than 250
according to ACLED and 150 according to UCDP and (ii) a number of conflict deaths
relative to the population above 10 per 100,000 according to both ACLED and UCDP.
Also per the FCV Strategy, countries with medium-intensity conflict over the evaluation
period are defined as having (i) an absolute number of conflict deaths of more than 250
according to ACLED and 150 according to UCDP and (ii) between 2 and 10 per 100,000
population according to ACLED and between 1 and 10 according to UCDP or (iii) a
rapid deterioration of the security situation, as measured by (i) a lower number of
conflict deaths relative to the population between 1 and 2 (ACLED) and 0.5 and 1
(UCDP) and (ii) the number of casualties more than doubling in the last year.
Figure 2.1. Evaluation Scope: Countries with Medium- or High-Intensity Conflict since
2014 on the List of Fragile and Conflict-Affected Situations
Sources: List of Fragile and Conflict-Affected Situations (FY10–20); UCDP and ACLED databases, most current years; FCV
Strategy Definition of Conflict Intensity
FY = fiscal year.
5
2.6 The evaluation does not address all aspects of the World Bank’s engagement in
countries on the List of Fragile and Conflict-Affected Situations. As shown in figure 2.1,
the evaluation excludes countries on the List that have only experienced fragility (for
example, institutional fragility, including most small island states). Likewise, the
evaluation does not include countries that have experienced conflict but that were never
on the List of Fragile and Conflict-Affected Situations. This evaluation also does not
cover issues pertaining to extreme violence, as opposed to conflict. Therefore, several
countries (especially in Latin America and the Caribbean) are excluded. An analysis of
the World Bank’s engagement requires analyses of activities conducted before, during,
and—when relevant—after conflict, so the evaluation period has been scoped to cover
the period between fiscal year (FY)10 and FY20.
2.7 This evaluation builds on the existing body of work on FCV issues of the
Independent Evaluation Group (IEG). Previous IEG evaluations that will be used
include the following: World Bank Assistance to Low-Income Fragile and Conflict-Affected
States (2014); World Bank Group Engagement in Situations of Fragility, Conflict, and Violence
(2016); Recent World Bank Experience with Risk and Resilience Assessments and Operational
Programming in FCV Countries (2018); and Learning from IDA Experience: Lessons from IEG
Evaluations, with a Focus on IDA Special Themes and Development Effectiveness (2019). To
minimize overlap with other IEG work, it will not address International Finance
Corporation activities or broader human resource issues pertaining to the World Bank’s
global footprint.
3. Evaluation Questions
Context
3.1 Evaluation questions were developed based on existing strategic and analytical
work within the World Bank on the topic. Specific lessons relevant for this evaluation
that have been incorporated in the FCV strategy include the following:
• World Bank engagement in countries affected by conflict should be based on
assessments and due diligence to determine which situations warrant engagement
during crisis and conflict. Deciding when to remain engaged should depend on a
clear analysis of the unique benefits and the added value of the World Bank
program in relation to other organizations.
• Effective engagement requires the World Bank to leverage its analytical depth to
stay prepared to engage in postcrisis interventions. As stated in the strategy, early
preparedness is essential for successful recovery. This includes timely political
economy analysis of some critical sectors, such as energy and extractives, which
should be carried out in anticipation of an intensification of engagement. It also
6
involves monitoring the macroeconomic situation because monetary and
exchange flows require special attention during active conflict. Analytical work
incorporating differential impacts across households and firms (for example,
Poverty and Social Impact Assessments) and mobilizing dedicated FCV country
economic expertise is necessary to help macroeconomic adjustment and debt
sustainability efforts be conflict sensitive.
• When engaging in conflict situations, focus should be placed on building
resilience, protecting essential institutions, and delivering essential services.
• Partnership is a key instrument to be employed by the World Bank. This allows it
to deliver essential services while remaining clearly focused on its development
mandate of achieving poverty reduction and shared prosperity.
Evaluation Questions
3.2 The key question to be addressed by this evaluation is how relevant and effective
World Bank engagement has been in contributing to the achievement of development
gains in situations of conflict.
3.3 To answer the main evaluation question, the following subquestions are
proposed:
• Q1: How relevant and adaptive has World Bank engagement in situations of
conflict been in terms of sequencing, prioritization, and instrument choice?
• Q2: How well has the World Bank identified, managed, and mitigated conflict-
related risks?
• Q3: How strategically and effectively has the World Bank worked with state
actors, nonstate actors, and development partners in pursuit of its development
objectives?
• Q4: What outcomes has the World Bank contributed to in situations of conflict?
To answer the fourth evaluation question, outcomes will be identified and assessed
holistically (see box 3.1). An inductive approach will be used to (i) capture process-
related outcomes separately from results in conflict-affected settings, and (ii) surface
examples of alternative metrics that can measure development gains more relevantly.
The proposed approach includes an analysis of Country Partnership Frameworks (CPFs)
and project-level results frameworks. It also includes in-depth interviews with a range of
country-relevant stakeholders (for example, World Bank staff, clients, development
partners, beneficiaries) that will be used to triangulate outcome-related claims.
7
Box 3.1. Assessing Outcomes in Conflict Situations
In the context of this evaluation, outcomes will be inferred and assessed against a range of
high-level objectives associated with the World Bank’s engagement. Project-level results for
lending activities will be tabulated and analyzed from the point of view of project realism,
conflict sensitivity and the identification of ex ante conflict-related risks, and evidence of
adaptive management. However, overall outcomes will be considered in the context of the
range of activities undertaken, both tangible and intangible (such as signaling, convening). For
example, in conflict-affected countries, outcomes can be achieved through maintaining
development gains or by preventing them from slipping backward. As such, the evaluation will
not rely exclusively on a portfolio review (of project outcomes and ratings), as these may not
form a dominant part of World Bank engagement in many cases.
4. Evaluation Methods
4.1 Given the idiosyncratic nature of conflict, the evaluation is country focused. It
includes three main levels of analysis and a range of methods.
4.2 At the level of the universe of countries on the List of Fragile and Conflict-
Affected Situations that have experienced conflict of at least medium intensity (n = 23),
the team will complete the following tasks:
Macrodata Level
• Data analysis. The assessment will analyze data and trends pertaining to the
countries selected for assessment, including conflict-related data, nonlending and
lending typologies (for example, sectors and themes), financing, instrument
selection, human resource data, and so on. The data analysis will juxtapose
longitudinal data (2006–19) on World Bank lending and nonlending operations
and activities with country-level conflict data, derived from the UCDP and
ACLED databases, to understand how the World Bank engages in areas affected
and not affected by conflict during different periods of conflict intensity and
complexity.
Country Strategy and Portfolio Level
• Review of regional and country conflict-related literature with a focus on
conflict dynamics and risks. At the country level, the evaluation will categorize
and analyze conflict as the result of both endogenous and exogenous drivers to
determine the extent to which the World Bank’s engagement in environments
characterized by drivers differs systematically.
8
• Structured analyses of World Bank conflict analyses and integration into the
Systematic Country Diagnostic or equivalent analysis and CPF (or strategy).
The evaluation will systematically assess the risks identified in available World
Bank conflict-related risk analyses, including fragility assessments, risk and
resilience assessments, political economy and governance analyses, country
social assessments, peace and conflict filters, and so on. The evaluation will
analyze how and how well risks identified in conflict-related assessments have
been incorporated into Systematic Country Diagnostics and CPFs to inform
conflict-sensitive development planning.
• Sequencing and coherence analysis of the country portfolio (lending and
nonlending) in line with conflict-related diagnostics and recommendations. At
the country portfolio level, the evaluation will assess how identified risks and
conflict-related recommendations (for example, sequencing, prioritization,
instrument choice) have been incorporated into country-level decision-making.
• Results analysis. At the CPF and portfolio level, the evaluation will review and
assess the country results frameworks and key projects to identify and derive
lessons on the types and levels of conflict-related metrics that are being used to
assess results and outcomes in conflict-affected situations.
Project Level
• Conflict sensitivity analysis will be conducted of key lending operations using
the analysis of risks derived from the literature and World Bank conflict-related
analyses and assessments (for example, fragility assessments, risk and resilience
assessments, conflict and peace filters).
• Spatial analyses. The assessment will undertake geocoded data analysis as a
basis to understand the relationship between the World Bank’s engagements and
the geographic nature of conflict (that is, subnational, national, diffuse,
concentrated). Lending operations will be geocoded, where coordinates are
available (and if specific project activities have specific geographic beneficiaries)
and mapped over geographic country conflict layers. To do this, the assessment
will build geotagged country maps using UCDP and ACLED data to visualize
geographically the presence and intensity of conflict.
Cross-Cutting Methods
4.3 Semi-structured interviews. The assessment will conduct semi-structured
interviews with key country stakeholders to collect richer qualitative data that helps
flesh out preliminary findings from the data and content analysis. Key experts and
9
World Bank staff will be selected based on their exposure to the World Bank’s decision-
making and operational processes in one or more of the countries during conflict-
affected periods through a stakeholder analysis. Questions may relate to institutional
incentives and decision-making processes that are not evident from the documentation
(including the political economy, risks, and resources). Questions may also probe issues
of the choice and cost of inaction (and other counterfactuals), and so on (evaluation
questions 2–4). Deeper dives, including on-the-ground research, will be conducted in
four to six countries to collect more granular data on World Bank engagements in
different country and conflict contexts. These choices of field-based case studies will be
determined by the findings of the country data and content analysis, and an identified
need to surface more granular lessons through client, partner, World Bank, and other
relevant actor interviews. Country visit selection will be stratified across conflict
situations and aligned with learning needs in conversation with Country Management
Units. Security and current public health considerations will be considered in
determining whether the analysis is undertaken by the IEG team is desk-based or in
country.
4.4 The field-based case studies will be conducted using:
• A concise, structured literature analysis to permit the case analysis to be
grounded in the regional and country context.
• Semi structured interviews of actors selected through a stakeholder analysis
covering the evaluation period (for example, key World Bank staff, UN and
humanitarian agency partners) to corroborate and deepen the information
gathered through the content analysis for each of the 23 countries (evaluation
questions 1–4).
• Document gathering and analysis of associated content collected through the
field-based interviews. It is envisioned that documentation related to risk
analysis and decision-making will be gathered as part of the field-level
interviews (evaluation questions 1–4).
• Citizen engagement through structured short message service (SMS)
interviews: Case studies will include either digital SMS or locally implemented
citizen surveys to integrate the voice of beneficiaries. Because of security
constraints, stakeholder perceptions about intended or unintended outcomes
may be hard to glean from traditional qualitative interview methods. As such,
for countries included in the case base analysis, the assessment will use an SMS
survey tool targeting persons living in areas where the World Bank provided
services (issues of connectivity permitting). The methodology and questions for
10
these targeted surveys will be developed as part of the evaluation process, as
part of the case-based reviews (focus on evaluation question 4).
4.5 Appendix A describes the evaluation building blocks and illustrates how the
evaluation’s questions and methods are aligned.
5. Assessment Limitations
5.1 Information and data required to assess the World Bank’s engagements may not
be available or of good quality. The evaluation requires access to historical information
on country-level decision-making during the evaluation period. The evaluation will
need to identify and access key decision-makers who may not be accessible. Although
lending data are available, documentation of nonlending activities tends to be
incomplete, and nonlending activities are infrequently evaluated. Fragility and risk
analyses, as well as country- and sector-level institutional, political economy, and social
analyses, may have had limited or restricted distribution.
6. Quality Assurance Process
6.1 The approach paper and evaluation will undergo several quality assurance
processes, including internal IEG and World Bank management and staff review, as well
as external peer review. This approach paper has been peer-reviewed by the following
external experts:
• Yves Daccord is the Director-General of the International Committee of the Red
Cross. A former journalist, Mr. Daccord joined the International Committee of
the Red Cross in 1992, running humanitarian operations in various challenging
contexts of armed conflict including in West Bank and Gaza, Sudan, the Republic
of Yemen, Chechnya, and Georgia. As Director-General, Mr. Daccord has led
significant institutional reforms in areas such as human resources and people
management, partnerships and stakeholder management, and innovation and
technology.
• Joseph Saba is the Chairman of American Near East Refugee Aid, an adjunct
professor at Georgetown University’s School of Foreign Service, and a former
Director, World Bank Middle East Region (1997–2010). Mr. Saba has served on
several evaluation panels for bilateral assistance programs in the Middle East. He
lectures periodically at the North Atlantic Treaty Organization Defense College
in Rome and Brussels and in several other forums, where he has delivered
presentations focused on strategies for engagement in fragile and conflict-
affected states. Mr. Saba has a JD from Yale Law School, an MA in Middle East
affairs from Harvard University.
11
• Honourable Minister Patricia Laverley is the Deputy Minister of Finance of the
Republic of Sierra Leone following 20 years of experience at the International
Monetary Fund, World Bank, HSBC, and the African Development Bank. She has
delivered presentations on macroeconomic issues in leading global finance
conferences, including at IDA Replenishment Consultations and UN Conference
on Trade and Development’s Intergovernmental Forum on Mining, Minerals and
Metals. She teaches development economics and finance and has authored
publications on the effectiveness of World Bank programs and African
Development Bank’s public financial management reforms across Africa. Her
academic background includes bachelor’s, master’s, and doctorate degrees from
the American University in Washington, DC, and London, UK.
7. Staffing and Resources
7.1 The evaluation is being led by Lauren Kelly, Lead Evaluation Officer, under the
guidance of Jeff Chelsky, manager, and Oscar Calvo-Gonzalez, director, and under the
overall guidance of Alison Evans, Director-General, Evaluation. The evaluation is
supported by a core team consisting of Daniel Nogueira-Budny, Harsh Anuj, Anis Dani,
Adam Lichtenheld, Michelle Rebosio, Elizabeth Dodds, Mees Daniel van der Werf,
Christian Freymeyer, Joy Butscher, and Dung Thi Kim Chu. Stephan Wegner, IEG’s FCV
Coordinator, is providing advisory support.
7.2 The evaluation has an estimated $580,000 budget and will be submitted to the
Committee on Development Effectiveness by the end of quarter 2 in FY21. The budget,
which excludes dissemination, was determined by estimating the costs of (i) staff time in
line with the required skills mix and (ii) staff and consultant time, and variable costs
associated with the methods necessary to assess patterns of engagement at the portfolio
and country level.
8. Engagement Strategy and Learning
The evaluation engagement strategy has been designed to operate at multiple levels. The
evaluation engages with the FCV group and operational colleagues working in conflict
situations. A key touchpoint for the evaluation is participants of the Stability, Peace and
Security Global Solutions Group with staff participation across Global Practices and
regions of the World Bank. A stakeholder analysis has been conducted to identify
relevant members of country teams associated with the evaluation countries, with whom
this evaluation will collaborate. A key aim of the evaluation is to surface lessons to
support operational learning. This will be done by engaging operational teams
throughout the evaluation process, including by seeking feedback on preliminary
12
country findings. Other formal venues will be sought to engage relevant actors to
encourage uptake of this work.
1
The World Bank Group first started compiling a list of fragile and conflict-affected situations in
fiscal year (FY)06. The classification of this list has changed, from the Low Income Countries
Under Stress List in 2006–09, to the Fragile States List in 2010, to the Harmonized List of Fragile
Situations in 2011–20. In FY20, the classification and methodology were changed; this list is now
called the List of Fragile and Conflict-Affected Situations. For the sake of simplicity, this
approach paper will use the current name to refer to the current and all former lists. For more
information, including the new methodology, see the fragility, conflict, and violence strategy
(2020–25) or http://pubdocs.worldbank.org/en/373511582764863285/FCS-Historial-note.pdf.
2
For example, in April 2017, the United Nations and World Bank launched a joint platform,
outlined in the United Nations–World Bank Partnership Framework for Crisis-Affected Situations, to
enable better collaboration between the two organizations in line with their mandates.
13
Bibliography
Cuaresma, J. C., Wolfgang Fengler, Homi Kharas, Karim Bekhtiar, Michael Brottrager, and
Martin Hofer. 2018. “Will the Sustainable Development Goals Be Fulfilled? Assessing
Present and Future Global Poverty.” Palgrave Communications 4 (29).
Eck, Kristine, and Lisa Hultman. 2007. “One-Sided Violence Against Civilians in War: Insights
from New Fatality Data.” Journal of Peace Research 44 (2): 233–46.
Gleditsch, Nils Petter, Peter Wallensteen, Mikael Eriksson, Margareta Sollenberg, and Håvard
Strand. 2002. “Armed Conflict 1946–2001: A New Dataset.” Journal of Peace Research 39 (5):
615–37.
International Monetary Fund. 2018. The IMF and fragile states: 2018 evaluation report.
Luc Girardin, Philipp Hunziker, Lars-Erik Cederman, Nils-Christian Bormann, and Manuel Vogt.
2015. GROW
up
—Geographical Research on War, Unified Platform. ETH Zurich.
Pettersson, Therese, Stina Högbladh, and Magnus Öberg. 2019. “Organized Violence, 1989–2018
and Peace Agreements.” Journal of Peace Research 56 (4): 589–603.
Pettersson, Therese. 2019. UCDP/PRIO Armed Conflict Dataset Codebook version 19.1.
Raleigh, Clionadh, Andrew Linke, Håvard Hegre, and Joakim Karlsen. (2010). “Introducing
ACLED-Armed Conflict Location and Event Data.” Journal of Peace Research 47 (5): 651–
60.
Samman, Emma, Paula Lucci, Jessica Hagen-Zanker, Tanvi Bhatkal, Amanda Telias Simunovic,
Susan Nicolai, Elizabeth Stuart, and Charlotte Caron. 2018. SDG Progress: Fragility, Crisis
and Leaving No One Behind. London: Overseas Development Institute.
Sundberg, Ralph, Kristine Eck, and Joakim Kreutz. 2012. “Introducing the UCDP Non-State
Conflict Dataset.” Journal of Peace Research 49 (2): 351–62.
United Nations and World Bank. 2018. Pathways for Peace: Inclusive Approaches to Preventing
Violent Conflict. Washington, DC: World Bank.
https://openknowledge.worldbank.org/handle/10986/28337.
World Bank. 2011. World Development Report 2011: Conflict, Security, and Development. Washington,
DC: World Bank. https://openknowledge.worldbank.org/handle/10986/4389.
World Bank. 2014. World Bank Group Assistance to Low-Income Fragile and Conflict-Affected States:
An Independent Evaluation. Washington, DC: World Bank. doi: 10.1596/978-1-4648-0219-5.
World Bank. 2016. World Bank Group Engagement in Situations of Fragility, Conflict, and Violence.
Washington, DC: World Bank.
https://openknowledge.worldbank.org/handle/10986/24915.
14
World Bank. 2017. Forcibly Displaced: Toward a Development Approach Supporting Refugees, the
Internally Displaced, and Their Hosts. Washington, DC: World Bank.
https://openknowledge.worldbank.org/handle/10986/25016.
15
Appendix A. Evaluation Design: Building Blocks
Evaluation Level
Methods
Data Requirements
Evaluation
Question
Key evaluation question: How relevant and effective has World Bank engagement been in contributing to
development gains in situations of conflict?”
Macrodata
Desk-level data analysis;
collaboration with
relevant units in the
World Bank Group
Standard reports; operational portal;
World Bank open finance data; Human
Resources data; Systems Applications
and Products; credible external data
sources on conflict events (Uppsala
Conflict Data Program, Armed Conflict
Location and Event Data Project);
interviews
1
Country strategy and
portfolio
Risk and Resilience
Assessment analysis
Systematic Country
Diagnostic (SCD)
screening tool
Country Partnership
Framework (CPF)
screening tool
Coherence analysis
(sequencing and
prioritization)
Stakeholders mapping
and interviews
Field missions
Fragility Assessments, Risk and
Resilience Assessments; country social
assessments; governance and political
economy analysis; SCDs; CPFs;
operational portal
Nonlending and lending documents,
such as project appraisal documents,
Implementation Status and Results
Reports, Implementation Completion
and Results Reports, Implementation
Completion and Results Report Reviews
2, 3, 4,
Project
Country-level portfolio
analysis
Conflict sensitivity
analysis
Interviews
Beneficiary feedback tools
Project appraisal documents;
Implementation Supervision Reports;
back-to-office reports; aide-mémoire;
access to key stakeholders
Access to former and present World
Bank staff, clients, United Nations (UN)
and humanitarian partners,
implementing partners, civil society
organizations, and project beneficiaries
Comprehensive data (already collected)
on UN–World Bank engagements at the
project level obtained from the World
Bank UN Partnership Team
Collaboration with the Geospatial
Operational Support Team to identify
World Bank intervention areas; mobile-
based surveys of key project-affected
persons in identified project areas
affected by conflict
1, 2, 3, 4,
Cross-cutting
Interviews
Access to key stakeholders
1, 2, 3, 4
16
Appendix B. Methodology
Delimitation of the Evaluation Universe
The universe of cases is defined by countries that have (i) experienced medium or high
conflict since 2014 and that have been listed on the World Bank Lists of Fragile and
Conflict-Affected Situations (2010–20). Conflict intensity is identified based on a
threshold number of conflict-related deaths relative to the population, using the method
described in the World Bank Group Strategy for Fragility, Conflict and Violence (FY20–25).
The strategy also categorizes the intensity of conflict as high and medium intensity
(explained in the Definitions of Conflict section).
Definitions of Conflict
This evaluation uses data on conflict deaths from the Armed Conflict Location and
Event Data Project (ACLED) and the Uppsala Conflict Data Project (UCDP). Countries
in high-intensity conflict are defined as those with: (i) an absolute number of conflict
deaths above 250 according to ACLED and 150 according to UCDP; and (ii) a number of
conflict deaths relative to the population above 10 per 100,000 according to both ACLED
and UCDP, reflecting widespread and intense violence across many parts of the country.
Countries in medium intensity conflict are defined as (1) countries with lower intensity
conflict, as measured by (i) an absolute number of conflict deaths above 250 according to
ACLED and 150 according to UCDP; and (ii) between 2 and 10 per 100,000 population
according to ACLED and between 1 and 10 according to UCDP; or (2) countries with a
rapid deterioration of the security situation, as measured by (i) a lower number of
conflict deaths relative to the population between 1 and 2 (ACLED) and 0.5 and 1
(UCDP) and (ii) the number of casualties more than doubling in the last year.
Selection of Country Cases
Countries in the evaluation universe: The evaluation has used the preceding
methodology to identify countries that have experienced medium or high-intensity
conflict since 2014. When data from one data set was incomplete for a given country in a
given year, the evaluation relied on the other data set. By opting to circumscribe cases to
countries that have appeared at least once on the List of Fragile and Conflict-Affected
Situations, and that have also experienced medium or high-intensity conflict since 2014,
this evaluation’s universe is 23 countries (see figure 2.1 in text).
Countries outside of the evaluation universe: The country selection excludes 11 Part II
countries that have experienced medium or high-intensity conflict since FY10 but that
have never been on the List of Fragile and Conflict-Affected Situations between FY10–
20. Many countries that have experienced extreme violence, especially in the Latin
17
America and the Caribbean region, fall into this category: they have not been included
on the list so are excluded from the evaluation. The country selection also excludes a
second category of countries that have only experienced fragility (for example,
institutional, including most small island states) or that have not experienced medium or
high-intensity conflict since 2014.
Figure B.1. Country Stratification for the Purpose of Field-Level Analyses
Source: Independent Evaluation Group.
Country Stratification for the Purpose of Field-Level Analyses
To better understand and assess World Bank engagements across different types of
conflict situations, this assessment has also gathered and used data from UCDP on
conflict duration and the University of Maryland’s Center for Systemic Peace on
complexity, using geographic scope over time as a proxy. This has allowed the evaluation
to identify and isolate those situations that have been most intense, lengthy, and
complex, since this impacts the way the World Bank routinely needs to operate. It has
also allowed the evaluation to track the movement of countries along these axes, which
shows that many conflicts in this evaluation are now of a long duration and complex.
Because this represents most cases, case base selection will mostly be derived from the
top two quadrants as shown in figure B.1.
Dimension of Conflict: Duration and Complexity
Conflict duration for a country is determined by counting the number of calendar years
in which it experienced either medium- or high-intensity conflict. As all countries
18
experienced conflict in at least one year, this yields a score between 1 and 10, which
determines the country’s place along the y-axis. The countries with above-average
duration are placed in the upper two squares and the others below.
The evaluation uses the magnitude scale for conflict from the Political Instability Task
Force State Failure Problem Set 1955–2018 published by the Integrated Network for
Societal Conflict Research. This network was established to coordinate and integrate
information resources produced and used by the Center for Systemic Peace. The
magnitude scale is used as a proxy for conflict complexity.
The magnitude scale reflects the portion of a country affected by fighting. The code is
based on source materials about how much of the country is directly or indirectly
affected by fighting or political protest each year. A province, region, or city is “directly
affected” if fighting, terrorist attacks, or political protests occur there at any time during
the year. It is “indirectly affected” if the area has significant spillover effects from nearby
fighting, for example, refugees’ flows, curtailment of public services, or martial law
imposed. If open conflict expands or contracts during the year, it is coded according to
its greatest extent.
• 0 = less than one-tenth of the country and no significant cities are directly or
indirectly affected
• 1 = one-tenth of the country (one province or state) or one or several provincial
cities are directly or indirectly affected
• 2 = more than one-tenth and up to one-quarter of the country (several provinces
or states) or the capital city are directly or indirectly affected
• 3 = from one-quarter to one-half the country or most major urban areas are
directly or indirectly affected
• 4 = more than one-half the country is directly or indirectly affected
The countries that would be placed in the lower half based on their below-average
duration score will be placed in the upper half if they score on average a one or higher
on the magnitude scale over the evaluation period. When data are lacking, it is
supplemented with expert insight based on geocoded conflict data from UCDP using the
same coding criteria.
19
Evaluation Tools
Risk and Resilience Assessment Tool
1. Does the RRA [risk and resilience assessment] identify
• Internal and external drivers of conflict?
Internal Drivers
1.
2.
External Drivers
1.
2.
• Political, social, and economic drivers of conflict?
Political Drivers
1.
2.
Social Drivers
1.
2.
Economic Drivers
1.
2.
• Drivers of conflict at a community level, drivers specific to different parts of the
country, and national-level issues?
Community-Level Drivers
1.
2.
Regional-Level Drivers
1.
2.
National-Level Drivers
1.
2.
20
2. Is there clear evidence presented for why specific drivers of conflict and
resilience were selected? What data are used to buttress these drivers? How
credible is the evidence (and data)?
3. Does the RRA explain how these drivers of conflict and resilience are directly
related to development issues (including governance, poverty, inequality)? How
exactly?
4. Does the RRA present clear guidance on how the analysis should impact the
World Bank’s overall country strategy, prioritization of activities, or apply a
conflict lens to its activities?
5. Are the recommendations written in such a way to make it evident to a non-
conflict specialist whether these were taken on board in strategic documents?
Please also list the recommendations.
21
SCD Screening Tool
Name country:
Date published:
1. Conflict tally tool: Please first tally all mentions of conflict. Exclude headings and
titles, use in graphs and mentions such as “conflict of interest” or “conflicting goals.”
2. Drivers
a. Which drivers of conflict are mentioned in the SCD? Please use the table below.
Feel free to add rows to these and other tables if necessary.
Name
Description
1
2
3
4
5
b. What is your analysis of the drivers mentioned? (For example, drivers are
correctly identified, relative importance of various drivers, are some more
contextual or localized than others, too generic, omissions from the RRA [if one
was available during the time of writing].)
3. Conflict and development
a. What are aspects of conflict that, according to the SCD, affect development?
Please use the table below.
Aspect
Impact on Development
1
2
3
4
5
b. Does the SCD discuss the need to address conflict to affect developmental
challenges? (Does the SCD just describe conflict, or does it indicate how the
diagnostic of development challenges should be different in conflict-affected
areas?)
22
4. Body and appendix
a. Please compare qualitatively the analysis done on conflict drivers in the main
text to that in the appendix if it exists.
b. If an appendix on drivers of conflict exists:
i. Please use the scale below to indicate to what degree conflict
permeates the SCD’s main text.
ii. Then please describe qualitatively when and how conflict permeates
the main text.
1
2
3
4
5
Never
Rarely
Sometimes
Often
Always
4. Analysis and data
a. Does the SCD draw on or leverage any conflict analysis?
In your analysis, please distinguish between World Bank internal products (for
example, RRA, political economy analysis, conflict assessments) and what is
available from other entities.
b. Which data on drivers of conflict does the SCD use?
c. Is it sourced?
Data/Analysis
Source
Internal/External
1
2
3
4
5
d. Does the SCD cite or use data from the RRA?
e. What is your analysis of the data used? (For example, is it sufficient, what is its
quality and reliability, what other data could have been used to strengthen the
SCD, is it relevant to the drivers?)
5. Risk
a. Does the SCD identify conflict risks to the project portfolio? Note: please only
discuss conflict-related risks. For instance, fluctuating commodity prices pose
23
risks in many countries, with and without conflict. Risks should only be included
when they relate to conflict.
b. What kind of tools are used for risk identification and assessment? (For example,
RRAs, World Bank analysis, expert consultations.)
c. What does it say about identifying, managing, and mitigating conflict risk at the
following levels?
Level
Identifying
Managing/Mitigating
1
Policy level
2
Strategy level
3
Portfolio level
d. What does it say about the following risk categories, and which tools are used for
identification, management, and mitigation? Feel free to adapt the list to reflect
the risks discussed.
Risk
Identifying
Managing
Mitigating
1
Performance
2
Corruption/state
capture
3
Behavioral
4
Macroeconomic
shocks
5
Societal cleavages
(that is, ethnic,
religious, linguistic)
6
Security or access
7
Lack of data or
information
8
Service delivery
e. Do the mentioned risks connect to the identified conflict drivers?
Driver
Risk
1
2
3
4
5
f. What is your analysis of the risk identification, management, and mitigation as it
relates to conflict? (For example, is it sufficient; are identification, management,
24
and mitigation adequately separated, sensitive to the difference between conflict
and nonconflict areas; does the SCD elsewhere reflect the risks identified here?)
6. Does the SCD identify priority areas?
a. If so, what is the argumentation for the described priority areas?
b. How do they relate to the conflict risks and conflict drivers?
c. What is your analysis of the priority areas? (Is it too generic, realistic, or too all-
inclusive?)
Priority Areas
Argumentation
Relation to Conflict
1
2
3
4
5
7. Does the SCD discuss that (because of identified risks) the portfolio should be
sequenced in a certain way to mitigate conflict risk?
a. If yes, how? For example, is it at the sector level (agriculture before mining) or
the instrument level (investment project financing versus development policy
operation)?
b. What is your analysis of the sequencing?
8. Partnerships
a. Which (potential) partners are mentioned (including state actors, nonstate actors,
and development or humanitarian partners)? For each partner include
i. Partner
ii. Typology (for example, state, nonstate, international organizations,
civil society, nongovernmental organization [NGO])
iii. Quote of reference
iv. Their role (for example, implementation, convening, dialogue,
leverage, peace, and reconciliation)
25
Partner
Typology
Their Role
Quote of Reference
1
2
3
4
5
b. What is your analysis of the partnerships discussed? (For example, are any
potential partners not included, is it too generic?)
9. Transnationality
a. Is the conflict understood as a regional issue?
b. What mention is made of regional partnerships/multicountry linkages between
World Bank projects?
c. What is your analysis of the SCD’s understanding of the possible regional nature
of the conflict?
10. Results and outcomes
a. Does the SCD discuss anticipated results (that is, tangible results) in a way that is
sensitive to the conflict context? How?
b. Does the SCD discuss desired outcomes as separate from results or indicators?
c. Does the SCD discuss the need to capture data in real time (to ensure conflict
sensitivity)?
d. What is your analysis of the outcomes mentioned? (For example, does the SCD in
the context of conflict diagnose how to reduce poverty, are results seen as
homogenous across conflict and nonconflict areas?)
11. Differentiation
a. Does the SCD differentiate between conflict- and non-conflict-affected areas of
the country?
b. What is your analysis of the SCD’s success in producing different diagnostics of
what to do in conflict and nonconflict areas?
26
CPF Screening Questionnaire
Country: Date published:
Date of RRA publication:
1. Where does conflict “live” within the document?
a. Does the CPF include a separate section on a conflict objective or pillar? If so,
please describe how.
b. Does the CPF include a conflict lens/approach across its objectives or pillars? If
so, please describe how.
c. Does it neglect certain conflict in the remainder of the CPF (that is, elsewhere is it
not conflict-sensitive)? From your evaluator opinion, which gaps exist in the
mainstreaming?
2. Informing the CPF
a. Does the CPF’s strategic objectives develop or address the drivers of conflict as
described in the SCD (or RRA)? Please make sure to include the conflict drivers
that were not addressed (and your thoughts on why they were excluded). Please
answer in the table below, and be sure to include your comments in the last
column.
CPF Strategic Objective
SCD/RRA Conflict Drivers
Addressed
Comments
SCD Conflict Drivers Not
Addressed
b. Overall, does the CPF pick up on lessons from working in conflict from prior
conflict strategies or relevant analyses or reviews (in the specific country or
others), and so on? Please specify the conflict lens of these lessons.
i. If so, how does it build on these?
ii. How were the lessons integrated in programming?
27
Specific Conflict-Related Lesson Learned
How Lesson Was Addressed in
Programming
3. Risk analysis
a. Is conflict-related risk a factor in the programming as set out in the CPF? Please
discuss risks that are a consequence of conflict or exacerbated by conflict rather
than general risks also present in nonconflict peer countries. Please use the table
below, which is characterized into
• Political risk (for example, contested elections, security, renewed fighting)
• Societal risk (for example, ethnic, religious, or identity cleavages or tensions)
• Corruption or state capture risk
• Macroeconomic risk or shocks
• Other
Type
Specific
Risk(s)
Identification
Management
Mitigation as Part
of its Programming
Political
Societal
Corruption or state
capture
Macroeconomic risk
or shock
Other
b. In your opinion, are certain risks missing?
c. How (if at all) have the above risks been used as a management tool for the
corporate programming across the CPF pillars/objectives? For example, is risk
language incorporated into framing pillars/objectives? (for example, if exclusion
is a risk how is inclusion integrated into programming).
i. Specifically discuss programming
ii. Does the CPF use risk analysis to apply adaptive management techniques?
28
4. Sequencing and prioritization
a. Does the CPF discuss prioritization or sequencing through a conflict lens
(yes/no)? Please describe briefly. If so, how does this align with the
discussion of prioritization or sequencing in the SCD?
b. Does the CPF treat conflict-affected areas differently from nonconflict areas
in terms of programming? If so, how?
5. Partnerships
Does the CPF mention partnerships with regard to conflict-related programming
and programming in conflict-affected areas? Which (potential) partners are
mentioned (including state actors, nonstate actors, and development/humanitarian
partners)? For each partner include
• Partner
• Typology (for example, state, nonstate, IO, civil society, NGO)
• Their role (for example, implementation, convening, dialogue, leverage, peace
and reconciliation)
• Quote of reference
Partner
Typology
Their Role
Paste Reference to Partnership
1
2
3
4
29
WBESOC Portfolio-Level Coherence and Sequencing Tool
The purpose of this tool is to understand (i) the nature of the country nonlending
portfolio—its topical mix, its sequencing and utility for making key conflict-related
lending decisions—and (ii) the nature of the lending portfolio—sequencing,
prioritization of sector, and project choices—in relation to the conflict-related advice and
guidance provided by RRAs/SCDs on operational choices and strategies. It also probes
the issue of instrument choice.
1. Name of Evaluator:
2. Name of Country:
3. Key Conflict Events and Dates, and World Bank Operations
Describe key conflict events and dates (for example, start of conflict, end of conflict,
escalation, de-escalation) along with information on World Bank operations (for
example, when the World Bank opened offices in the country, commenced or
suspended operations)
Date
Event
4. Key pieces of analytical work
Describe the key pieces of analytical work that were conducted that could provide
insights to the task teams preparing operations in conflict-affected areas. This section
should not be limited to conflict analysis but should include, at the author’s
discretion, more traditional types of World Bank nonlending that could be used at
least partially to increase conflict sensitivity. Examples include the following:
• Political economy analysis
• Public Expenditure Reviews
• Governance and Corruption
• RRA, FA, Peace and Conflict Filter, Country Social Assessment
30
• Sector-specific assessments that use a conflict lens or are conflict-sensitive, and
focus on conflict-related sectors as specified in the SCD
Along with the date and description of the product, please specify the category of
the nonlending activity among the following options: (i) service delivery in conflict-
affected areas; (ii) conflict analysis; (iii) macrofiscal products (for example, public
financial management, fiduciary assessment, payment system, or other operational
analysis); and (iv) governance.
Date
Product
Category
5. Key lending operations in conflict areas
Together with the Macrodata Team, provide a list and description (name, sector,
dates, financing) of the key lending operations, including canceled operations that
have geographic overlap with conflict-affected areas. Include national, regional, and
more siloed projects.
Dates
Name
Sector
Financing
Coherence analysis (Use charts provided by the macro data team plus your own analysis
of the above portfolios describe trends).
a. Please use the above data on non-lending and lending and the portfolio
charts to describe the major issues affecting the coherence of the portfolio.
• Were nonlending projects used to inform lending? Was the nonlending
inclusive of issues that would help the lending be conflict sensitive, even
if it is not labeled as such (i.e., governance, macrofiscal risks,
institutional/corruption issues).
b. Across which sectors over time do you find coherence in the nonlending and
lending portfolios and which ones do you find more idiosyncratic?
c. How well were the key lending operations – sector choice, timing, and
instrument choice – in line with the risk/conflict analysis guidance provided
by the RRA, FA, SCD or other specific conflict assessments?
31
d. Please include any additional comments on the thematic sequencing, volume,
instrument/volume/sector trends from the graphs provided by the macro
team if they provide additional ideas to the above.
e. Were there any shifts in the focus or timing of nonlending operations (e.g.,
governance, institutional analysis, conflict sensitivity, political analysis) in
response to changes in the conflict context (start or end of conflict, escalation
or de-escalation, etc.)? Please describe them. In other words, as nonlending
being conducted “in advance of, in real time” to help task teams assess the
changing nature of conflict risks and dynamics, including the changing
nature or political or institutional issues that have conflict implications?
f. Describe whether projects were restructured (level I, II). This will inform the
choice of projects for the CSA.
g. Did projects scale back, close, or leave areas affected by conflict? Backfill your
response to this question once you have completed the project-level CSA
analysis.
6. Geography of aid
a. Please describe the outlay of your portfolio over time from the point of view
of conflict geography.
b. Which projects were already operating in areas where conflict(s) broke out?
Which sectors?
c. Which projects were designed to operate in conflict areas and were made
effective after conflict broke out?
d. Which projects are national, explain links to conflict issues. This would
include macro/fiscal/governance. But for the real sectors, we are also wanting
to know that there are investments that were planned nationally (let’s take
transport) in areas where conflict existed or broke out as well as peaceful
areas, so that we can probe the conflict sensitivity in the CSA tool.
7. Conclusion
a. How do lending and nonlending operations line up in line with conflict
events and the need for preparedness and informed risk taking?
b. Was there adequate differentiation between conflict and non-conflict-affected
analysis and operations?
32
Analysis of Conflict Sensitivity in Project Documentation*
*This CSA [conflict sensitivity approach] tool is intended to evaluate the conflict
sensitivity of individual World Bank projects. It draws on DFID’s [Department for
International Development’s] Strategic Conflict Assessment tool, GIZ’s Peace and
Conflict Assessment, and USAID’s [U.S. Agency for International Development’s]
Conflict Assessment Framework. It is administered mainly on desk but also requires key
interviews to “fill in the gaps” about the design and effectiveness of conflict
identification and mitigation measures.
Instructions
Only projects that have worked partly or fully in conflict-affected or fragile areas should
be selected for analysis using this tool. Each project should be reviewed using the
following:
• Project appraisal document (or equivalent)
• Latest ISR [Implementation Status and Results Report] and aide-mémoire (if
project is closed, try to find the last aide-mémoire)
• For closed projects, ICR [Implementation Completion and Results Report]
• Interview with TTL [task team leader], social specialist, or operations officer (as
needed)
• For some projects, there could be relevant studies or social safeguards documents
(the PAD [project appraisal document] or ICR might refer to them)
Project Information
Project Name:
Date of Approval:
Date of Effectiveness:
Closing:
Amount:
Implementing Agencies:
PDO:
33
Global Practice:
Main activities:
Is the project national? Subnational?
Nature of the Project and Conflict Context
1. Did the project try to address a conflict-related issue or was it focused on
achieving traditional development objectives? Did it work “in” conflict (e.g.,
programming that seeks to minimize conflict-related risks/“do no harm”); “on”
conflict (programming that focuses on conflict
prevention/management/resolution); or “around” conflict (treating conflict as an
impediment to be avoided)? If the project is working “in” conflict, what conflict
driver(s) is it seeking to address?
2. What was the stage of conflict when this project was being implemented?
(escalating, de-escalating, latent conflict, or postconflict peacebuilding)
3. What was the nature of violence in the project area?
Project Design
1. Please describe whether and how the project identified conflict-related risks.
a. Was a conflict specialist involved in the project design?
b. What sources of analysis were leveraged (e.g., RRA, Political Economy
Analysis, Conflict assessments, peace and conflict filter, experts)? In your
analysis please distinguish between internal WB [World Bank] products and
what’s available from other entities. Dimensions of conflict analysis could
include the following:
i. An analysis of governance and institutions and how these are related
to conflict.
ii. An analysis of horizontal and vertical inequalities between different
groups.
iii. An analysis of what drives inclusion, cohesion, or social
accountability/participation.
iv. Economic factors that fund or contribute to conflict.
c. What was the quality of the analysis of the conflict risks to and stemming
from the project in the project documentation?
34
2. Please describe how conflict analysis was integrated into the design of the project.
a. Did it inform the sector and/or geographic areas targeted for programming
(e.g., communities most or least affected by conflict)? The lending
instruments used? The main beneficiaries (e.g., inclusive or exclusive of
particular groups? How were these decisions justified?
b. Describe any conflict-related mechanisms that were integrated into the
project design (e.g., participatory grievance processes).
Project Implementation
3. Did the project implement activities in a conflict-sensitive way? Consider the
following conflict-related risks, and how well the project identified and mitigated
them:
a. Contextual risks
i. Did the project consider what changes might arise in the general
environment (e.g., insufficient capacity of partner) as a result of
conflict, and how those changes could impair or prevent
implementation of the project and the attainment of its objectives?
b. Programmatic risks
i. Did the project take adequate measures to prevent negative
externalities of program activities as they relate to conflict (e.g.,
increased GBV [gender-based violence], labor influx upsetting fragile
communal balance, new rents or unequal gains destabilizing
community, restricting access of displaced people to communal
resources)?
ii. Were project activities inclusive of different groups? (Consider ethnic,
linguistic, religious minorities, as well as ways that the project tries to
include youth, women, disabled, or ex-combatants, IDPs, refugees).
iii. How has the project disseminated information about projects to
different social groups (including women, youth, displaced persons,
etc.)? Are there groups that normally do not have access to this
information?
iv. Was the project able to identify key individuals, governing structures,
or partners that could drive change and involve them in the project?
35
v. What did the project do to manage actors can be identified as
spoilers? For example, groups benefiting from war economy
(combatants, arms/drug dealers), smugglers, etc.
c. Corruption risks
vi. If corruption/elite capture was a concern, how did the project address
this? Did the project go beyond implementing World Bank fiduciary
procedures?
d. Personnel risks
vii. What were the mechanisms to hire project staff, including staff
responsible for project implementation? Were these mechanisms
conflict sensitive?
viii. How might conflict jeopardize the security of project and/or partner
personnel (e.g., murder, robbery, kidnapping and medical care)?
b. Risk Mitigation and Management
i. How well were risks monitored? Did the project measure and track its
impact on conflict (e.g., inequality or the poverty of specific excluded
groups)?
ii. What contingency measures were prepared to facilitate an
appropriate response in the event that risks manifested?
iii. How well did the project respond to the effects of its projects
(programmatic and operational), and either adjusted its activities or
developed new initiatives in response? Please only discuss effects that
pertain to conflict.
