The Politics of Evaluation in International Organizations
This project challenges the conventional understanding of evaluation as a value-free activity and demonstrates how a seemingly neutral technocratic tool can serve as an instrument for power in international governance.
Project Team
Jun.-Prof. Dr. Steffen Eckhard
Principal Investigator
Post-Doctoral Research Fellow
Research Associate
Ian Burton, Alina Becker, Phillip Rothe
Research Assistants
Project Timeline
First results published in the journal Evaluation
This article develops the concept of evaluation stakeholder influence potential, which builds on four political resources for influence (agenda-setting powers, staff and budgetary resources, access to evaluation results, and access to evaluators). These resources are measured for both member states and international public administrations in 24 United Nations organizations.
Eckhard, S., Jankauskas, V. (2019): The Politics of Evaluation in International Organizations: A comparative Study of Stakeholder Influence Potential. Evaluation. 25(1), https://doi.org/10.1177%2F1356389018803967.
To the article
Study of the European Commission published in Politische Vierteljahresschrift
Drawing on interviews with senior Commission officials, we demonstrate that the Better Regulation Reform enhances the Commission’s internal coherence and its ability to justify actions vis-à-vis member states. We reveal how an international public administration can conform to member statesʼ demands for more accountability and transparency yet design the overall evaluation system in a way that contributes to its strategic actorness.
Jankauskas, V., Eckhard, S. (2019): International Bureaucracies as Strategic Actors: How the Better Regulation Reform Strengthens the European Commission. Politische Vierteljahresschrift. 60, 681–699, https://doi.org/10.1007/s11615-019-00189-3.
To the article
First comparative study on political evaluation use in IOs published in Policy Sciences
We argue that formally independent IO evaluation units informally orientate towards either member states or the IO administration, depending on who controls the unit’s budget, staff, and agenda resources. This should enable either actor to also use evaluation results along pre-defined strategic interests. Interview data gathered among evaluators, secretariat officials, and member state representatives of six IOs support the expected pattern, highlighting striking differences in the orientation of evaluation staff and evaluation use. Findings challenge the technocratic, apolitical image of evaluation, offering practical and theoretical implications for future research.
Eckhard, S., Jankauskas, V. (2020): Explaining the political use of evaluation in international organizations. Policy Sciences 53, 667–695, https://doi.org/10.1007/s11077-020-09402-2.
To the article
German Research Foundation (DFG) announced funding for the second phase
In the second phase of the project (2021-2023), the team starts analysing potential political biases in IO evaluation reports, employing both qualitative and quantitative research techniques.
Book contract signed with Oxford University Press
The upcoming book "The Politics of Evaluation in International Organizations" is first to offer a rigorous test of the hypotheses linked to the politics of evaluation in international organizations. It offers nuanced and empirically grounded insights into the political interests of evaluation stakeholders, the control and design of IO evaluation systems, and the content and use of evaluation results.
It should appear in the first half of 2022, printed by Oxford University Press.
Eckhard, S., Jankauskas, V. (2022): The Politics of Evaluation in International Organizations. Oxford University Press. Forthcoming.
Best Comparative EGPA Paper Award
'Evaluations of International Organizations: A 'Gold Standard' of Institutional Assessment?' won the Best Comparative EGPA Paper 2021 award by the Journal of Comparative Empirical Analysis.
The paper is our first attempt to analyse hundreds of IO evaluation reports using machine learning based language models.
Book contract signed with Oxford University Press
The upcoming book "The Politics of Evaluation in International Organizations" will provide for a first comprehensive overview of IO evaluation function, offering nuanced and empirically grounded insights into the political interests of evaluation stakeholders, the control and design of IO evaluation systems, and the content and use of evaluation reports.
The book should appear in the first half of 2022, printed by Oxford University Press.
German Research Foundation (DFG) announced funding for the second project phase
Analyzing a large database with over 2,000 evaluation reports across different IOs, we will for the first time provide comparative empirical insights into the content of IO evaluation reports, especially focusing on the existence of potential political biases.
About the project:
Evaluation has experienced spectacular proliferation at the international level. Today, the vast majority of international organizations (IOs) have institutionalized evaluation as an integral part of their organizational practices. More than 160 full time evaluators with several thousand expert consultants are now working in the UN system as compared to only 60 officers in the 1980s. Hundreds of reports are produced annually by major IOs and the number is growing. In the UN system, around 700 reports are produced each year which should lead to annual IO-evaluation expenses of up to half a billion Euros (about the entire annual budget of UNESCO or ILO).
Essentially, evaluation has become the ‘mantra of modernity’. Its main strength, as seen by the UN, is the ability “to provide evidence that is robust, valid, reliable and credible and can be used with confidence in decision-making” (JIU 2014a, p. 2). This is how policy makers typically conceive of evaluation – as a functional tool in the final phase of a cyclical policy process.
Yet, does evaluation really deliver on its promise of scientific evidence and policy adjustment? Contributing to such broader questions about the (in-)effectiveness of evidence-based policymaking, the project challenges the conventional understanding of evaluation as a value-free activity and demonstrates how a seemingly neutral technocratic tool can serve as an instrument for power in international governance.
While the first phase of the project (2017-2020) analysed the political use of evaluation in IOs, the second phase (2021-2023) delves into evaluation reports themselves, scrutinizing their content and potential political biases.
Following a mixed-method research design, we conducted field research with over 70 interviews at 19 international organizations, and even participated in training exercises for professional evaluators, to gain qualitative insights into evaluation processes. At the same time, creation of an original databank with 2,000 evaluation reports for the first time allowed to quantitatively scrutinize the actual content of IO evaluation reports, aiming at the identification of systematic political biases.
The project is associated to the DFG Research Unit "International Public Administration" (www.ipa-research.com).
Research grant: Phase I: 223.644 Euro, Phase II: 237.292 Euro, funded by DFG (2017-2023).
Project Publications
Eckhard, S., Jankauskas, V. (2022): The Politics of Evaluation in International Organizations. Oxford University Press. Forthcoming.
Eckhard, S., Jankauskas, V. (2020): Explaining the political use of evaluation in international organizations. Policy Sciences 53, 667–695, https://doi.org/10.1007/s11077-020-09402-2.
Jankauskas, V., Eckhard, S. (2019): International Bureaucracies as Strategic Actors: How the Better Regulation Reform Strengthens the European Commission. Politische Vierteljahresschrift. 60, 681–699, https://doi.org/10.1007/s11615-019-00189-3.
Eckhard, S., Jankauskas, V. (2019): The Politics of Evaluation in International Organizations: A comparative Study of Stakeholder Influence Potential. Evaluation. 25(1), https://doi.org/10.1177%2F1356389018803967.