Metrics for Evaluating Policy Commitments in a Fragmented World: The Challenges of Equity and Integrity

Summary

Despite the uncertainties about the nature and stringency of national emission reduction commitments, some things are clear: the international negotiations not only will include national targets and timetables, but also will have to take account of diverse policies and measures undertaken by individual nations, including those inside the current Kyoto group, as well as among developing countries. The evaluation of these diverse policies poses a number of challenges. For example, how can one assess the fairness of the relative contributions of different nations? And even if fairness is agreed upon, how is it possible to determine the credibility of the commitments?

Ex post, determining whether particular policies have been implemented is a relatively simple matter, although assessing their effectiveness is not always straightforward. Ex-ante, however, the integrity of the international process requires at least some evaluation of the policies and measures proposed by individual nations to estimate their likely impacts. The absence of such evaluation may handicap the negotiators in reaching credible agreements.

The current system for reporting national actions to the international community is highly nonuniform and insufficient to understand differences among countries’ policies and their effectiveness. Thus, a first order of business should be the development of a much tighter, narrowly defined set of guidelines designed to reflect genuine differences in activities among nations.

Regarding the fairness of the commitments, certain metrics, like emissions as a share of GDP, population, or historical emissions, are straightforward to calculate, and generally informative, albeit imperfect indicators of burden. Other metrics, like emissions reductions or total costs of policies undertaken, are unlikely to be reported reliably. The metric of marginal abatement costs—at least among market-based policies—has the advantage of indicating the cost-effectiveness of the international distribution of effort. It is also an important indicator of the competitiveness impacts of climate policies vis-à-vis trading partners. We recommend greater focus on this measure, but caution the difficulty in attributing the marginal costs of non-market-based policies, especially inefficient measures. The key question is what carbon price would achieve the same reductions as the suite of policies selected, either by sector or for the whole economy. This would be analogous to the calculation of the level of effective protection applied in analyses of trade disputes.

Regarding the integrity of the commitments, we see related but distinct issues associated with the ex-post verification of performance—essentially compliance—and the ex-ante challenges faced by international negotiators in comparing often quite dissimilar policies and measures. For ex-post verification, the simplicity of an aggregate, economy-wide emissions target, or even one expressed as emissions intensity, is quite appealing. Existing data and reporting systems are certainly compatible with such approaches. When subnational or specific regulatory or voluntary policies are used, the commitment should be expressed as a transparent, verifiable goal, such as a fuel efficiency standard or level of technology deployment. However, while such goals may be clear, their effects on emissions are less transparent. Therefore, descriptive, institutionally-oriented information must be supplemented with micro data on the actual implementation and performance of these measures. Focusing on specific emission goals as opposed to regulatory standards can help avoid excessive reliance on model-based counterfactuals. R&D programs are by their long-term nature difficult to compare to near-term emissions targets, but these activities should at least be made more comparable across countries. We see no alternative to relying on actual expenditure and deployment data, although care should be taken to link such data to specific program activities, and to include transparent baseline information.

Assessment of the integrity of ex-ante commitments is, perhaps, the most important but also the most problematic area. The greatest challenges are associated with the unavoidable need to model counterfactuals, with all the attendant complexities. New guidelines should focus on the need for greater transparency in models and data, and greater standardization in methodologies in order to improve the consistency of analysis across sectors, policies and countries. Another priority is the strengthening of the mandate of the international group of experts that evaluates the submissions. The current practice of UNFCCC peer review is far too loose an arrangement for the reports to be credible inputs to climate negotiations. Other international processes may provide lessons for evaluating the quality, consistency and value of the estimates of ex-ante commitments. For example, the World Trade Organization has a Trade Policy Review Mechanism, by agreement in the Uruguay Round, which offers regular, comprehensive reviews of individual Members’ trade policies and practices and their impacts on the functioning of the multilateral trading system.

While the multilateral trading system offers some lessons in negotiating and supporting international agreements, the circumstances and party incentives are quite different for a climate framework. A greater role may need to be played by independent institutions, international organizations, academic researchers, and other third-party groups in supporting the evaluation efforts that, in turn, support the negotiations.