Using NLP to build causal maps from emissions trading policy analysis literature: A more comprehensive means of analysis

Abstract

Despite their best intentions, policy interventions often fail to adequately address the challenges they were designed to tackle. Disparities in scope, taxonomy and performance perspectives employed by different policy studies, make it difficult to obtain a system perspective of the policy effects of a certain domain. Additionally, a prohibitively large body of literature often makes manual methods of review infeasible. To combat these shortcomings of existing policy analysis methods and provide an insightful way of informing policy evaluation and development, this project proposes a novel five-step policy analysis method that can semi-automatically derive and aggregate causal relations from policy literature into a causal map of policy effects. The method has been applied to a collection of 28 emissions trading scheme (ETS) policy analysis literature sources, producing a causal map consisting of 159 causal links with a recall of 38% and precision of 84%. The results demonstrate that applying this approach produces an aggregated system perspective of the analysed policy’s effects, and garners relevant insights for policy evaluation and development that would otherwise be difficult to obtain. This approach can therefore provide analysts from across different policy domains with a more comprehensive understanding of the factors and relations affecting policy and so provide a more comprehensive evidence base from which to inform policy development.