NUSAP net

An overview of recent papers in the field of Post Normal Science

Knol et al., 2009 Dealing with uncertainties: The case of environmental burden of disease assessment Environmental Health 8:21
Turnpenny et al. 2009, Noisy and definitely not normal: responding to wicked issues in the environment, energy and health Environmental Science and Policy 12 (3), pp. 347-358.
Boone et al., 2009, NUSAP method for evaluating the data quality in a quantitative microbial risk assessment model for salmonella in the pork production chain Risk Analysis 29 (4), pp. 502-517
Van der Sluijs et al., 2008, Exploring the quality of evidence for complex and contested policy decisions, Environ. Res. Lett. 3 (2008) 024008 (9pp)
Wardekker et al., 2008 Uncertainty Communication in Environmental Assessments: Views from the Dutch Science-Policy interface Environmental Science & Policy, 11, 627-641
Farrell, 2008, The Politics of Science and Sustainable Development: Marcuse’s New Science in the 21st Century Capitalism Nature Socialism, 19 (4), 68-83.
Maxim and Van der Sluijs, 2007, Uncertainty: cause or effect of stakeholders' debates? Analysis of a case study: the risk for honey bees of the insecticide Gaucho® Science of the Total Environment, 376, 1-17.
Refsgaard et al., 2007, Uncertainty in the Environmental Modelling Process: A Review Environmental Modelling & Software. 22 (11), 1543-1556.
Kloprogge and Van der Sluijs, 2006, The inclusion of stakeholder knowledge and perspectives in integrated assessment of climate change Climatic Change, 75 (3) 359-389
Risbey et al., 2005, Application of a Checklist for Quality Assistance in Environmental Modelling to an Energy Model. Environmental Modeling & Assessment 10 (1), 63-79.
Van der Sluijs et al., 2005 Combining Quantitative and Qualitative Measures of Uncertainty in Model based Environmental Assessment: the NUSAP System, Risk Analysis, 25 (2). p. 481-492
Van der Sluijs 2005, Uncertainty as a monster in the science policy interface: four coping strategies, Water science and technology, 52 (6) 87–92
Janssen et al, 2005, A guidance for assessing and communicating uncertainties, Water science and technology, 52 (6) 125–131
Craye et al, 2005 A reflexive approach to dealing with uncertainties in environmental health risk science and policy International Journal for Risk Assessment and Management, 5 (2), p. 216-236

Knol et al., 2009 Dealing with uncertainties: The case of environmental burden of disease assessment Environmental Health 8:21
Abstract Disability Adjusted Life Years (DALYs) combine the number of people affected by disease or mortality in a population and the duration and severity of their condition into one number. The environmental burden of disease is the number of DALYs that can be attributed to environmental factors. Environmental burden of disease estimates enable policy makers to evaluate, compare and prioritize dissimilar environmental health problems or interventions. These estimates often have various uncertainties and assumptions which are not always made explicit. Besides statistical uncertainty in input data and parameters – which is commonly addressed – a variety of other types of uncertainties may substantially influence the results of the assessment. We have reviewed how different types of uncertainties affect environmental burden of disease assessments, and we give suggestions as to how researchers could address these uncertainties. We propose the use of an uncertainty typology to identify and characterize uncertainties. Finally, we argue that uncertainties need to be identified, assessed, reported and interpreted in order for assessment results to adequately support decision making.

Turnpenny et al. 2009, Noisy and definitely not normal: responding to wicked issues in the environment, energy and health Environmental Science and Policy 12 (3), pp. 347-358.
Abstract Attempts by researchers and policy-makers to address the ‘wicked’ issues which pervade environmental policy usually revolve around attempting – or recommending – both more participatory and transparent, and more systematic and evidence-based, policy-making. Post-normal science (PNS), with its ‘extended peer community’, has emerged as one approach, whilst others focus on procedural reforms of the policy process, particularly on enhancing democratic decision-making. This paper applies a novel analytical framework to a primarily documentary analysis of three cases we argue are wicked—Canadian regulatory review of health products and food, European union (EU) environmental thematic strategies, and United Kingdom (UK) energy and climate change policy. It explores how various responses to wicked issues are implemented, through the ‘lenses’ of PNS and, more generally, ‘democratic and effective decision-making’. It finds such responses are often limited by practical and fundamental barriers relating to handling of uncertainty, issue framing, participation, power, politics, and attitude to evidence. We draw conclusions about future research on PNS, particularly the need to more clearly relate theory to different strands of literature on the evidence–policy-making relationship, and to continue empirical testing.

Boone et al., 2009, NUSAP method for evaluating the data quality in a quantitative microbial risk assessment model for salmonella in the pork production chain Risk Analysis 29 (4), pp. 502-517
Abstract The numeral unit spread assessment pedigree (NUSAP) system was implemented to evaluate the quality of input parameters in a quantitative microbial risk assessment (QMRA) model for Salmonella spp. in minced pork meat. The input parameters were grouped according to four successive exposure pathways: (1) primary production (2) transport, holding, and slaughterhouse, (3) postprocessing, distribution, and storage, and (4) preparation and consumption. An inventory of 101 potential input parameters was used for building the QMRA model. The characteristics of each parameter were defined using a standardized procedure to assess (1) the source of information, (2) the sampling methodology and sample size, and (3) the distributional properties of the estimate. Each parameter was scored by a panel of experts using a pedigree matrix containing four criteria (proxy, empirical basis, method, and validation) to assess the quality, and this was graphically represented by means of kite diagrams. The parameters obtained significantly lower scores for the validation criterion as compared with the other criteria. Overall strengths of parameters related to the primary production module were significantly stronger compared to the other modules (the transport, holding, and slaughterhouse module, the processing, distribution, and storage module, and the preparation and consumption module). The pedigree assessment contributed to select 20 parameters, which were subsequently introduced in the QMRA model. The NUSAP methodology and kite diagrams are objective tools to discuss and visualize the quality of the parameters in a structured way. These two tools can be used in the selection procedure of input parameters for a QMRA, and can lead to a more transparent quality assurance in the QMRA.

Van der Sluijs et al., 2008, Exploring the quality of evidence for complex and contested policy decisions, Environ. Res. Lett. 3 (2008) 024008 (9pp)
Abstract Policy decisions on complex environmental risks often involve contested science. Typically there are no 'facts' that entail a unique correct policy. The evidence that is embodied in scientific policy advice requires quality assessment. Advice should be relevant to the policy issue, scientifically tenable and robust under societal scrutiny. In 2003, the Netherlands Environmental Assessment Agency adopted a standardized method, referred to as 'guidance', whereby key quality aspects of knowledge production and use are exhibited through a checklist for uncertainty assessment and communication. Although the guidance is not fully used within all projects yet, it is increasingly used, attitudes towards dealing with uncertainty in performing and reporting environmental assessments have changed, and communication on uncertainty in the agency's reports has improved over the past five years. In this letter, we present results from the application of the guidance to controversies on the risks of ambient particulate matter. The active deliberation on uncertainty in the policy–advisory setting brings about a joint learning process for advisors and policy makers, which leads to a deeper understanding and increased awareness of the phenomenon of uncertainty and its policy implications.

Wardekker et al., 2008 Uncertainty Communication in Environmental Assessments: Views from the Dutch Science-Policy interface Environmental Science & Policy, 11, 627-641
Abstract Scientific assessments of environmental problems, and policy responses to those problems, involve uncertainties of many sorts. Meanwhile, potential impacts of wrong decisions can be far-reaching. This article explores views on uncertainty and uncertainty communication in the Dutch science-policy interface and studies several issues concerning presentation of uncertainty information. Respondents considered uncertainty communication to be important, but it should be concise and policy relevant. Several factors influence policy relevance, including the place of an issue in the policy cycle, and its novelty, topicality and controversiality. Respondents held particular interest in explicit communication on the implications of uncertainty. Related to this, they appreciated information on different sources and types of uncertainty and qualitative aspects of uncertainty (e.g. pedigree charts). The article also studies probability terms, particularly for IPCC's 33–66% probability interval (‘about as likely as not’). Several terms worked reasonably well, with a median interpretation of 40–60%. Finally, as various target groups have different information needs and different amounts of attention for various parts of a report or communication process, it is important to progressively disclose uncertainty information throughout the communication. Improved communication of uncertainty information leads to a deeper understanding and increased awareness of the phenomenon of uncertainty and its policy implications.

Farrell, 2008, The Politics of Science and Sustainable Development: Marcuse’s New Science in the 21st CenturyCapitalism Nature Socialism, 19 (4), 68-83.
Abstract The arguments presented in this essay build on the work of the critical theorist Herbert Marcuse, in order to address the role of science in environmental governance. The main focus of the essay is the simultaneous domination of humans and non-humans*of ‘‘man’’ and ‘‘nature’’*that takes place in the course of building and maintaining industrial systems of economic production.
The core thesis is that postnormal science,1 a discourse on scientific methodology and its related techniques, can be understood as a realization of the new modality of science that Marcuse predicted in One-Dimensional Man, where: ‘‘pacified existence . . . the repressed final cause behind the scientific enterprise. . . were [it] to materialize and become effective, [is accompanied by a situation where] the Logos of technics would open a universe of qualitatively different relations between man and man, and man and nature.’’
The proposition that postnormal science is a manifestation of Marcuse’s ‘‘new modality of science’’ is significant for the study of 21st century environmental politics, because Marcuse proposed that this new modality of science might provide a means for escaping the oppression of one-dimensional thinking.

Maxim and Van der Sluijs, 2007, Uncertainty: cause or effect of stakeholders' debates? Analysis of a case study: the risk for honey bees of the insecticide Gaucho® Science of the Total Environment, 376, 1-17.
Abstract The social construction of uncertainty plays a major role in environmental decision-making. Methods for assessing this aspect of scientific knowledge quality are lacking. Our analysis of the French debate on the risk that the insecticide Gaucho® (active substance: imidacloprid) forms for honeybees is particularly relevant to this theoretical and practical gap. Based on our analysis, we propose six knowledge quality criteria that can assist in assessing the information communicated in an argumentative public process: reliability of the information – it must be based on all available scientific knowledge; robustness of the information – it must take into account criticism; use of the information produced by other stakeholders; relevancy of the arguments for issue under debate; logical coherence of the discourse; and legitimacy of the information source. Further, our findings deepen the understanding of the relationships between the social, economic, and institutional stakes of the actors involved in the debate and their strategies of ‘creating uncertainty’. Finally, we compare the findings of this case study with the twelve lessons drafted by the European Environmental Agency (EEA) in its report Late lessons from early warnings, and we draft two more lessons. These lessons can be applied to future policy in order to minimize the repetition of past mistakes.

Refsgaard et al., 2007, Uncertainty in the Environmental Modelling Process: A ReviewEnvironmental Modelling & Software. 22 (11), 1543-1556.
Abstract A terminology and typology of uncertainty is presented together with a framework for the modelling process, its interaction with the broader water management process and the role of uncertainty at different stages in the modelling processes. Brief reviews have been made of 14 different (partly complementary) methods commonly used in uncertainty assessment and characterisation: data uncertainty engine (DUE), error propagation equations, expert elicitation, extended peer review, inverse modelling (parameter estimation), inverse modelling (predictive uncertainty), Monte Carlo analysis, multiple model simulation, NUSAP, quality assurance, scenario analysis, sensitivity analysis, stakeholder involvement and uncertainty matrix. The applicability of these methods has been mapped according to purpose of application, stage of the modelling process and source and type of uncertainty addressed. It is concluded that uncertainty assessment is not just something to be added after the completion of the modelling work. Instead uncertainty should be seen as a red thread throughout the modelling study starting from the very beginning, where the identification and characterisation of all uncertainty sources should be performed jointly by the modeller, the water manager and the stakeholders.

Kloprogge and Van der Sluijs, 2006, The inclusion of stakeholder knowledge and perspectives in integrated assessment of climate change Climatic Change, 75 (3) 359-389
Abstract Over the past few decades Integrated Assessment (IA) has emerged as an approach to link knowledge and action in a way that is suitable to accommodate uncertainties, complexities and value diversities of global environmental risks. Responding to the complex nature of the climate problem and to the changing role of climate change in the international climate policy process, the scientific community has started to include stakeholder knowledge and perspectives in their assessments. Participatory Integrated Assessment (PIA) is in its early stage of development. Methodology varies strongly across PIA projects. This paper analyzes four recent IA projects of climate change that included knowledge or perspectives from stakeholders in one-way or another. Approaches and methods used turn out to differ in whether stakeholders are involved actively or passively, whether the approach is bottom-up or top-down, and whether the different functions in the IA process are open or closed to stakeholder input. Also, differences can be seen in the degree to which boundaries are pre-set that limit the roles and domains of competencies attributed to each scientific or non-scientific participant (so-called boundary work). The paper discusses pros and cons of the various approaches identified, and outlines heuristics and considerations to assist those who plan, design or fund new IA processes with stakeholder input on what approaches best to choose in view of the objectives for stakeholder involvement, in view of the role that the IA plays in the overall risk management process and in view of considerations regarding boundary work.

Risbey et al., 2005, Application of a Checklist for Quality Assistance in Environmental Modelling to an Energy Model. Environmental Modeling & Assessment 10 (1), 63-79.
Abstract Large, complex energy models present considerable challenges to develop and test. Uncertainty assessments of such models provide only partial guidance on the quality of the results. We have developed a model quality assistance checklist to aid in this purpose. The model checklist provides diagnostic output in the form of a set of pitfalls for the model application. The checklist is applied here to an energy model for the problem of assessing energy use and greenhouse gas emissions. Use of the checklist suggests that results on this issue are contingent on a number of assumptions that are highly value-laden. When these assumptions are held fixed, the model is deemed capable of producing moderately robust results of relevance to climate policy over the longer term. Checklist responses also indicate that a number of details critical to policy choices or outcomes on this issue are not captured in the model, and model results should therefore be supplemented with alternative analyses.

Van der Sluijs et al., 2005 Combining Quantitative and Qualitative Measures of Uncertainty in Model based Environmental Assessment: the NUSAP System, Risk Analysis, 25 (2). p. 481-492
Abstract This article discusses recent experiences with the Numeral Unit Spread Assessment Pedigree (NUSAP) system for multidimensional uncertainty assessment, based on four case studies that vary in complexity. We show that the NUSAP method is applicable not only to relatively simple calculation schemes but also to complex models in a meaningful way and that NUSAP is useful to assess not only parameter uncertainty but also (model) assumptions. A diagnostic diagram can be used to synthesize results of quantitative analysis of parameter sensitivity and qualitative review (pedigree analysis) of parameter strength. It provides an analytic tool to prioritize uncertainties according to quantitative and qualitative insights in the limitations of available knowledge. We show that extension of the pedigree scheme to include societal dimensions of uncertainty, such as problem framing and value-laden assumptions, further promotes reflexivity and collective learning. When used in a deliberative setting, NUSAP pedigree assessment has the potential to foster a deeper social debate and a negotiated management of complex environmental problems.

Van der Sluijs 2005, Uncertainty as a monster in the science policy interface: four coping strategies, Water science and technology, 52 (6) 87–92
Abstract Using the metaphor of monsters, an analysis is made of the different ways in which the scientific community responds to uncertainties that are hard to tame. A monster is understood as a phenomenon that at the same moment fits into two categories that were considered to be mutually excluding, such as knowledge versus ignorance, objective versus subjective, facts versus values, prediction versus speculation, science versus policy. Four styles of coping with monsters in the science–policy interface can be distinguished with different degrees of tolerance towards the abnormal: monster-exorcism, monsteradaptation, monster-embracement, and monster-assimilation. Each of these responses can be observed in the learning process over the past decades and current practices of coping with uncertainties in the science policy interface on complex environmental problems. We might see this ongoing learning process of the scientific community of coping with complex systems as a dialectic process where one strategy tends to dominate the field until its limitations and shortcomings are recognized, followed by a rise of one of the other strategies. We now seem to find ourselves in a phase with growing focus on monster assimilation placing uncertainty at the heart of the science–policy and science–society interfaces.

Janssen et al, 2005, A guidance for assessing and communicating uncertainties, Water science and technology, 52 (6) 125–131
Abstract In the daily practice of science for policy, as experienced by governmental agencies which inform the policy and the public on the state and outlook of the environment, there is a pressing need for guidance in assessing and communicating uncertainties. This need extends beyond the quantitative assessment of uncertainties in model results, and focuses on the entire process of environmental assessment, running from problem framing towards reporting the results of the study. Using the Netherlands Environmental Assessment Agency (RIVM/MNP) as a case, the development, structure and content of such a guidance system is highlighted. Conditions for a successful implementation of the guidance system are discussed, and some prospects for future work are outlined.

Craye et al, 2005 A reflexive approach to dealing with uncertainties in environmental health risk science and policy International Journal for Risk Assessment and Management, 5 (2), p. 216-236
Abstract Based on insights obtained through an analysis of an environmental health risk controversy, we developed a reflexive approach to uncertainty assessment, explicitly acknowledging the complexity of the knowledge production process. The approach aims at interactively exploring uncertainty in relation to different scientific framings, societal perspectives and policy options. The structure of the discussion scheme used for the exploration is based on the concept of `pedigree of knowledge'. The discussion protocol is designed to guarantee conditions for a reasoned debate. A workshop has been organised, during which the approach has been deployed to assess scientific studies, that had been produced in the context of a socio-political debate on possible health effects from waste incineration. The results obtained show the approach has potential to trigger a profound social debate and a negotiated management of risk. Pro-active use of the approach could enhance the quality and robustness of the knowledge input in policy making.