Ecological Modelling 230 (2012) 50-62
Ecological Modelling 230 (2012) 50-62 Contents lists available at SciVerse ScienceDirect Ecological Modelling ELSEVlER jou rna I homepa g e: www .elsevier .co m/locate/eco I model Metrics for evaluating performance and uncertainty of Bayesian network models Bruce G. Marcot* U.S. Forest Service, Pacific Northwest Research Station, 620 S. W. Main Street. Portland, OR 9720~. United States ARTICLE INFO ABSTRACT Article history: This paper presents a selected set of existing and new metrics for gauging Bayesian network model Received 12 September 2011 performance and uncertainty. Selected existing and new metrics are discussed for conducting model Received in revised form 10 january 2012 sensitivity analysis (variance reduction, entropy reduction, case file simulation); evaluating scenarios Accepted 11 january 2012 (influence analysis); depicting model complexity (numbers of model variables, links, node states, con ditional probabilities, and node cliques); assessing prediction performance (confusion tables, covariate Keywords: and conditional probability-weighted confusion error rates, area under receiver operating characteristic Bayesian network model Uncertainty curves, k-fold cross-validation, spherical payoff. Schwarz' Bayesian information criterion, true skill statis Model performance tic, Cohen's kappa); and evaluating uncertainty of model posterior probability distributions (Bayesian Model validation credible interval, posterior probability certainty index, certainty envelope, Gini coefficient). Examples are Sensitivity analysis presented of applying the metrics to 3 real-world models of wildlife population analysis and manage Error rates ment. Using such metrics can vitally bolster model credibility, acceptance, and appropriate application, Probability analysis particularly when informing management decisions. Published by Elsevier B.V. 1. Introduction different outcome states, that is, the spread of alternative predic tions.
[Show full text]