Calibrating Expert Assessments Using Hierarchical Gaussian Process Models
Perälä, T., Vanhatalo, J., & Chrysafi, A. (2020). Calibrating Expert Assessments Using Hierarchical Gaussian Process Models. Bayesian Analysis, 15(4), 1251-1280. https://doi.org/10.1214/19-BA1180
Published inBayesian Analysis
International Society for Bayesian Analysis
Expert assessments are routinely used to inform management and other decision making. However, often these assessments contain considerable biases and uncertainties for which reason they should be calibrated if possible. Moreover, coherently combining multiple expert assessments into one estimate poses a long-standing problem in statistics since modeling expert knowledge is often difficult. Here, we present a hierarchical Bayesian model for expert calibration in a task of estimating a continuous univariate parameter. The model allows experts’ biases to vary as a function of the true value of the parameter and according to the expert’s background. We follow the fully Bayesian approach (the so-called supra-Bayesian approach) and model experts’ bias functions explicitly using hierarchical Gaussian processes. We show how to use calibration data to infer the experts’ observation models with the use of bias functions and to calculate the bias corrected posterior distributions for an unknown system parameter of interest. We demonstrate and test our model and methods with simulated data and a real case study on data-limited fisheries stock assessment. The case study results show that experts’ biases vary with respect to the true system parameter value and that the calibration of the expert assessments improves the inference compared to using uncalibrated expert assessments or a vague uniform guess. Moreover, the bias functions in the real case study show important differences between the reliability of alternative experts. The model and methods presented here can be also straightforwardly applied to other applications than our case study. ...
PublisherInternational Society for Bayesian Analysis
Publication in research information system
MetadataShow full item record
Related funder(s)European Commission
The content of the publication reflects only the author’s view. The funder is not responsible for any use that may be made of the information it contains.
Additional information about fundingTP was funded by the Academy of Finland (through Academy Research Fellow Grant to Anna Kuparinen), by the University of Jyväskylä, and by the European Research Council (COMPLEX-FISH 770884 to Anna Kuparinen). AC was funded by the University of Helsinki through DENVI Doctoral Program Fellowship. JV has additionally been funded by Academy of Finland [Grants 304531 and 317255] and the research funds of University of Helsinki [decision No. 465/51/2014]. ...
Showing items with similar title or keywords.
Kuronen, Mikko; Särkkä, Aila; Vihola, Matti; Myllymäki, Mari (Springer Science and Business Media LLC, 2022)We propose a hierarchical log Gaussian Cox process (LGCP) for point patterns, where a set of points xx affects another set of points yy but not vice versa. We use the model to investigate the effect of large trees on the ...
Prediction of leukocyte counts during paediatric acute lymphoblastic leukaemia maintenance therapy Karppinen, Santeri; Lohi, Olli; Vihola, Matti (Nature Publishing Group, 2019)Maintenance chemotherapy with oral 6-mercaptopurine and methotrexate remains a cornerstone of modern therapy for acute lymphoblastic leukaemia. The dosage and intensity of therapy are based on surrogate markers such as ...
Chrysafi, Anna; Cope, Jason M.; Kuparinen, Anna (Pergamon Press, 2019)Data-limited fisheries are a major challenge for stock assessment analysts, as many traditional data-rich models cannot be implemented. Approaches based on stock reduction analysis offer simple ways to handle low data ...
Heikkinen, Risto; Sipilä, Juha; Ojalehto, Vesa; Miettinen, Kaisa (Inderscience Publishers, 2022)We study data-driven decision support and formalise a path from data to decision making. We focus on lot sizing in inventory management with stochastic demand and propose an interactive multi-objective optimisation approach. ...
Myllymäki, Mari (University of Jyväskylä, 2009)