Assessing whether decisions are more sensitive to preference or prediction uncertainty with a value of information approach
Fridolin Haag
Arjun Chennu
Abstract
In many decisions, we are not only uncertain about the predicted outcomes of decision alternatives but also about stakeholder preferences regarding these outcomes. Further information collection may reduce uncertainties, but is costly. We present and apply a framework to identify the most decisive uncertainties and prioritize data collection efforts based on value of information (VoI) sensitivity analysis. Preference uncertainty is usually not explicitly considered in VoI analysis or in standard utility theory. Based on the expected expected utility (EEU) concept, we consider uncertain predictions and preferences jointly in decisions and subsequent VoI analysis. We focus on the expected value of partially perfect information (EVPPI) and adapt a fast, given-data algorithm for estimating this metric. The framework is motivated by complex environmental decision problems and we apply it to a hypothetical multi-criteria decision regarding coral reef management with conflicting stakeholder perspectives. The results show that better understanding of stakeholder positions can be as relevant as improving system understanding. For one perspective, preference model parameters had the highest EVPPI, while for another predictive uncertainties of the reef system attributes were more relevant. For two perspectives, the decision was largely insensitive. By considering predictive and preferential uncertainty on an equal footing in VoI analysis, we open up possibilities to design data collection for decision support processes more efficiently.
Haag F. and Chennu A. (2023) Assessing whether decisions are more sensitive to preference or prediction uncertainty with a value of information approach. 121: 102936. 10.1016/j.omega.2023.102936
@article{Haag2023,
Title = {Assessing whether decisions are more sensitive to preference or prediction uncertainty with a value of information approach},
Author = {Haag, Fridolin and Chennu, Arjun},
Editor = {},
Year = {2023},
Pages = {102936},
Volume = {121},
Doi = {10.1016/j.omega.2023.102936},
Abstract = {In many decisions, we are not only uncertain about the predicted outcomes of decision alternatives but also about stakeholder preferences regarding these outcomes. Further information collection may reduce uncertainties, but is costly. We present and apply a framework to identify the most decisive uncertainties and prioritize data collection efforts based on value of information (VoI) sensitivity analysis. Preference uncertainty is usually not explicitly considered in VoI analysis or in standard utility theory. Based on the expected expected utility (EEU) concept, we consider uncertain predictions and preferences jointly in decisions and subsequent VoI analysis. We focus on the expected value of partially perfect information (EVPPI) and adapt a fast, given-data algorithm for estimating this metric. The framework is motivated by complex environmental decision problems and we apply it to a hypothetical multi-criteria decision regarding coral reef management with conflicting stakeholder perspectives. The results show that better understanding of stakeholder positions can be as relevant as improving system understanding. For one perspective, preference model parameters had the highest EVPPI, while for another predictive uncertainties of the reef system attributes were more relevant. For two perspectives, the decision was largely insensitive. By considering predictive and preferential uncertainty on an equal footing in VoI analysis, we open up possibilities to design data collection for decision support processes more efficiently.},
}
TY - JOUR
AU - Haag, Fridolin
AU - Chennu, Arjun
TI - Assessing whether decisions are more sensitive to preference or prediction uncertainty with a value of information approach
PY - 2023
SP - 102936
VL - 121
DO - 10.1016/j.omega.2023.102936
AB - In many decisions, we are not only uncertain about the predicted outcomes of decision alternatives but also about stakeholder preferences regarding these outcomes. Further information collection may reduce uncertainties, but is costly. We present and apply a framework to identify the most decisive uncertainties and prioritize data collection efforts based on value of information (VoI) sensitivity analysis. Preference uncertainty is usually not explicitly considered in VoI analysis or in standard utility theory. Based on the expected expected utility (EEU) concept, we consider uncertain predictions and preferences jointly in decisions and subsequent VoI analysis. We focus on the expected value of partially perfect information (EVPPI) and adapt a fast, given-data algorithm for estimating this metric. The framework is motivated by complex environmental decision problems and we apply it to a hypothetical multi-criteria decision regarding coral reef management with conflicting stakeholder perspectives. The results show that better understanding of stakeholder positions can be as relevant as improving system understanding. For one perspective, preference model parameters had the highest EVPPI, while for another predictive uncertainties of the reef system attributes were more relevant. For two perspectives, the decision was largely insensitive. By considering predictive and preferential uncertainty on an equal footing in VoI analysis, we open up possibilities to design data collection for decision support processes more efficiently.
ER -
