Quantifying uncertainty in ranking problems with composite indicators: a Bayesian approach

Abstract

Purpose

The purpose of this paper is to present an inductive methodology, which supports ranking of entities. Methodology is based on Bayesian latent variable measurement modeling and makes use of assessment across composite indicators to assess internal and external model validity (uncertainty is used in lieu of validity). Proposed methodology is generic and it is demonstrated on a well‐known data set, related to the relative position of a country in a “doing business.”

Design/methodology/approach

The methodology is demonstrated using data from the World Banks' “Doing Business 2008” project. A Bayesian latent variable measurement model is developed and both internal and external model uncertainties are considered.

Findings

The methodology enables the quantification of model structure uncertainty through comparisons among competing models, nested or non‐nested using both an information theoretic approach and a Bayesian approach. Furthermore, it estimates the degree of uncertainty in the rankings of alternatives.

Research limitations/implications

Analyses are restricted to first‐order Bayesian measurement models.

Originality/value

Overall, the presented methodology contributes to a better understanding of ranking efforts providing a useful tool for those who publish rankings to gain greater insights into the nature of the distinctions they disseminate.

Citation

Zampetakis, L.A. and Moustakis, V.S. (2010), "Quantifying uncertainty in ranking problems with composite indicators: a Bayesian approach", Journal of Modelling in Management, Vol. 5 No. 1, pp. 63-80. https://doi.org/10.1108/17465661011026176