The Art and Science of Being Uncertain

This post from Dr. Jakub Bijak from the University of Southampton discusses the challenges surrounding the management and communication of demographic uncertainty, and the methods available for approaching these problems.

Some of the most captivating questions in contemporary demography are about what we do not know: the realms of uncertainty. This is especially visible in population forecasts, which remain one of the best-selling products of population sciences. There is a lot of demand for knowledge about future population size, structure, or specific demographic processes. However, uncertainty is everywhere. Forecasts are far from being the only area of demography where knowledge is lacking. Population estimates, and in fact all other statements about demographic processes, also come with a tacit question: how certain are we about them? In this essay, I look at demographic uncertainty, and discuss some ways of dealing with it.

Uncertainty

To start with demographic forecasts, it should be hopefully obvious that they are not a clichéd crystal ball, and that perfect foresight is not possible, especially in social sciences. Still, even uncertain forecasts can be very useful to help the national or local governments as well as businesses make better and more robust decisions on a range of topics, from pension systems or health care, through school places, to the location of new supermarkets.

What makes demography a unique area for developing and testing methods to aid decisions under uncertainty is its highly empirical orientation, and strong links with statistics and probability – since its beginnings as a scientific discipline in 1660s, demography has not existed without data. Besides, population is somewhat better predictable than other social domains, such as the economy or politics. In demography, a lot of information about the future population size and structure is already encoded in the structures of today. In the absence of severe shocks, this demographic inertia allows for a reasonable accuracy of prediction, at least up to a horizon of one generation ahead.

The Horizon

There are many sources of demographic uncertainty: the characteristics of population processes, the different drivers behind them, the models used and their parameters, the imprecise or conflicting measurements, and the key, irreducible uncertainty about the future. The natural framework for analysing all sources of uncertainty jointly and coherently is offered by Bayesian statistics, a 18^th century invention, rediscovered in the 1950s. There are already numerous articles following the principles of Bayesian demography, including a post on this blog.

Amongst the components of population change, migration, and especially international migration, is notoriously difficult to forecast. Measurement of migration is challenging, its theoretical background is weak and fragmented, and migration processes are susceptible to unpredictable shocks: political events, policy changes, or natural and human-made disasters. As a result, uncertainty increases very rapidly as the forecast horizon moves into the future. Longer-term migration forecasts remain a special challenge.

There are hardly any methods that would capture the unpredictable character of migration beyond the horizon of a few years, even though we may expect the long-term, secular processes to exhibit stable regularities – for example, in the recent decades around 3 per cent of the world population have been living outside their country of birth.. The results from the Bayesian prototype models currently being prepared for the United Nations are very encouraging, even though more works still needs to be done on bridging the short-term and long-term perspectives.

Some methodological solutions to these challenges could come from strengthening the theoretical foundations of demography as long as such theories are scientifically rigorous and based in the observations of the social world. Currently, the weak theoretical background of demography adds yet another layer of uncertainty. In particular, the discipline is lacking theoretical micro-foundations, which should ideally be formulated through formal modelling. There are excellent frameworks, such as multilevel models, event history analysis and biographical models, microsimulations, or agent-based models, which could all be used for this purpose.

The main epistemological challenge is to make sure that theories are based on observations of the demographic processes, rather than on some not always realistic assumptions and hypotheses. Ideally, they should be underpinned by as much evidence as possible – quantitative as well as qualitative. Clearly, generalizable inferences on populations have to be based on statistical methods and quantitative evidence (see also on this blog), but there is scope for qualitative information to provide insights into the mechanisms or drivers of behaviour of simulated agents, or to guide the construction of models.

Some tools, such as the agent-based computational models, are especially capable of incorporating different sources of knowledge about mechanisms driving the social and demographic change. In the process of building such models, a correct description of uncertainty and a proper design of computational experiments are crucial. The work on demographic applications of such approaches is still in its early stages, but methods based on emulators (meta-models: statistical models of the computational models), or on Bayesian melding, are especially promising here.

The final key component of uncertainty is related to the way it is communicated to the users of the demographic analyses. The task is far from trivial: successful communication needs to acknowledge the cognitive burden associated with processing information about uncertainty, and help use it appropriately for making decisions. There already exist many tools to facilitate this, from decision analysis, through foresight studies, to risk management approaches; however, what is absolutely necessary is an honest exchange of information on the user needs, and on what kind of support demography can realistically offer.

Possible Pathways

No matter how hard we try, demographic uncertainty will not disappear. We have to accept it, embrace it, and make it a part of a demographic routine, in the same way as demographers of the past did with empiricism and the use of observational data. Acknowledging uncertainty in the results of the analysis is not a weakness. Conversely, being explicit about the true limitations forms an honest and transparent way of doing research in social sciences. However, dealing with uncertainty requires both the science of statistical methods and the art of communication. It is not an easy task; however, as quipped by Bertrand Russell in Philosophy for Laymen,

“To endure uncertainty is difficult, but so are most of the other virtues […] For it is not enough to recognize that all our knowledge is, in a greater or less degree, uncertain and vague; it is necessary, at the same time, to learn to act upon the best hypothesis without dogmatically believing it.”

Thus, our current endeavours are just the beginning of a fascinating journey into the unknown.

Acknowledgements: This essay presents my personal account, but the ideas and thoughts included in it were honed during many discussions with my colleagues, students and several co-authors, whom I wish to sincerely thank.

Tags: Bayesian methodology, Forecasting, Uncertainty