Valuing Probabilistic Foresight
The devaluing of probable foresight may be the greatest challenge presently facing our emerging profession. Most people in society, perhaps because they don’t know much science, are guilty of undetermination bias, of imagining the future is substantially freer, is less predictable or constrained, than is the case. We discuss this and other Emotional-Cognitive Biases in Chapter 4.
Even today, too many well-meaning foresight professionals still declare a personal belief that “the future cannot be predicted”, at least in meaningful ways in the area in which they practice. In their current world view, they see few to no highly probable destinations ahead that are relevant to the strategy, plans, and actions they consider for themselves and their clients. As a result, they naively imagine the future as far more of an “open field” than it actually is. They see the future as free to be crafted to their whim, missing the social and environmental forces, trends, and constraints that are busy creating predictable futures all around them. That means they miss obvious future destinations they would be very wise to discover, share with their clients, and use to their best advantage.
Even the World Future Society, our field’s oldest organization, regularly champions this misguided view. For example, the opening lines in the WFS 2014 conference brochure read: “The future is not a destination. It’s the end result of the actions we take today.” So too Reddit Futurology, today the largest online platform dedicated to the future, with over 12 million “Futurists”, has the following language in its definition of future(s) studies on its home page: “practitioners realize there is no single future, only alternative futures ahead.” Even the home page of the new School for the Future of Innovation in Society at ASU proclaims, in adaptation from Shakespeare’s Julius Ceasar: “It is not in the stars to hold our destiny, but in ourselves.”
As we will see, all of these statements, innocuous and well-meaning as they might seem, are dangerously biased half-truths. A good foresighter learns to see that both highly probable and possible futures always lie ahead. If either perspective is ignored, our foresight is greatly weakened as a result.
Even the Association for Professional Futurists claims, at the top of their “What is a Futurist?” page: “It is not the goal of a futurist to predict what will happen in the future.” This perspective is simply wrong. Futurists like Hans Rosling of Gapminder constantly make both forecasts and predictions in their public presentations. We rely on implicit and explicit predictions from futurists and foresighters all the time. Those foresight professionals who don’t like to do prediction, either because they don’t feel confident in any particular futures, or because they prefer not to share their internal confidences, need to stop saying that prediction and uncovering the probable future are not key parts of our field. They are.
If something like the 95/5 rule operates in our universe, then while the vast majority of our future is best thought of as freely chosen evolutionary paths determined by us, a critical minority of our future will always be developmental destinations, determined by our universal environment and its laws and structure, unfolding all around us. Science already tells us that several processes and parts of our future are convergent destinations, single, predictable outcomes that human societies everywhere are funneling toward, regardless of our individual choices, or as a result of the average distribution of those choices. As we’ll see in Chapter 3, these special destinations will arrive in roughly the same way on all Earth-like planets in our universe, whether we individually want them to or not.
For example, we can confidently predict that if humanity persists, our scientific and technical knowledge will continue to advance at accelerating rates, an outcome most of us are happy to expect. We also know that in our current biological form we’ll all inevitably grow old and die, though we’re less uniformly happy about that expectation. We can also see and measure the greatly increasing resiliency and redundancy of the knowledge we use to run our modern societies. Though many pessimists conveniently forget this fact, no catastrophe could seriously set back the hard-won, stratospheric gains we’ve made in our scientific and technical knowledge in recent generations, unless it erased our entire species and its many redundant archives in the process. For more on that, see Lewis Dartnell’s The Knowledge: How to Rebuild Civilization in the Aftermath of a Cataclysm (2015)
We can forecast an exponentially growing standard of living, at least as long as our tempers hold and our environment continues to survive its mistreatment. My favorite monthly, The Atlantic, does a great job reporting that story. See Charles Kenny’s 2015: The Best Year In History for the Average Human Being, Atlantic.com (2015). See also Kenny’s Getting Better: Why Global Development is Succeeding (2011). My favorite weekly, the Economist, also does an excellent job on this front. For some Economist global development data, and more, read Bill and Melinda Gate’s excellent update on the impressive gains made by the poorest people in the world over the last 25 years, Warren Buffet’s Best Investment, Gatesnotes.com (2017). Global technology-aided philanthropy, both from institutions and from individuals, is an increasingly powerful and measurable force for good, and it’s still just getting started. With the connectivity, quantification, and AI that are coming, we can expect digital systems like groupnets (see The Internet of Families: The Powerful and Intimate Future of Groupnets, Eversmarterworld.com (2015) to help us manage the deprivations and violence that some of us face daily, and bind us into one global family more tightly than ever before.
Groupnets and other societal developmental futures are happening because they predictably satisfy some of our deepest biological preferences. These preferences include our longstanding trend of decreasing average intensity of interpersonal violence as our societies and their institutions continue to develop. See Stephen Pinker’s masterful The Better Angels of Our Nature: Why Violence Has Declined (2012) for more on that surprisingly predictable megatrend in human civilization. Violence reduction is also accelerating, though you wouldn’t believe it if you read the evening news, which opportunistically reports every short-term and local reversal in the most apocalyptic terms—a great way to keep distracting a public that doesn’t yet know better. But they will.
Most practitioners will reluctantly recognize there are parts of the future that are highly predictable (demography, humanity’s negative impact on the biosphere, information growth, accelerating technological change) but there is a vocal minority that misconstrues nonlinear science and chaos theory to argue against predictability “where it matters,” in social and organizational issues and questions. Too many foresighters still neglect to find predictable trends and constancies which science, experience, and evidence tell us are likely to influence or constrain their contemplated future, before they jump into exploring creative possibilities and converging on preferences.
We may wish the world had no such predictable futures, but wishing doesn’t make them go away. Understanding as many as possible of these highly likely destinations is critical if we are to make wise strategies, plans, and actions in other, less predictable parts of the future that lie directly in the realm of our free moral choice.
Good foresight practitioners do prediction analysis and collect critical feedback on their past predictions, to measure their accuracy, to understand what methods and bias were involved, and record how much they cost to generate. They estimate how useful their best predictions would have been if they had been made well, communicated to the right decisionmakers, and acted on in a timely manner. They use this feedback to make better predictions with things that matter most, and to tell better stories that include probabilities and predictions. We’ll offer a few of our favorite forecasts and predictions in Chapter 2.
Many of us don’t realize how many aspects of our social, economic, and technological future are now broadly probabilistically predictable. Even the weather, a classic nonlinear complex adaptive system, has seen great improvements in replicable predictability. Our prediction horizon for useful weather forecasts has grown from two days ahead in the 1950s, to a current twenty-one days in advance, in our best weather models, with the use of increasingly high resolution and distributed sensing, deterministic forecasting, ensembles of competing weather models and lots of human judgment.
As the world digitizes, more and more of its systems are being forecast in probabilistic ways. For more on this global social megatrend, read Nate Silver’s The Signal and the Noise (2012), Patrick Tucker’s The Naked Future (2014), and Eric Siegel’s Predictive Analytics (2012). Each of these is an introduction to our recent great strides in probabilistic prediction. As Siegel describes in his chapter on IBM’s Watson, the computer that beat the two best humans on the planet at the Jeopardy! quiz game in 2011, increasingly fast computers, big data, and simple statistical and crudely biologically-inspired algorithms (e.g., artificial neural networks) are allowing computers to predict the most useful answer to human queries across a vast domain of cultural knowledge. This is only one of the more stunning examples of predictive processes where what was once random guessing on the part of the machine is transitioning to probabilistic certainty.
Consider Philip Tetlock’s IARPA-sponsored Good Judgment Project, a collective forecasting and prediction platform that weights the inputs of a cognitively diverse group of amateur forecasters invited from the general public. Now in its third year, this research platform is outperforming even CIA analysts at many types of social, political and economic prediction as this NPR article, and Tetlock and Gardner’s new book,
Superforecasting: The Art and Science of Prediction, 2015, attests.
Please read Superforecasting if you still have a conscious or unconscious bias against finding more of the probable future for your clients. Tetlock’s book demonstrates the promise of using evidence-based methods in prediction, and it shows the great promise of a still-emerging foresight tool, online forecasting and prediction markets. Prediction markets are finally emerging for each of the most important topical domains of foresight, which can be remembered by the acronym STEEPS (science, technology, economics, environment, politics, and society). Such markets are a great crowd-wisdom tool you can use to explore futures with your clients. For a few examples, Metaculus is a new science and technology prediction market, and PredictIt is a new real-money political prediction market. Augur is decentralized real-money prediction market built on top of the Ethereum blockchain (a platform similar to Bitcoin). Its leaders are perhaps a bit too utopian to have a meaningful impact at present, but it is just one of many new efforts at incentivizing and accelerating social prediction, at tapping the “wisdom of the crowd.”
Another impressive new collaborative foresight and strategy platform is Wikistrat, a “massively multiplayer online consultancy (MMOC)” that runs multiday strategy and policy simulations with 100 or so of its 1,000+ online experts in an online version of two fifty year old foresight tools, the Delphi method and Organizational Wargaming. We’ll explore Delphi further in Chapter 4.
With good facilitators, group training, cognitive diversity, and good software, foresight practitioners can now build useful models of the future much faster and more robustly than the traditional research-and-report methods used by scholars and management consultancies.
Consider also the European Commission’s Futurium digital futures project which engaged 3500 people in online and in-person workshops over three years, eliciting visions for a digital Europe in 2050. This is an inspiring example of collective preference foresight, and such platforms will greatly proliferate and become more probabilistic in coming years, as connectivity and automation makes them ever cheaper and easier to build. See their final report, A Journey into 2050 (2014) for insightful crowdsourced perspectives on digital futures and their policy implications.
Projects like these will eventually become permanent, free, and machine-readable collective foresight platforms. Independently, Kevin Kelly, Michelle Bowman and I have all called for the development of a Futurepedia, a permanent free global foresight content encyclopedia and polling and prediction platform to emerge. FERN reserved Futurepedia.org for starting that project. See our Futurepedia project overview in Chapter 6 for more on this evolving project, and let us know if you’d like to help fund or work on this worthy platform for humanity.
For examples of businesses on the frontier of probabilistic prediction, see Palantir and Recorded Future, two of a new breed of web intelligence platforms that use an ensemble of social network analysis, semantic analysis, data analytics, and human intelligence techniques to visualize relationships and make forecasts on current and historical data. Just as with weather forecasting, coming versions of these tools will greatly improve our collective and statistical foresight with previously unpredictable systems.
The better our global sensors, computers, databases, software, communications systems, and collaborative foresight software get in coming years, the more we’ll see anticipation skills move into every walk of life. As media theorist Alvis Brigis proposes in his “total systems quantification” vision, our leading information services companies, like Google, are figuring out how to use a blend of unsupervised machine learning and human training to “tag the world”, and build increasingly accurate semantic and quantitative maps and correlative models of our past, present, and future. Once we have public semantic and quantitative maps and models that update in real-time, we’ll be able to do constant experiments aimed at improving their accuracy. Science, strategy, and collaboration will be more immediate than ever before.
We can see that over the next generation, social foresight will become a far more probabilistic and evidence-based activity. Because of accelerating change, significantly more of both the possibilities and the predictabilities of our future will be discoverable in advance, by some combination of web-aided human collective intelligence, statistical models, prediction platforms, and biologically-inspired computational techniques. Believe it, because it’s true.
Overdetermination: Overvaluing Probabilistic Foresight
While most of us tend to undervalue probabilistic foresight, in some contexts the opposite occurs. Some future thinkers overvalue looking for probable futures, to the neglect of both possible and preferable futures. Such thinkers will often also presume that the future is more deterministic, or fundamentally and precisely predictable, than it actually is. This cognitive bias is called overdetermination. It involves seeing only developmental processes, while ignoring the equally or more important evolutionary factors (see the 95/5 Rule). Again, see Emotional-Cognitive Biases in Chapter 4 for others.
Overdetermination is common with founders of some political, economic, social, or religious movements. We call such individuals “Fanatics” or “Utopians”. We see it with overconfident CEOs, afflicted with a “God Complex”, who have had fantastic successes in one area, and then falsely assume everything will work the same way in a new domain, or that their organizational future must be as it has been in the past. When strategy groups make far too many simplifying assumptions about their systems, or imagine conditions being predictable much further than they are, we call it “overplanning”. Nassim Taleb in Antifragile (2012) calls it a “Fragility Complex”, and the people who engage it in “Fragilistas.” In neural network theory, we call it “overtraining”, the brittleness that emerges when neural networks (in computers or in minds) get too much of a specific type of training data, when the links between their nodes get too precise, and they falsely assume the future must be just like the past. In aging, we can call it “Old Mind Syndrome”, the rigid and increasingly brittle views of the future that can afflict us, if we aren’t careful to keep learning new things, and stay open to all the ways we are wrong. Whatever we call it, we must guard against it.
Not seeing the predictable future has led many companies into inevitable and eventually catastrophic failures. Think of all the companies, like Blockbuster or Kodak, that had years to change, but never did. But overdetermination, though it tends to afflict a smaller group of people, those with particularly large egos, advanced experience, or advanced age, has also caused some of our costliest personal, organizational, and societal failures. Both extremes must be guarded against to do good foresight work.
Ignoring the predictable destinations ahead of us is seductive, as it seemingly frees us up to focus entirely on envisioning and creating the futures we want. But for some, seeing everything as more predictable than it actually is even more seductive. When we ignore what is likely to come irrespective of our personal desires, many poor decisions, lost opportunities, wrong turns, needless waste and risk invariably occur. The same happens when we assume the future is more predictable than it actually is. When we seek too strongly to impose our vision on the world without understanding its predictable constraints, trends, patterns and cycles, we have a prescription for disaster.
Good leaders recognize that guiding the future is a continual negotiation between what we want and what our universal and planetary environment is delivering, and will allow. We’ll say more about the need for an adaptive balance between possible and probable futures thinking in Chapter 3.