Chapter 1. Introduction – Our Emerging Foresight Field

Keeping Intelligent Optimism

How do we get better at balancing our own and our clients positive and negative ways of feeling and thinking about the future? To address this very important practice challenge, we must visit the literature on optimism and pessimism and understand the concept of intelligent optimism.

Dilip Jeste, professor of psychology and neuroscience at UCSD, describes four key types of optimism and pessimism that all foresight practioners should recognize in ourselves and others:

  1. Dispositional optimism and pessimism.
    How our personality (nature & nurture) guides our future expectations (positive or negative).
  1. Strategic optimism.
    Focusing on upsides and opportunities when faced with potential risks and stresses.
  1. Defensive pessimism.
    Lowering expectations and anticipating bad outcomes when faced with risks and stresses.
  1. Explanatory optimism and pessimism.
    The ways we explain good and bad outcomes, and our ability to influence them.
Click the image for Wall Street Journal infographic. Where do you fit in to these four types?

Click the image for Wall Street Journal infographic. Where do you fit in to these four types?

Some of us are born and raised as dispositional optimists, others as dispositional pessimists. You may think that dispositional pessimism is somehow bad, but as Jeste explains, both orientations to the future are quite useful and adaptive in various contexts. There is a folk saying that “pessimists are more accurate, but optimists get more done.” People who are naturally more optimistic tend to be good executives, creatives, and entrepreneurs, trying new things and setting stretch goals for their teams. People who are naturally more pessimistic make good deputy managers, operations, and security leaders, anticipating and heading off bad outcomes, ensuring compliance, and managing risks. These are of course generalizations, but they ring true in our personal lives and organizations. Sometimes what we need is more accuracy and caution, and sometimes we need more risktaking and opportunity seeking.

Strategic optimism, sometimes called “ambition” or “opportunism”, is our emotional and cognitive responses that raise our future expectations and focus us on upsides and opportunities when faced with potential risks and stresses. Excitement, joy, hope, courage, and curiosity are among the dominant emotions and thoughts involved in strategic optimism. All of us have these positive feelings and thoughts, but natural pessimists exhibit them in much more limited ways.

Defensive pessimism, also called “caution” or “realism”, is our emotional and cognitive responses that lower our future expectations, and focus us on potential downsides and problems. Fear, distrust, caution, and worry are among the dominant emotions involved in defensive pessimism. Again, we all have these negative feelings and thoughts, but natural optimists exhibit them in much more limited ways. Clearly, both types are useful and they balance each other, creating a more cognitively diverse world.

Whether we want to be feeling and thinking as a strategic optimist or a defensive pessimist varies by context. When you are in danger, or are responsible for security, defense, or things that may go quickly or expensively wrong, and thus need high reliability or accountability, your defensive pessimism should be in command. But if you are amid a field of opportunity, if you have a safety net, if experimentation is relatively safe, and when the consequences of failure aren’t life threatening, you want your strategic optimism to be driving most of the time. In such situations, a positive outlook on the future will best motivate yourself, your team, and your client.

How we explain events and outcomes, our explanatory optimism and pessimism, is a product of our natural optimism or pessimism and our current feelings and thoughts, which are some a blend of opportunistic and cautious in relation to the future. Sometimes our explanations are evidence-based and adaptive, and at other times they are not. So how do we get our positive and negative feelings, thoughts, and explanations to match better with reality?

We can begin by understanding our dispositional biases, and asking whether they are adaptive or not. An often-made point is that as a species, humanity seems to still have an evolutionary pessimism, fear, or caution bias, making us naturally more pessimistic, fearful and cautious than optimistic, joyful and curious in our orientation to the world. Behavioral economist Daniel Kahneman introduces this bias, and several others in Thinking Fast and Slow (2013), a book that describes our two main neural information processing systems. What Kahneman calls System 1 is fast, older, mostly unconscious, and emotional, and System 2 is slow, newer, more conscious, and rational.

Because System 1, our emotion, is set to be both faster and stronger than System 2, our thinking, emotions will quickly flood our psyches and temporarily suspend our rationality. Because we are biased so that the negative emotions are stronger than the positive ones, fear, shame, depression, and disgust will shut down our rationality much faster and longer than than joy, hope, excitement, and curiosity will strengthen our rationality. Negative emotions are simply stronger. Psychologist Daniel Goleman, in his book Emotional Intelligence, 1995 coined the term the amydgala hijack to describe a negative emotional response that is far out of proportion in its effect on our thinking and behavior to the actual degree of risk presented by the stimulus.

Dystopias Over Protopias

We must also realize that in System 1, we have much stronger set of evolutionary reasons to pay both faster and stronger attention to our negative emotions than to our positive emotions.  On average, far more of us are ready to quickly and broadly imagine dystopias (stories of regressive futures). We only slowly and reluctantly think about protopias (stories  progress-related, positive futures). It’s much easier to stand against something than for something. We’ll readily join protest marches (against something, which requires only criticism) but only occasionally join cause marches (for something, which requires an idea of progress).

Because negative emotions such as fear, danger, and caution motivate us both faster and more strongly than positive emotions like excitement, joy, and curiosity, we rarely feel the same degree of drama in stories of positive futures as we do in negative futures. Dystopian dramas outnumber and outsell protopian ones by such a large degree, it’s hard for most of us to cite good examples of the latter. Even when each offers us the same degree of absolute change, the negative potential outcome always motivates us emotionally at a deeper level than the positive one.

We likely evolved this bias, this ability to be more strongly hijacked by negative stimuli than positive ones, for good evolutionary reasons. Until very recently, we all lived in a far more dangerous and primitive world, one ruled by violence, disease, and sudden death. It was only in the late Enlightenment, the 1700’s, could the average person in the most developed parts of the world finally notice unmistakeable signs of progress over their own lifetimes. Only then did our public outlook on the possibilities for a better future start to change. So we haven’t had enough evolutionary time for our genetic proclivities to pessimism to change. As a result, biases related to this, such as loss or risk aversion bias are common in society.

There’s more to the pessimism story. A study by Lang et.al. in 2013 found that adults who have healthy doses of defensive pessimism live at least 10% longer and healthier lives. They take more precautions and avoid unnecessary risks in life, rather than being foolhardy in their health habits and behaviors. This small survival benefit may be one of the evolutionary reasons our pessimism bias persists. Also, the older we get, the more pessimistic and realistic we get about our own internal abilities, as they start to slow down. Unfortunately however, many aging people confuse their own naturally declining internal abilities with the state of the world, which gets better faster every year, as we’ll see in Chapter 2.

So as our planet continues to age, our cultural pessimism bias has grown with it, and is increasingly out of step with what the world is actually doing. Only 27% of the world’s population is under 25 today, and that fraction will keep going down the more developed the world gets. We might call this aging of our planet a longevity pessimism bias, and it’s another real factor shaping our expected future.

Furthermore, while almost all cultures are more fearful and pessimistic toward the future than they should be, some are significantly more fearful and fatalistic than others due to their recent centuries of history, as in Russia, some former Soviet republics, and parts of Africa and Asia. And of course all cultures have their “hot button” issues, that revive past traumas and reopen unhealed wounds.

These natural conditions make it easy and tempting for our media and opportunistic leaders to manipulate us for their own ends. They often play on our fears and negativity, creating media pessimism bias and opportunistic pessimism biases, each of which, if we are subject to them, push us even further from a healthy view of the world.

Any poor souls unfortunate enough to live in a filter bubble of network news are constantly being assaulted with scare stories that blow all kinds of minor issues into major dramas. Those who watch only news that fits their biases quickly become scared and dismissive of the “other”, meaning anyone who doesn’t agree with their world view. Some of our leaders play to these fears, scaring us with extreme stories, and never pairing those stories with a sincere presentation of the latest data, which usually tell a far more nuanced and moderate view.

Glassner’s The Culture of Fear, 1999, Gore’s The Assault on Reason, 2008,  and Gardner’s The Science of Fear, 2009 explore how pervasive and calculated the selling and use of fear has become in our society over the last century, and the way fear erodes our personal agency, responsibility, rationality and evidence-based approaches to reality. Messages of fear and negativity play well in a pessimism-biased culture, especially the older it gets. Our media know that “If a story bleeds, it leads.”

Because of these pessimism biases we tend focus on “headlines over trendlines”, and assume our headlines are good samples of reality (they usually aren’t). That leads us to develop poor models for how the world actually works. As Nick Kristof notes in a 2015 New York Times editorial on media pessimism bias, two-thirds of Americans today believe that the proportion of the world population living in extreme poverty has almost doubled over the last 20 years. In fact extreme poverty has done the opposite, falling by more than half over that period, falling from 35 to 14% between 1993 and 2011, per the World Bank.

All of these pessimism biases help us understand the observation by Stephen Pinker in The Better Angels of Our Nature, 2012, that even as our world has accelerated its development and become vastly richer and safer in recent centuries and decades, we adults are as or more pessimistic and worried about the future than ever before. Some of this new worry is adaptive, stemming from our increasingly fine sense of social justice, and our increasingly high value on human life as the world continues to develop. But a good deal of this worry is irrational, and must be addressed.

So if we carry an pessimism bias in our genes, one that grows with age, and if the media and our leaders often exploit and amplify this bias, it is no wonder it is more fashionable to be cynical and disbelieving than to be expectant and upbeat about the future in most intellectual communities.

In sum, we can see that pessimism biases exact a steep social cost that we are just beginning to account for. They cause us to live in ways where we are more easily manipulated by fear, have poor world models, and are less open, free, healthy, and willing to risk, innovate, and experiment. Fearful and overcautious parents become overprotective helicopter parents, always “hovering” nearby to rescue their children from the first sign of minor defeat or disappointment. As Hanna Rosin reminds us in “The Overprotected Kid,” The Atlantic, Apr 2014, and Bronson and Merrryman argue in NurtureShock, 2011, children of these helicopter parents grow up disempowered, less creative, unwilling to take risks, and ready to quit as soon as things gets too difficult and don’t fit their naively overoptimistic expectations.

There’s more to the optimism story too. As Talia Sharot explains in The Optimism Bias: A Tour of the Irrationally Positive Brain, 2012, on an individual level, all of us tend to distort past and present events when we are reaching for optimist visions of the future. We forget our past and present difficulties, focus on the few positive bits that support our optimistic expectations, and imagine better and different outcomes. To some degree, these positive stories are adaptive, as Shelly Taylor explores in Positive Illusions: Creative Self-Deception and the Healthy Mind, 1991. Seeing better future outcomes can motivate us to strive harder for them. But the more we forget the negatives and limits of present realities, the more our optimistic thinking can become self-deception. Socially, too much optimism can slide into groupthink and other mass delusions, like irrational exuberance in the stock market, manias where we mimic others engaging in the same distortion, until the bubble finally bursts.

At the same time, it’s easy for any of us, for brief periods, and often in reaction to an escape from a recent bout of grinding fear and pessimism, to get carried away with our optimism, to forget the real difficulties we, our organizations, or the world have had in the past and present, and to turn our positive illusions into fantasies and manias. That usually doesn’t last long, and it isn’t very adaptive. Bipolar mood disorder and its milder cousin cyclothymia are of course both examples of cycling between too much optimism and pessimism in our moods. What we can call bipolar foresight disorder is a common affliction in a minority of futurists, in which they cycle between impractical optimism for their personal or organizational futures, then fall into serious depressions as their uncritical visions fail to realize. Both extremes are destructive, and they can’t stay on the productive ground between them for long.

Ridley (2011)

Clearly, good foresighters and futurists need to be both strategic optimists (“opportunists”) and defensive pessimists (“realists”) at the same time. We need to strike a dynamic balance between these two poles, and find the blend of these two feeling and thinking styles that is the most adaptive for each issue, time, and context. We can give that dynamically adaptive blend a name: intelligent optimism. For more, see Rohan Robert and Lara Matossian’s Intelligent Optimism community, Frank Robinson’s The Case for Rational Optimism (2009), Matt Ridley’s The Rational Optimist (2011), and Diamandis and Kotler’s Abundance (2014). Ridley’s book suffers from taking a too-uncritical look at capitalism, but each offer excellent introductions to the general mindset we think all evidence-based futurists should cultivate, in a world of accelerating change. A mindset that is generally optimistic, but also quick to identify dangers and shortcomings of our models

It is naïve and dangerous to be overly optimistic on the outcomes of many of our specific problems, in the short run. But it is foolhardy and ignorant not to be rationally optimistic on our prospects in general, over the longer run. In the long run we are not all dead, as John Maynard Keynes famously said. In the long run we’re all headed to a profoundly better place, as we’ll argue in Chapter 2. We get there with intelligent optimism, a step at a time.

The Two to One (2:1) Ratio: Ideal Feeling, Thinking and Learning

Let’s now ask a provocative question: Is there a roughly ideal ratio of strategic optimism to defensive pessimism in our emotions and thinking, across a broad range of contexts? In other words, an average set point we should strive for to be intelligent optimists? Dilip Jeste guesses a typical healthy personal ratio, if we are not immediately in danger, may be 70% optimistic and 30% pessimistic, or roughly a 2:1 ratio. If Jeste is right in his guess, that means that if you are in a safe environment, you should have roughly two hope and opportunity feelings and thoughts for every one fear and caution feeling and thought about the issues presently in front of you. While this ratio will surely differ from person to person and context to context, a 2:1 ratio sounds like a reasonable average set point for good foresight to occur.

In a curious convergence, computer adaptive testing experts have discovered that an adaptive testing platform that serve you test questions with the conceptual difficulty adaptively calibrated so that you to get 70% of the questions correct, and 30% incorrect, is ideally the ideal set point to maximize things like marginal learning per hour, as measured in post-engagement assessments, motivation to learn (average hours on the platform) and self-assessed user satisfaction with the platform. In other words, crafting your “tests,” both in the computer and in life, such that you succeed with two of your challenges for every one at which you fail is usually going to be your best training and learning regimen. Add in a quick feedback cycle for the “failure”, and this may also be a rough prescription for staying in the “flow” state. We certainly need a lot more evidence for or against these claims, but they are a nice place to start.

So what is the typical optimism to pessimism ratio in our advanced democracies today? As I know of no formal study, I’ve thought up a rough first estimate for this variable, the ratio of protopian and hope-driven fiction to dystopian and fear-driven fiction in bookstores and movies. Quick research and personal experience tells me this ratio is perhaps one to four (1:4), in North America at least.

So if Jeste’s ratio is a good first estimate for an adaptive set point for intelligent optimism, and if this 1:4 protopian:dystopian (or hope:fear) entertainment ratio is a good rough measure of North America’s cultural pessimism bias, this means our cultural outlook on the future is eight times worse (1:4 instead of 2:1) than is adaptive in average developed-world environments.

An eightfold cultural pessimism bias with respect to our perceived futures over our actual futures, if true, is no small matter. It means we’ll be far less evidence-based and growth and progress oriented than we would otherwise be. As we’ll see in Chapter 2, our cultural pessimism bias makes it particularly hard for forsight professionals and our clients to see and take advantage of accelerating positive change. Apparently we just aren’t genetically equipped to accurately perceive or believe we live in a world of exponential improvement.

Yet all around us accelerating complexification, intelligence, interdependence, and immunity from disaster are occuring. All of humanity’s historical catastrophes have directly catalyzed the further acceleration of global complexity, intelligence, interdependence, and immunity. The future we have been evolved to imagine is not the future that is developing all around us. We are biased to not see what is actually happening, and we will eventually get beyond that blindness, one way or another. The path we take is ours to choose, but not the destination.

Of all the challenges of navigating human psychology to produce better foresight, this cultural pessimism bias may be the most important. Hilary Tindle’s book Up: How Positive Outlook Can Transform Our Health and Aging, 2013, and Michelle Gielan’s Broadcasting Happiness: The Science of Igniting and Sustaining Positive Change, 2015, are two great surveys of concrete ways to fight this bias in ourselves, our clients, and culture.

Looking ahead, we’ll need to integrate the kinds of insights Tindle and Gielan offer us into all our digital environments in coming years. Either we can do it for ourselves, or the machines will do it for us in a few decades, as their intelligence grows. Again, one way or another, I’m convinced this problem will be fixed, as it isn’t adaptive in the long run.

We won’t go further into these fascinating topics at this point. But books like Wright’s Nonzero, 2000, Robinson’s The Case for Rational Optimism, 2009, Ridley’s The Rational Optimist, 2011, Reese’s Infinite Progress, 2013 and others to be discussed later all set the record straight. Read these books if you are not yet yourself an intelligent optimist, and realize that, for reasons of biology and environment, the natural first response of most of our clients, when contemplating the future and its changes, will be some variation of fear, doubt, and pessimism first, and curiosity, courage, and optimism as weaker and slower secondary responses. This is our heritage and current condition, but it is not who we are becoming. When human nature isn’t adaptive, evolution tells us that nature must eventually change.

Share your Feedback

Better Wording? References? Data? Images? Quotes? Mistakes?

Thanks for helping us make the Guide the best intro to foresight on the web.