Risks and Riddles

The Soviet Union was a puzzle. Al Qaeda is a mystery. Why we need to know the difference

  • By Gregory F. Treverton
  • Smithsonian magazine, June 2007, Subscribe
 

More from Smithsonian.com


(Page 2 of 3)

Puzzle-solving is frustrated by a lack of information. Given Washington's need to find out how many warheads Moscow's missiles carried, the United States spent billions of dollars on satellites and other data-collection systems. But puzzles are relatively stable. If a critical piece is missing one day, it usually remains valuable the next.

By contrast, mysteries often grow out of too much information. Until the 9/11 hijackers actually boarded their airplanes, their plan was a mystery, the clues to which were buried in too much "noise"—too many threat scenarios. So warnings from FBI agents in Minneapolis and Phoenix went unexplored. The hijackers were able to hide in plain sight. After the attacks, they became a puzzle: it was easy to pick up their trail.

Solving puzzles is useful for detection. But framing mysteries is necessary for prevention.

That's one reason the FBI embarked on a change of mission after 9/11, from almost pure law enforcement to intelligence—from solving puzzles to framing mysteries. That change in mission requires an enormous change in organizational culture. For the puzzles of law enforcement, the measures of effectiveness are pretty clear—you can count the suspects collared and bad guys convicted. Terrorists, however, may commit but one crime, and by the time they do, it is too late. That scarcity of "collars" is a main reason why, rhetoric aside, counterterrorism was not a marquee FBI mission before 9/11.

For the mysteries of intelligence, measures of effectiveness are elusive. The goal of prevention is...nothing—an absence of attacks. But if no major terrorist attack occurs, does that represent the effectiveness of prevention, simple good luck or the fact that the threat was overstated in the first place?

That's one uncertainty we'll have to learn to live with. There are others that framing mysteries can help us understand.

No matter how much patients may seek the clarity of a puzzle, healthcare, too, is largely a mystery. The goal of medicine, like that of counterterrorism, is an absence—of illness and disease. Achieving that goal depends on many different factors. Behavior matters, but only in some complicated interaction with propensity; what is lethal obesity for one person is only a nuisance for another. Tests are imperfect predictors of illness, and treatments interact or have side effects.

While few doctors would put it this way, they act upon something that might be called Bayesian mystery framing. Bayes' theorem is a statistical technique for adjusting subjective probabilities in light of new, but inconclusive, evidence. Doctors base an initial assessment of a patient's health on propensity, as revealed by his or her medical history, and on diagnosis, determined through an examination. If the doctor's initial assessment is of a high probability of disease, he or she orders more tests, which in turn refine that probability. For chronic concerns, such as high blood pressure leading to heart disease, the initial assessment leads to a decision about whether and how to treat, followed by subsequent tests to see if the original probability of problems can be revised downward.


There's a reason millions of people try to solve crossword puzzles each day. Amid the well-ordered combat between a puzzler's mind and the blank boxes waiting to be filled, there is satisfaction along with frustration. Even when you can't find the right answer, you know it exists. Puzzles can be solved; they have answers.

But a mystery offers no such comfort. It poses a question that has no definitive answer because the answer is contingent; it depends on a future interaction of many factors, known and unknown. A mystery

cannot be answered; it can only be framed, by identifying the critical factors and applying some sense of how they have interacted in the past and might interact in the future. A mystery is an attempt to define ambiguities.

Puzzles may be more satisfying, but the world increasingly offers us mysteries. Treating them as puzzles is like trying to solve the unsolvable—an impossible challenge. But approaching them as mysteries may make us more comfortable with the uncertainties of our age.

During the cold war, much of the job of U.S. intelligence was puzzle-solving—seeking answers to questions that had answers, even if we didn't know them. How many missiles did the Soviet Union have? Where were they located? How far could they travel? How accurate were they? It made sense to approach the military strength of the Soviet Union as a puzzle—the sum of its units and weapons, and their quality.

But the collapse of the Soviet Union and the rise of terrorism changed all that. Those events upended U.S. intelligence, to the point that its major challenge now is to frame mysteries, as I learned as vice chairman of the National Intelligence Council, managing the process for producing National Intelligence Estimates.

To analysts in the Pentagon, for instance, terrorists present the ultimate asymmetric threat. But the nature of the threat is a mystery, not a puzzle. Terrorists shape themselves to our vulnerabilities, to the seams in our defenses; the threat they pose depends on us. The 9/11 hijackers, for instance, did not come to their plan of attack because they were aviation buffs. They came to it because they had identified gaps in our aviation defenses.

Whether Saddam Hussein's Iraq had nuclear or chemical weapons seemed a quintessential puzzle, and U.S. intelligence treated it that way. And got it wrong. But suppose the issue of Iraq's weapons of mass destruction had been treated not as a puzzle but as a mystery. That might have turned the exercise away from technical details and toward Saddam's thinking. It might have raised the question: Could Saddam be more afraid of his local enemies than he is of the United States? Could that lead him to boast that he had weapons he really didn't have?

Puzzle-solving is frustrated by a lack of information. Given Washington's need to find out how many warheads Moscow's missiles carried, the United States spent billions of dollars on satellites and other data-collection systems. But puzzles are relatively stable. If a critical piece is missing one day, it usually remains valuable the next.

By contrast, mysteries often grow out of too much information. Until the 9/11 hijackers actually boarded their airplanes, their plan was a mystery, the clues to which were buried in too much "noise"—too many threat scenarios. So warnings from FBI agents in Minneapolis and Phoenix went unexplored. The hijackers were able to hide in plain sight. After the attacks, they became a puzzle: it was easy to pick up their trail.

Solving puzzles is useful for detection. But framing mysteries is necessary for prevention.

That's one reason the FBI embarked on a change of mission after 9/11, from almost pure law enforcement to intelligence—from solving puzzles to framing mysteries. That change in mission requires an enormous change in organizational culture. For the puzzles of law enforcement, the measures of effectiveness are pretty clear—you can count the suspects collared and bad guys convicted. Terrorists, however, may commit but one crime, and by the time they do, it is too late. That scarcity of "collars" is a main reason why, rhetoric aside, counterterrorism was not a marquee FBI mission before 9/11.

For the mysteries of intelligence, measures of effectiveness are elusive. The goal of prevention is...nothing—an absence of attacks. But if no major terrorist attack occurs, does that represent the effectiveness of prevention, simple good luck or the fact that the threat was overstated in the first place?

That's one uncertainty we'll have to learn to live with. There are others that framing mysteries can help us understand.

No matter how much patients may seek the clarity of a puzzle, healthcare, too, is largely a mystery. The goal of medicine, like that of counterterrorism, is an absence—of illness and disease. Achieving that goal depends on many different factors. Behavior matters, but only in some complicated interaction with propensity; what is lethal obesity for one person is only a nuisance for another. Tests are imperfect predictors of illness, and treatments interact or have side effects.

While few doctors would put it this way, they act upon something that might be called Bayesian mystery framing. Bayes' theorem is a statistical technique for adjusting subjective probabilities in light of new, but inconclusive, evidence. Doctors base an initial assessment of a patient's health on propensity, as revealed by his or her medical history, and on diagnosis, determined through an examination. If the doctor's initial assessment is of a high probability of disease, he or she orders more tests, which in turn refine that probability. For chronic concerns, such as high blood pressure leading to heart disease, the initial assessment leads to a decision about whether and how to treat, followed by subsequent tests to see if the original probability of problems can be revised downward.

Most discussions about energy, as well, treat it as a puzzle: so many million barrels of proven reserves in country X, production to "peak" in country Y at a particular date and so on. From a geological point of view, the puzzle perspective makes sense: any individual drill hole or field has so much oil. Yet energy futures are a mystery, not a puzzle. How much oil a given well can produce is not the same as how much oil is there: whether it makes sense to use secondary or tertiary recovery methods after primary methods no longer suffice depends on price. And beyond a single well, the factors multiply. How fast will the global economy grow? What new energy discoveries will be made? Which alternative sources will come on line at what price?

In a recent issue of the New Yorker, writer Malcolm Gladwell made a fascinating observation about the Enron scandal. He noted that law-enforcement investigators and, later, prosecutors treated the case against former Enron CEO Jeffrey Skilling as a puzzle: Enron either lied or dissembled about its finances; either way, the public received too little information.

But a year before the scandal broke, Gladwell also noted, several financial analysts treated Enron as a mystery. Parsing the company's voluminous financial statements and quarterly findings, they realized that its vaunted profits—it claimed $111 billion in revenues in 2000—constituted a giant mystery: that is, the gains had not yet been realized, and depended on events that had not yet occurred. The company's actual cash flow was virtually nil. The analysts pursued the matter with Enron, and the company flew its accounting team to Dallas for a meeting. For the mystery-framers, the challenge was not too little information, but too much noise. A summary of a single set of Enron's complicated financial deals would have come to over a hundred thousand dense pages.

Ultimately, the mystery ended calamitously, as Enron's profit scenario failed to materialize, and the company collapsed. At that point, Enron became a puzzle, and the prosecutors' approach to solving it worked. After Skilling was convicted of fraud, he was sentenced to 24 years in prison, one of the most severe prison sentences ever given for a white-collar crime.

You might say that the puzzle-solvers succeeded, in that criminality was punished. Or you might conclude that the mystery-framers failed, in that the crime was not prevented.

Gregory F. Treverton is the director of the RAND Corporation's Center for Global Risk and Security.


Single Page « Previous 1 2 3 Next »

    Subscribe now for more of Smithsonian's coverage on history, science and nature.



 

Add New Comment


Name: (required)

Email: (required)

Comment:

Comments are moderated, and will not appear until Smithsonian.com has approved them. Smithsonian reserves the right not to post any comments that are unlawful, threatening, offensive, defamatory, invasive of a person's privacy, inappropriate, confidential or proprietary, political messages, product endorsements, or other content that might otherwise violate any laws or policies.

Comments


Advertisement


Advertisement

Follow Us





First Name
Last Name
Address 1
Address 2
City
State   Zip
Email


Travel with Smithsonian




Smithsonian Store