zmVault/hubbard_2020_failure.md

---
id:
aliases: []
title: "The Failure of Risk Management: Why It's Broken and How to Fix It, Second Edition"
tags:
  - authorship/other
  - destiny/permanent
  - exclude-from-word-count
  - status/incomplete
  - topic/risk
  - type/media/book
authors: Douglas W. Hubbard
publisher: John	Wiley & Sons, Inc.
type: book
year: 2020
---
# The Failure of Risk Management: Why It's Broken and How to Fix It, Second Edition

%%
This note, with the exception of comments like this one
(reserved for notes on transcription)
consists only of content from the text.
For commentary see the companion
[[the-failure-of-risk-management]].
%%

## Part One: An Introduction To The Crisis

### Chapter 1: Healthy Skepticism For Risk Management

#### A "Common Mode Failure"

#### Key Definitions: Risk Management And Some Related Terms

#### What Failure Means

#### Scope And Objectives Of This Book

#### Notes

### Chapter 2: A Summary Of The Current State Of Risk Management

#### A Short And Entirely-Too-Superficial History Of Risk

#### Current State Of Risk Management In The Organization

#### Current Risks And How They Are Assessed

#### Notes

### Chapter 3: How Do We Know What Works?

#### Anecdote: The Risk Of Outsourcing Drug Manufacturing

#### Why It's Hard To Know What Works

#### An Assessment Of Self-Assessments

#### Potential Objective Evaluations Of Risk Management

#### What We May Find

#### Notes

### Chapter 4: Getting Started: A Simple Straw Man Quantitative Model

#### A Simple One-For-One Substitution

#### The Expert As The Instrument

#### A Quick Overview Of "Uncertainty Math"

#### Establishing Risk Tolerance

#### Supporting The Decision: A Return On Mitigation

#### Making The Straw Man Better

#### Note

## Part Two: Why It's Broken

### Chapter 5: The "Four Horsemen" Of Risk Management: Some (Mostly) Sincere Attempts To Prevent An Apocalypse

#### Actuaries

#### War Quants: How World War II Changed Risk Analysis Forever

#### Economists

#### Management Consulting: How A Power Tie And A Good Pitch Changed Risk Management

#### Comparing The Horsemen

#### Major Risk Management Problems To Be Addressed

#### Notes

### Chapter 6: An Ivory Tower Of Babel: Fixing The Confusion About Risk

#### The Frank Knight Definition

#### Knight's Influence In Finance And Project Management

#### A Construction Engineering Definition

#### Risk As Expected Loss

#### Defining Risk Tolerance

#### Defining Probability

#### Enriching The Lexicon

#### Notes

### Chapter 7: The Limits Of Expert Knowledge: Why We Don't Know What We Think We Know About Uncertainty

#### The Right Stuff: How A Group Of Psychologists Might Save Risk Analysis

#### Mental Math: Why We Shouldn't Trust The Numbers In Our Heads

#### "Catastrophic" Overconfidence

#### The Mind Of "Aces": Possible Causes And Consequences Of Overconfidence

#### Inconsistencies And Artifacts: What Shouldn't Matter Does

#### Answers To Calibration Tests

#### Notes

### Chapter 8: Worse Than Useless: The Most Popular Risk Assessment Method And Why It Doesn't Work

#### A Few Examples Of Scores And Matrices

#### Does That Come In "Medium"?: Why Ambiguity Does Not Offset Uncertainty

#### Unintended Effects Of Scales: What You Don't Know Can Hurt You

#### Different But Similar-Sounding Methods And Similar But Different-Sounding Methods

#### Notes

### Chapter 9: Bears, Swans And Other Obstacles To Improved Risk Management

#### Algorithm Aversion And A Key Fallacy

#### Algorithms Versus Experts: Generalizing The Findings

#### A Note About Black Swans

#### Major Mathematical Misconceptions

#### We're Special: The Belief That Risk Analysis Might Work, But Not Here

#### Notes

### Chapter 10: Where Even The Quants Go Wrong: Common And Fundamental Errors In Quantitative Models

#### A Survey Of Analysts Using Monte Carlos

#### The Risk Paradox

#### Financial Models And The Shape Of Disaster: Why Normal Isn't So Normal

#### Following Your Inner Cow: The Problem With Correlations

#### The Measurement Inversion

#### Is Monte Carlo Too Complicated?

#### Notes

## Part Three: How To Fix It

### Chapter 11: Starting With What Works

#### Speak The Language

#### Getting Your Probabilities Calibrated

#### Using Data For Initial Benchmarks

#### Checking The Substitution

#### Simple Risk Management

#### Notes

### Chapter 12: Improving The Model

#### Empirical Inputs

#### Adding Detail To The Model

#### Advanced Methods For Improving Expert's Subjective Estimates

#### Other Monte Carlo Tools

#### Self-Examinations For Modelers

#### Notes

### Chapter 13: The Risk Community: Intra- And Extra-Organizational Issues Of Risk Management

#### Getting Organized

#### Managing The Model

#### Incentives For A Calibrated Culture

#### Extraorganizational Issues: Solutions Beyond Your Office Building

##### Growing the Profession

Of all the professions in risk management,
that of the actuary is the only one
that is actually a legally recognized profession.
Becoming an actuary requires a demonstration of proficiency
through several standardized tests.
It also means adopting a code of professional ethics
enforced by some licensing body.
When actuaries sign their name
to the Statement of Actuarial Opinion of an insurance company,
they put their license on the line.
As with doctors and lawyers,
if they lose their license, they cannot just get another job next door.
The industry of modelers of uncertainties outside of insurance
could benefit greatly from this level of professional standards.

Standards organizations,
government affiliated and otherwise,
have always been a key part of what makes a profession a profession.
But standards organizations such as PMI, NIST, and others
are all guilty of explicitly promoting
the ineffectual methods previously debunked.
The scoring methods developed by these institutions
should be disposed of altogether.
These organizations should stay out of the business
of designing risk analysis methods
until they begin to involve people with quantitative decision analysis backgrounds
in their standards-development process.
Professionals should take charge of the direction their profession evolves
by insisting the standards move in this direction.

#### Practical Observations From Trustmark

#### Final Thoughts On Quantitative Models And Better Decisions

#### Notes

## Appendix: Additional Calibration Tests And Answers

### Calibration Test for Ranges: A

1. How many feet tall is the Hoover Dam?
2. How many inches long is a \$20 bill?
3. What percentage of aluminum is recycled in the United States?
4. When was Elvis Presley born?
5. What percentage of the atmosphere is oxygen by weight?
6. What is the latitude of New Orleans? \[_Hint_: Latitude is 0 degrees at the equator and 90 degrees at the North Pole.\]
7. In 1913, the United States military owned how many airplanes?
8. The first European printing press was invented in what year?
9. What percentage of all electricity consumed in US households was used by kitchen appliances in 2001?
10. How many miles tall is Mount Everest?
11. How long is Iraq\'s border with Iran in kilometers?
12. How many miles long is the Nile?
13. In what year was Harvard founded?
14. What is the wingspan (in feet) of a Boeing 747 jumbo jet?
15. How many soldiers were in a Roman legion?
16. What is the average temperature of the abyssal zone (where the oceans are more than 6,500 feet deep) in degrees F?
17. How many feet long is the Space Shuttle _Orbiter_ (excluding the external tank)?
18. In what year did Jules Verne publish _20,000 Leagues Under the Sea_?
19. How wide is the goal in field hockey (in feet)?
20. The Roman Coliseum held how many spectators?

### Answers to Calibration Test for Ranges: A

1. 726 feet
2. 63/16ths (6.1417) inches
3. 45 percent
4. 1935
5. 21 percent
6. 29.95
7. 23
8. 1450
9. 26.7 percent
10. 5.5 miles
11. 1,458 kilometers
12. 4,160 miles
13. 1636
14. 196 feet
15. 6,000
16. 39 F
17. 122 feet
18. 1870
19. 12 feet
20. 50,000

### Calibration Test for Ranges: B

1. The first probe to land on Mars, _Viking 1,_ landed there in what year?
2. How old was the youngest person to fly into space?
3. How many meters tall is the Sears Tower?
4. What was the maximum altitude of the _Breitling Orbiter 3,_ the first balloon to circumnavigate the globe, in miles?
5. On average, what percentage of the total software development project effort is spent in design?
6. How many people were permanently evacuated after the Chernobyl nuclear power plant accident?
7. How many feet long were the largest airships?
8. How many miles is the flying distance from San Francisco to Honolulu?
9. The fastest bird, the falcon, can fly at a speed of how many miles per hour in a dive?
10. In what year was the double helix structure of DNA discovered?
11. How many yards wide is a football field?
12. What was the percentage growth in internet hosts from 1996 to 1997?
13. How many calories are in 8 ounces of orange juice?
14. How fast would you have to travel at sea level to break the sound barrier (in mph)?
15. How many years was Nelson Mandela in prison?
16. What is the average daily calorie intake in developed countries?
17. In 1994, how many nations were members of the United Nations?
18. The Audubon Society was formed in the United States in what year?
19. How many feet high is the world\'s highest waterfall (Angel Falls, Venezuela)?
20. How deep beneath the sea was the Titanic found (in miles)?

### Answers to Calibration Test for Ranges: B

1. 1976
2. 26
3. 443 meters
4. 6.9 miles
5. 20 percent
6. 350,000
7. 803 feet
8. 2,394 miles
9. 200 mph
10. 1953
11. 53.3 yards
12. 70 percent
13. 120
14. 760 mph
15. 27
16. 3,300 calories
17. 184
18. 1905
19. 3,212 feet
20. 3.36 miles

### Calibration Test for Binary: A

1. The Lincoln Highway was the first paved road in the United States, and it ran from Chicago to San Francisco.
2. The Silk Road joined the two ancient kingdoms of China and Afghanistan.
3. More American homes have microwaves than telephones.
4. _Doric_ is an architectural term for a shape of roof.
5. The World Tourism Organization predicts that Europe will still be the most popular tourist destination in 2020.
6. Germany was the second country to develop atomic weapons.
7. A hockey puck will fit in a golf hole.
8. The Sioux were one of the Plains Indian tribes.
9. To a physicist, _plasma_ is a type of rock.
10. The Hundred Years\' War was actually more than a century long.
11. Most of the fresh water on Earth is in the polar ice caps.
12. The Academy Awards ("Oscars") began over a century ago.
13. There are fewer than two hundred billionaires in the world.
14. In Excel, \^ means "take to the power of."
15. The average annual salary of airline captains is over \$150,000.
16. By 1997, Bill Gates was worth more than \$10 billion.
17. Cannons were used in European warfare by the eleventh century.
18. Anchorage is the capital of Alaska.
19. Washington, Jefferson, Lincoln, and Grant are the four presidents whose heads are sculpted into Mount Rushmore.
20. John Wiley & Sons is not the largest book publisher.

### Answers for Calibration Test Binary: A

1. FALSE
2. FALSE
3. FALSE
4. FALSE
5. TRUE
6. FALSE
7. TRUE
8. TRUE
9. FALSE
10. TRUE
11. TRUE
12. FALSE
13. FALSE
14. TRUE
15. FALSE
16. TRUE
17. FALSE
18. FALSE
19. FALSE
20. TRUE

### Calibration Test for Binary: B

1. Jupiter\'s "Great Red Spot" is larger than Earth.
2. The Brooklyn Dodgers\' name was short for "trolley car dodgers."
3. _Hypersonic_ is faster than _subsonic_.
4. A _polygon_ is three-dimensional and a _polyhedron_ is two-dimensional.
5. A 1-watt electric motor produces 1 horsepower.
6. Chicago is more populous than Boston.
7. In 2005, WalMart sales dropped below \$100 billion. > 80% 90% 100%
8. Post-it Notes were invented by 3M.
9. Alfred Nobel, whose fortune endows the Nobel Peace Prize, made his fortune in oil and explosives.
10. A BTU is a measure of heat.
11. The winner of the first Indianapolis 500 clocked an average speed of under 100 mph.
12. Microsoft has more employees than IBM.
13. Romania borders Hungary.
14. Idaho is larger (in area) than Iraq.
15. Casablanca is on the African continent.
16. The first manmade plastic was invented in the nineteenth century.
17. A chamois is an alpine animal.
18. The base of a pyramid is in the shape of a square.
19. Stonehenge is located on the main British island.
20. Computer processors double in power every three months or less.

### Answers for Calibration Test Binary: B

1. TRUE
2. TRUE
3. TRUE
4. FALSE
5. FALSE
6. TRUE
7. FALSE
8. TRUE
9. TRUE
10. TRUE
11. TRUE
12. FALSE
13. TRUE
14. FALSE
15. TRUE
16. TRUE
17. TRUE
18. TRUE
19. TRUE
20. FALSE