zmVault/hubbard_2020_failure.md

id, aliases, title, tags, authors, publisher, type, year

id	aliases	title	tags	authors	publisher	type	year
		The Failure of Risk Management: Why It's Broken and How to Fix It, Second Edition	authorship/other exclude-from-word-count status/incomplete topic/risk type/media	Douglas W. Hubbard	John Wiley & Sons, Inc.	book	2020

The Failure of Risk Management: Why It's Broken and How to Fix It, Second Edition

%% This note, with the exception of comments like this one (reserved for notes on transcription) consists only of content from the text. For commentary see the companion the-failure-of-risk-management. %%

Part One: An Introduction To The Crisis

Chapter 1: Healthy Skepticism For Risk Management

A "Common Mode Failure"

Key Definitions: Risk Management And Some Related Terms

What Failure Means

Scope And Objectives Of This Book

Notes

Chapter 2: A Summary Of The Current State Of Risk Management

A Short And Entirely-Too-Superficial History Of Risk

Current State Of Risk Management In The Organization

Current Risks And How They Are Assessed

Notes

Chapter 3: How Do We Know What Works?

Anecdote: The Risk Of Outsourcing Drug Manufacturing

Why It's Hard To Know What Works

An Assessment Of Self-Assessments

Potential Objective Evaluations Of Risk Management

What We May Find

Notes

Chapter 4: Getting Started: A Simple Straw Man Quantitative Model

A Simple One-For-One Substitution

The Expert As The Instrument

A Quick Overview Of "Uncertainty Math"

Establishing Risk Tolerance

Supporting The Decision: A Return On Mitigation

Making The Straw Man Better

Note

Part Two: Why It's Broken

Chapter 5: The "Four Horsemen" Of Risk Management: Some (Mostly) Sincere Attempts To Prevent An Apocalypse

Actuaries

War Quants: How World War II Changed Risk Analysis Forever

Economists

Management Consulting: How A Power Tie And A Good Pitch Changed Risk Management

Comparing The Horsemen

Major Risk Management Problems To Be Addressed

Notes

Chapter 6: An Ivory Tower Of Babel: Fixing The Confusion About Risk

The Frank Knight Definition

Knight's Influence In Finance And Project Management

A Construction Engineering Definition

Risk As Expected Loss

Defining Risk Tolerance

Defining Probability

Enriching The Lexicon

Notes

Chapter 7: The Limits Of Expert Knowledge: Why We Don't Know What We Think We Know About Uncertainty

The Right Stuff: How A Group Of Psychologists Might Save Risk Analysis

Mental Math: Why We Shouldn't Trust The Numbers In Our Heads

"Catastrophic" Overconfidence

The Mind Of "Aces": Possible Causes And Consequences Of Overconfidence

Inconsistencies And Artifacts: What Shouldn't Matter Does

Answers To Calibration Tests

Notes

Chapter 8: Worse Than Useless: The Most Popular Risk Assessment Method And Why It Doesn't Work

A Few Examples Of Scores And Matrices

Does That Come In "Medium"?: Why Ambiguity Does Not Offset Uncertainty

Unintended Effects Of Scales: What You Don't Know Can Hurt You

Different But Similar-Sounding Methods And Similar But Different-Sounding Methods

Notes

Chapter 9: Bears, Swans And Other Obstacles To Improved Risk Management

Algorithm Aversion And A Key Fallacy

Algorithms Versus Experts: Generalizing The Findings

A Note About Black Swans

Major Mathematical Misconceptions

We're Special: The Belief That Risk Analysis Might Work, But Not Here

Notes

Chapter 10: Where Even The Quants Go Wrong: Common And Fundamental Errors In Quantitative Models

A Survey Of Analysts Using Monte Carlos

The Risk Paradox

Financial Models And The Shape Of Disaster: Why Normal Isn't So Normal

Following Your Inner Cow: The Problem With Correlations

The Measurement Inversion

Is Monte Carlo Too Complicated?

Notes

Part Three: How To Fix It

Chapter 11: Starting With What Works

Speak The Language

Getting Your Probabilities Calibrated

Using Data For Initial Benchmarks

Checking The Substitution

Simple Risk Management

Notes

Chapter 12: Improving The Model

Empirical Inputs

Adding Detail To The Model

Advanced Methods For Improving Expert's Subjective Estimates

Other Monte Carlo Tools

Self-Examinations For Modelers

Notes

Chapter 13: The Risk Community: Intra- And Extra-Organizational Issues Of Risk Management

Getting Organized

Managing The Model

Incentives For A Calibrated Culture

Extraorganizational Issues: Solutions Beyond Your Office Building

Growing the Profession

Of all the professions in risk management, that of the actuary is the only one that is actually a legally recognized profession. Becoming an actuary requires a demonstration of proficiency through several standardized tests. It also means adopting a code of professional ethics enforced by some licensing body. When actuaries sign their name to the Statement of Actuarial Opinion of an insurance company, they put their license on the line. As with doctors and lawyers, if they lose their license, they cannot just get another job next door. The industry of modelers of uncertainties outside of insurance could benefit greatly from this level of professional standards.

Standards organizations, government affiliated and otherwise, have always been a key part of what makes a profession a profession. But standards organizations such as PMI, NIST, and others are all guilty of explicitly promoting the ineffectual methods previously debunked. The scoring methods developed by these institutions should be disposed of altogether. These organizations should stay out of the business of designing risk analysis methods until they begin to involve people with quantitative decision analysis backgrounds in their standards-development process. Professionals should take charge of the direction their profession evolves by insisting the standards move in this direction.

Practical Observations From Trustmark

Final Thoughts On Quantitative Models And Better Decisions

Notes

Appendix: Additional Calibration Tests And Answers

Calibration Test for Ranges: A

1. How many feet tall is the Hoover Dam?
2. How many inches long is a $20 bill?
3. What percentage of aluminum is recycled in the United States?
4. When was Elvis Presley born?
5. What percentage of the atmosphere is oxygen by weight?
6. What is the latitude of New Orleans? [Hint: Latitude is 0 degrees at the equator and 90 degrees at the North Pole.]
7. In 1913, the United States military owned how many airplanes?
8. The first European printing press was invented in what year?
9. What percentage of all electricity consumed in US households was used by kitchen appliances in 2001?
10. How many miles tall is Mount Everest?
11. How long is Iraq's border with Iran in kilometers?
12. How many miles long is the Nile?
13. In what year was Harvard founded?
14. What is the wingspan (in feet) of a Boeing 747 jumbo jet?
15. How many soldiers were in a Roman legion?
16. What is the average temperature of the abyssal zone (where the oceans are more than 6,500 feet deep) in degrees F?
17. How many feet long is the Space Shuttle Orbiter (excluding the external tank)?
18. In what year did Jules Verne publish 20,000 Leagues Under the Sea?
19. How wide is the goal in field hockey (in feet)?
20. The Roman Coliseum held how many spectators?

Answers to Calibration Test for Ranges: A

1. 726 feet
2. 63/16ths (6.1417) inches
3. 45 percent
4. 1935
5. 21 percent
6. 29.95
7. 23
8. 1450
9. 26.7 percent
10. 5.5 miles
11. 1,458 kilometers
12. 4,160 miles
13. 1636
14. 196 feet
15. 6,000
16. 39 F
17. 122 feet
18. 1870
19. 12 feet
20. 50,000

Calibration Test for Ranges: B

1. The first probe to land on Mars, Viking 1, landed there in what year?
2. How old was the youngest person to fly into space?
3. How many meters tall is the Sears Tower?
4. What was the maximum altitude of the Breitling Orbiter 3, the first balloon to circumnavigate the globe, in miles?
5. On average, what percentage of the total software development project effort is spent in design?
6. How many people were permanently evacuated after the Chernobyl nuclear power plant accident?
7. How many feet long were the largest airships?
8. How many miles is the flying distance from San Francisco to Honolulu?
9. The fastest bird, the falcon, can fly at a speed of how many miles per hour in a dive?
10. In what year was the double helix structure of DNA discovered?
11. How many yards wide is a football field?
12. What was the percentage growth in internet hosts from 1996 to 1997?
13. How many calories are in 8 ounces of orange juice?
14. How fast would you have to travel at sea level to break the sound barrier (in mph)?
15. How many years was Nelson Mandela in prison?
16. What is the average daily calorie intake in developed countries?
17. In 1994, how many nations were members of the United Nations?
18. The Audubon Society was formed in the United States in what year?
19. How many feet high is the world's highest waterfall (Angel Falls, Venezuela)?
20. How deep beneath the sea was the Titanic found (in miles)?

Answers to Calibration Test for Ranges: B

1. 1976
2. 26
3. 443 meters
4. 6.9 miles
5. 20 percent
6. 350,000
7. 803 feet
8. 2,394 miles
9. 200 mph
10. 1953
11. 53.3 yards
12. 70 percent
13. 120
14. 760 mph
15. 27
16. 3,300 calories
17. 184
18. 1905
19. 3,212 feet
20. 3.36 miles

Calibration Test for Binary: A

1. The Lincoln Highway was the first paved road in the United States, and it ran from Chicago to San Francisco.
2. The Silk Road joined the two ancient kingdoms of China and Afghanistan.
3. More American homes have microwaves than telephones.
4. Doric is an architectural term for a shape of roof.
5. The World Tourism Organization predicts that Europe will still be the most popular tourist destination in 2020.
6. Germany was the second country to develop atomic weapons.
7. A hockey puck will fit in a golf hole.
8. The Sioux were one of the Plains Indian tribes.
9. To a physicist, plasma is a type of rock.
10. The Hundred Years' War was actually more than a century long.
11. Most of the fresh water on Earth is in the polar ice caps.
12. The Academy Awards ("Oscars") began over a century ago.
13. There are fewer than two hundred billionaires in the world.
14. In Excel, ^ means "take to the power of."
15. The average annual salary of airline captains is over $150,000.
16. By 1997, Bill Gates was worth more than $10 billion.
17. Cannons were used in European warfare by the eleventh century.
18. Anchorage is the capital of Alaska.
19. Washington, Jefferson, Lincoln, and Grant are the four presidents whose heads are sculpted into Mount Rushmore.
20. John Wiley & Sons is not the largest book publisher.

Answers for Calibration Test Binary: A

1. FALSE
2. FALSE
3. FALSE
4. FALSE
5. TRUE
6. FALSE
7. TRUE
8. TRUE
9. FALSE
10. TRUE
11. TRUE
12. FALSE
13. FALSE
14. TRUE
15. FALSE
16. TRUE
17. FALSE
18. FALSE
19. FALSE
20. TRUE

Calibration Test for Binary: B

1. Jupiter's "Great Red Spot" is larger than Earth.
2. The Brooklyn Dodgers' name was short for "trolley car dodgers."
3. Hypersonic is faster than subsonic.
4. A polygon is three-dimensional and a polyhedron is two-dimensional.
5. A 1-watt electric motor produces 1 horsepower.
6. Chicago is more populous than Boston.
7. In 2005, WalMart sales dropped below $100 billion. > 80% 90% 100%
8. Post-it Notes were invented by 3M.
9. Alfred Nobel, whose fortune endows the Nobel Peace Prize, made his fortune in oil and explosives.
10. A BTU is a measure of heat.
11. The winner of the first Indianapolis 500 clocked an average speed of under 100 mph.
12. Microsoft has more employees than IBM.
13. Romania borders Hungary.
14. Idaho is larger (in area) than Iraq.
15. Casablanca is on the African continent.
16. The first manmade plastic was invented in the nineteenth century.
17. A chamois is an alpine animal.
18. The base of a pyramid is in the shape of a square.
19. Stonehenge is located on the main British island.
20. Computer processors double in power every three months or less.

Answers for Calibration Test Binary: B

1. TRUE
2. TRUE
3. TRUE
4. FALSE
5. FALSE
6. TRUE
7. FALSE
8. TRUE
9. TRUE
10. TRUE
11. TRUE
12. FALSE
13. TRUE
14. FALSE
15. TRUE
16. TRUE
17. TRUE
18. TRUE
19. TRUE
20. FALSE