Files
zmVault/hubbard_2020_failure.md
T

6.2 KiB

id, aliases, title, tags, authors, publisher, type, year
id aliases title tags authors publisher type year
The Failure of Risk Management: Why It's Broken and How to Fix It, Second Edition
authorship/other
exclude-from-word-count
status/incomplete
topic/risk
type/media
Douglas W. Hubbard John Wiley & Sons, Inc. book 2020

The Failure of Risk Management: Why It's Broken and How to Fix It, Second Edition

%% This note, with the exception of comments like this one (reserved for notes on transcription) consists only of content from the text. For commentary see the companion the-failure-of-risk-management. %%

Part One: An Introduction To The Crisis

Chapter 1: Healthy Skepticism For Risk Management

A "Common Mode Failure"

What Failure Means

Scope And Objectives Of This Book

Notes

Chapter 2: A Summary Of The Current State Of Risk Management

A Short And Entirely-Too-Superficial History Of Risk

Current State Of Risk Management In The Organization

Current Risks And How They Are Assessed

Notes

Chapter 3: How Do We Know What Works?

Anecdote: The Risk Of Outsourcing Drug Manufacturing

Why It's Hard To Know What Works

An Assessment Of Self-Assessments

Potential Objective Evaluations Of Risk Management

What We May Find

Notes

Chapter 4: Getting Started: A Simple Straw Man Quantitative Model

A Simple One-For-One Substitution

The Expert As The Instrument

A Quick Overview Of "Uncertainty Math"

Establishing Risk Tolerance

Supporting The Decision: A Return On Mitigation

Making The Straw Man Better

Note

Part Two: Why It's Broken

Chapter 5: The "Four Horsemen" Of Risk Management: Some (Mostly) Sincere Attempts To Prevent An Apocalypse

Actuaries

War Quants: How World War II Changed Risk Analysis Forever

Economists

Management Consulting: How A Power Tie And A Good Pitch Changed Risk Management

Comparing The Horsemen

Major Risk Management Problems To Be Addressed

Notes

Chapter 6: An Ivory Tower Of Babel: Fixing The Confusion About Risk

The Frank Knight Definition

Knight's Influence In Finance And Project Management

A Construction Engineering Definition

Risk As Expected Loss

Defining Risk Tolerance

Defining Probability

Enriching The Lexicon

Notes

Chapter 7: The Limits Of Expert Knowledge: Why We Don't Know What We Think We Know About Uncertainty

The Right Stuff: How A Group Of Psychologists Might Save Risk Analysis

Mental Math: Why We Shouldn't Trust The Numbers In Our Heads

"Catastrophic" Overconfidence

The Mind Of "Aces": Possible Causes And Consequences Of Overconfidence

Inconsistencies And Artifacts: What Shouldn't Matter Does

Answers To Calibration Tests

Notes

A Few Examples Of Scores And Matrices

Does That Come In "Medium"?: Why Ambiguity Does Not Offset Uncertainty

Unintended Effects Of Scales: What You Don't Know Can Hurt You

Different But Similar-Sounding Methods And Similar But Different-Sounding Methods

Notes

Chapter 9: Bears, Swans And Other Obstacles To Improved Risk Management

Algorithm Aversion And A Key Fallacy

Algorithms Versus Experts: Generalizing The Findings

A Note About Black Swans

Major Mathematical Misconceptions

We're Special: The Belief That Risk Analysis Might Work, But Not Here

Notes

Chapter 10: Where Even The Quants Go Wrong: Common And Fundamental Errors In Quantitative Models

A Survey Of Analysts Using Monte Carlos

The Risk Paradox

Financial Models And The Shape Of Disaster: Why Normal Isn't So Normal

Following Your Inner Cow: The Problem With Correlations

The Measurement Inversion

Is Monte Carlo Too Complicated?

Notes

Part Three: How To Fix It

Chapter 11: Starting With What Works

Speak The Language

Getting Your Probabilities Calibrated

Using Data For Initial Benchmarks

Checking The Substitution

Simple Risk Management

Notes

Chapter 12: Improving The Model

Empirical Inputs

Adding Detail To The Model

Advanced Methods For Improving Expert's Subjective Estimates

Other Monte Carlo Tools

Self-Examinations For Modelers

Notes

Chapter 13: The Risk Community: Intra- And Extra-Organizational Issues Of Risk Management

Getting Organized

Managing The Model

Incentives For A Calibrated Culture

Extraorganizational Issues: Solutions Beyond Your Office Building

Growing the Profession

Of all the professions in risk management, that of the actuary is the only one that is actually a legally recognized profession. Becoming an actuary requires a demonstration of proficiency through several standardized tests. It also means adopting a code of professional ethics enforced by some licensing body. When actuaries sign their name to the Statement of Actuarial Opinion of an insurance company, they put their license on the line. As with doctors and lawyers, if they lose their license, they cannot just get another job next door. The industry of modelers of uncertainties outside of insurance could benefit greatly from this level of professional standards.

Standards organizations, government affiliated and otherwise, have always been a key part of what makes a profession a profession. But standards organizations such as PMI, NIST, and others are all guilty of explicitly promoting the ineffectual methods previously debunked. The scoring methods developed by these institutions should be disposed of altogether. These organizations should stay out of the business of designing risk analysis methods until they begin to involve people with quantitative decision analysis backgrounds in their standards-development process. Professionals should take charge of the direction their profession evolves by insisting the standards move in this direction.

Practical Observations From Trustmark

Final Thoughts On Quantitative Models And Better Decisions

Notes

Appendix: Additional Calibration Tests And Answers