zmVault/hubbard_2025_project-management.md

title, tags, author

title	tags	author
How to Measure Anything in Project Management	authorship/other type/media/book exclude-from-word-count	Douglas W. Hubbard & Dr. Alexander Budzier & 3

How to Measure Anything in Project Management

Foreword

Preface

Acknowledgments

About the Authors

Chapter 1 --- A World-scale Risk and a World-scale Opportunity

The Size of Projects

The Size of Project Problems

Efforts to Fix Projects: The Emergence of Project Management

A Path Forward: The Meta-Project

Notes

Chapter 2 --- A Measurement Primer for Project Management

The Concept of Measurement

A Definition of Measurement

Definition

Measurement and Probabilities for Practical Decision-making

Are Scales Really Measurements?

The Object of Measurement

What Do You See When You See More of It?

Why Do You Care?

The Methods of Measurement

Statistical Significance: What's the Significance?

Small Samples Tell You More Than You Think

The Rule of Five

Other Sources of Measurement Aversion

The Cost Objection

Measurements Change What Is Being Measured

Statistics Can Prove Anything

Ethical Objections to Measurement

Notes

Chapter 3 --- How We Know What Works

Skepticism for Project Managers

The Analysis Placebo

The Problem of Feedback and Learning

How to Test Methods

Controlled Experiments and Component Testing

Evaluating Sources

The Performance of Quantitative Methods

Experts Versus Algorithms

The Exsupero Ursus Fallacy: Algorithm Aversion

Potential Reasons for Exsupero Ursus

Improving the Human Expert

Calibrating the Expert

The Expert Consistency Component

Collaboration on Estimates

The Decomposition Component

A Summary of Research on Other Project Planning and Management Methods

Reference Class Forecasting

Various Project Management Methods

The Performance of Monte Carlo Simulations

Notes

Chapter 4 --- The Project Decision Model: The Reason for Measurements

Two Types of Project Measurements

Proto-purpose Discovery Measurements

Decision-driven Measurements

Unproductive Incentives vs. Measurements

Decisions Before: Thinking Slow

Exploration vs. Exploitation

Tracking the Outside World

Choosing How to Run the Project

How Models Indicate What to Measure

The Expected Value of Information: A Simple Introduction

The Measurement Inversion: Measuring the Wrong Things

The Value of Imperfect Measurements

An Aspirational Model

The Rise of Digital Twins

Digital Twins in Project Management

A Practical Path Forward

Notes

Chapter 5 --- Project Uncertainty and Risk: A Primer

Basic Concepts and Definitions

Uncertainty as a Probability Distribution

Risk: A Special Case of Uncertainty

Definitions for Uncertainty, Risk, and Their Measurements

The Problem with Current Methods

Why Risk "Scores" Don't Work

How the Risk Matrix Makes Scores Worse

A Quantitative Risk Model: Starting Very Simple

The One-for-One Substitution

Monte Carlo Mechanics: A Brief Introduction

Supporting Decisions

A Return on Mitigation

How Much Risk Do You Tolerate?

Risk Versus Return: The Powerful Theory of Utility

Simple Tools for Measuring Uncertainty and Risk

A First Estimate of a Discrete Probability

A First Estimate of a Continuous Probability

Final Clarifications

Case Examples for What Probability Means

Uncertainty Versus Risk Versus Opportunity

Epistemic Versus Aleatory Uncertainty

Even More Ordinal Scales

Risk as Governance or Compliance

The Problem of "Black Swans"

Some Items That Aren't Really Risks

More Improvements to Come

Notes

Chapter 6 --- Calibrated Subjective Probability Estimates

Introduction to Subjective Probability

Two Extremes of Subjective Confidence

Calibration Exercise

The Calibration Exercises

Evaluating Performance and Typical Results

Improving Calibration

The Equivalent Bet

More Techniques

More Advanced Calibration Topics to Come

The Effects of Calibration

Conceptual Obstacles to Calibration

Conflating Uncertainty with Knowing Nothing

Hypotheses That Contradict the Data

Objections Based on the Philosophical Debate in Statistics

Notes

Chapter 7 --- Cost and Schedule Measurements

The Big Plan Versus Iteration: Meta-measurements of Common Estimation Methods

Top-down Estimations: Reference Class Forecasting

Bottom-up Forecasting with Monte Carlo

A Deterministic View of Tasks

Probability Distributions for Project Tasks

Correlations

Multiple Prerequisites and Stochastic Critical Paths

Parade of Trades

Comparing Top-Down and Bottom-Up: Case Examples

The Swedish Nuclear Waste Program

High-speed Rail

How to Improve the Models

The Granularity of the Monte Carlo Model

Distributions and Biases

Correlations

Improving the RCF with Monte Carlo

Notes

Chapter 8 --- Betting on Benefits

Meta-Measurements of Benefits

How Much Should Benefits Be to Justify a Project?

Why This May Be Optimistic

Why Measuring Benefits Is Rare

Fermi Decompositions for Benefits

Introduction to Fermi

Some Example Decompositions

Monetizing Benefits

Forecasts of Monetary Impacts

Preferences

Quantifying Preferences

The Use of Scores and Multiple Objectives

An Example of Challenging Benefit Measurement: Biodiversity

Measuring What Matters in Projects

A (Slightly) More Realistic Information Value Calculation

The High Information Values for Projects

Getting Started on Measuring What Matters

Considering Risk and Return

A Risk Neutral Decision-maker for Projects

Adding Utility Theory to Projects

Some Alternatives Within Utility Math

Are Executives Too Risk Averse for Projects?

The (Apparent) Utility Paradox

A Framework and Its Consequences

Findings from Quantitative Analysis of Past Projects

How and When, Not Just Whether

Benefits Are Not Just for Project Approval Decisions

Notes

Chapter 9 --- Measuring Progress

The Progress Problem

Simple Progress, Simple Interventions

Earned Value Management

EVM Basics

The XRL Example

Recovery vs. Performance

Forecasting with EVM

Progress in Information Projects

Waterfall

Agile and Measurement in Other Software Development Methods

Summarizing Software Metric Difficulties

Four Stories and Lessons

Interfaces in a Global Bank Transformation

An Energy Project Front End

Construction Constraints

Testing as Software Checkpoints

Lessons

The Remaining Project Simulation

Conditional Reference Class Forecasting (CRCF)

The Bottom-Up Simulation for the Remaining Project

Further Considerations for the RPA

Notes

Chapter 10 --- More Measurement Methods Made Easy

Intuition for the Habitually Scientific

A Jelly Bean Example

A Little Probability Theory

Consequences of Probability Theory

Myths Exposed by Probability Theory

Significant Points About Statistical Significance

Basic Sampling Methods

The "Mathless" Table for Medians

Estimating a Population Proportion

Project Cancellation Rates as a Function of Duration

Measuring Population Size

Measuring Some Things by Knowing Other Things

Controlled Experiments

Regression

More Advanced Methods of Regression and Classification

Estimating the Whole Distribution

Summarizing Methods

Brainstorming a Measurement Approach

Data Gathering Methods

A Review of Methods in This Chapter

Notes on Surveys

Notes

Chapter 11 --- The Meta-Project: Implementing Better Project Measurements

Start with the End in Mind: The Continuous Improvement Process

Measure What Matters

(Real) Skepticism and Meta-measurements

Measuring and Forecasting the Outside World

AI: The Most Important Project Ecosystem Measurement?

More Thinking, Fewer Projects, Bigger Wins

Start Your Meta-Project

Examples of Meta-Projects Deliverables: Continuous Improvement

Develop an Initial Team

Assess the Current State of the Project Portfolio

Considerations for the Meta-Project Plan

The Pilot Project

Scaling to the Final Deliverable

Organizational Challenges

Resistance to Change

Addressing Organizational Objections to Measurement

The Politics of Measurement

Notes

Chapter 12 --- A Call to Action for the Industry

Call to Action for Project Software Vendors

Put Decisions at the Center

Deal in Uncertainties

Build the User-buyer-builder Federation

Be the Vendor That Measures Its Performance

Be Forward-Looking

Call to Action for the Standard-Setting Bodies

Call to Action for Consultants, Researchers, and Advisory Firms

Big Future Projects

A Mars Mission

Stopping Hurricanes

The Meta-Project

Notes

Appendix 1 --- Analysis of Survey Responses on Project Management Practices

Introduction and Data Overview

Success Metrics: Cost and Schedule Overrun Ratios

Overview of Project Management Practices Reported in the Survey

Project Management Methodologies

Cost and Schedule Estimation Methods

Uncertainty and Risk Assessment Tools

Certifications

Results

Project Management Methodologies

Cost and Schedule Estimation Methods

Uncertainty and Risk Assessment Tools

Certifications

Interpreting the (Mostly) Statistically Insignificant Results

Appendix 2 --- Reference Class Data on Project Cost, Schedule, and Benefit Overruns

Relevance of the Data and Reference Class Forecasting

Using Historical Data to Improve Estimates -- An Example

Notes

Appendix 3 --- Selected Distributions

Uniform

Beta

Beta PERT

Triangular

Binary

Normal

Lognormal

Power Law

Truncated Power Law

Quantile-parameterized

Gamma Poisson

Stochastic Information Packet

Appendix 4 --- Chapter 6 Calibration Question Answers

Appendix Answers to Confidence Interval Questions

Answers to True/False Questions

Appendix 5 --- Measuring Biodiversity

The Benefits of Biodiversity

Variables to Measure for Biodiversity

Notes