---
title: How to Measure Anything in Project Management
tags:
  - authorship/other
  - type/media/book
authors:
  - Douglas W. Hubbard
  - Dr. Alexander Budzier
  - Andreas Bang Leed
---
# How to Measure Anything in Project Management

## Foreword

## Preface

## Acknowledgments

## About the Authors

## Chapter 1 --- A World-scale Risk and a World-scale Opportunity

### The Size of Projects

### The Size of Project Problems

### Efforts to Fix Projects: The Emergence of Project Management

### A Path Forward: The Meta-Project

### Notes

## Chapter 2 --- A Measurement Primer for Project Management

### The Concept of Measurement

#### A Definition of Measurement

#### Definition

#### Measurement and Probabilities for Practical Decision-making

#### Are Scales Really Measurements?

### The Object of Measurement

#### What Do You See When You See More of It?

#### Why Do You Care?

### The Methods of Measurement

#### Statistical Significance: What's the Significance?

#### Small Samples Tell You More Than You Think

#### The Rule of Five

### Other Sources of Measurement Aversion

#### The Cost Objection

#### Measurements Change What Is Being Measured

#### Statistics Can Prove Anything

#### Ethical Objections to Measurement

### Notes

## Chapter 3 --- How We Know What Works

### Skepticism for Project Managers

#### The Analysis Placebo

#### The Problem of Feedback and Learning

### How to Test Methods

#### Controlled Experiments and Component Testing

#### Evaluating Sources

### The Performance of Quantitative Methods

#### Experts Versus Algorithms

#### The <span class="class_s9dd">Exsupero Ursus</span> Fallacy: Algorithm Aversion

#### Potential Reasons for <span class="class_s9dd">Exsupero Ursus</span>

### Improving the Human Expert

#### Calibrating the Expert

#### The Expert Consistency Component

#### Collaboration on Estimates

#### The Decomposition Component

### A Summary of Research on Other Project Planning and Management Methods

#### Reference Class Forecasting

#### Various Project Management Methods

#### The Performance of Monte Carlo Simulations

### Notes

## Chapter 4 --- The Project Decision Model: The Reason for Measurements

### Two Types of Project Measurements

#### Proto-purpose Discovery Measurements

#### Decision-driven Measurements

#### Unproductive Incentives vs. Measurements

### Decisions Before: Thinking Slow

#### Exploration vs. Exploitation

#### Tracking the Outside World

#### Choosing How to Run the Project

### How Models Indicate What to Measure

#### The Expected Value of Information: A Simple Introduction

#### The Measurement Inversion: Measuring the Wrong Things

#### The Value of Imperfect Measurements

### An Aspirational Model

#### The Rise of Digital Twins

#### Digital Twins in Project Management

#### A Practical Path Forward

### Notes

## Chapter 5 --- Project Uncertainty and Risk: A Primer

### Basic Concepts and Definitions

#### Uncertainty as a Probability Distribution

#### Risk: A Special Case of Uncertainty

#### Definitions for Uncertainty, Risk, and Their Measurements

### The Problem with Current Methods

#### Why Risk "Scores" Don't Work

#### How the Risk Matrix Makes Scores Worse

### A Quantitative Risk Model: Starting <span class="class_s9dd">Very</span> Simple

#### The One-for-One Substitution

#### Monte Carlo Mechanics: A Brief Introduction

### Supporting Decisions

#### A Return on Mitigation

#### How Much Risk Do You Tolerate?

#### Risk Versus Return: The Powerful Theory of Utility

### Simple Tools for Measuring Uncertainty and Risk

#### A First Estimate of a Discrete Probability

#### A First Estimate of a Continuous Probability

### Final Clarifications

#### Case Examples for What Probability Means

#### Uncertainty Versus Risk Versus Opportunity

#### Epistemic Versus Aleatory Uncertainty

#### Even More Ordinal Scales

#### Risk as Governance or Compliance

#### The Problem of "Black Swans"

#### Some Items That Aren't Really Risks

### More Improvements to Come

### Notes

## Chapter 6 --- Calibrated Subjective Probability Estimates

### Introduction to Subjective Probability

#### Two Extremes of Subjective Confidence

### Calibration Exercise

#### The Calibration Exercises

#### Evaluating Performance and Typical Results

### Improving Calibration

#### The Equivalent Bet

#### More Techniques

#### More Advanced Calibration Topics to Come

### The Effects of Calibration

### Conceptual Obstacles to Calibration

#### Conflating Uncertainty with Knowing Nothing

#### Hypotheses That Contradict the Data

#### Objections Based on the Philosophical Debate in Statistics

### Notes

## Chapter 7 --- Cost and Schedule Measurements

### The Big Plan Versus Iteration: Meta-measurements of Common Estimation Methods

### Top-down Estimations: Reference Class Forecasting

### Bottom-up Forecasting with Monte Carlo

#### A Deterministic View of Tasks

#### Probability Distributions for Project Tasks

#### Correlations

#### Multiple Prerequisites and Stochastic Critical Paths

#### Parade of Trades

### Comparing Top-Down and Bottom-Up: Case Examples

#### The Swedish Nuclear Waste Program

#### High-speed Rail

### How to Improve the Models

#### The Granularity of the Monte Carlo Model

#### Distributions and Biases

#### Correlations

#### Improving the RCF with Monte Carlo

### Notes

## Chapter 8 --- Betting on Benefits

### Meta-Measurements of Benefits

#### How Much Should Benefits Be to Justify a Project?

#### Why This May Be Optimistic

#### Why Measuring Benefits Is Rare

### Fermi Decompositions for Benefits

#### Introduction to Fermi

#### Some Example Decompositions

### Monetizing Benefits

#### Forecasts of Monetary Impacts

#### Preferences

#### Quantifying Preferences

#### The Use of Scores and Multiple Objectives

#### An Example of Challenging Benefit Measurement: Biodiversity

### Measuring What Matters in Projects

#### A (Slightly) More Realistic Information Value Calculation

#### The High Information Values for Projects

#### Getting Started on Measuring What Matters

### Considering Risk and Return

#### A Risk Neutral Decision-maker for Projects

#### Adding Utility Theory to Projects

#### Some Alternatives Within Utility Math

#### Are Executives Too Risk Averse for Projects?

#### The (Apparent) Utility Paradox

### A Framework and Its Consequences

#### Findings from Quantitative Analysis of Past Projects

#### How and When, Not Just Whether

#### Benefits Are Not Just for Project Approval Decisions

### Notes

## Chapter 9 --- Measuring Progress

### The Progress Problem

#### Simple Progress, Simple Interventions

### Earned Value Management

#### EVM Basics

#### The XRL Example

#### Recovery vs. Performance

#### Forecasting with EVM

### Progress in Information Projects

#### Waterfall

#### Agile and Measurement in Other Software Development Methods

#### Summarizing Software Metric Difficulties

### Four Stories and Lessons

#### Interfaces in a Global Bank Transformation

#### An Energy Project Front End

#### Construction Constraints

#### Testing as Software Checkpoints

#### Lessons

### The Remaining Project Simulation

#### Conditional Reference Class Forecasting (CRCF)

#### The Bottom-Up Simulation for the Remaining Project

#### Further Considerations for the RPA

### Notes

## Chapter 10 --- More Measurement Methods Made Easy

### Intuition for the Habitually Scientific

#### A Jelly Bean Example

#### A Little Probability Theory

#### Consequences of Probability Theory

#### Myths Exposed by Probability Theory

#### Significant Points About Statistical Significance

#### Basic Sampling Methods

#### The "Mathless" Table for Medians

#### Estimating a Population Proportion

### Project Cancellation Rates as a Function of Duration

#### Measuring Population Size

### Measuring Some Things by Knowing Other Things

#### Controlled Experiments

#### Regression

#### More Advanced Methods of Regression and Classification

### Estimating the Whole Distribution

### Summarizing Methods

#### Brainstorming a Measurement Approach

#### Data Gathering Methods

### A Review of Methods in This Chapter

### Notes on Surveys

### Notes

## Chapter 11 --- The Meta-Project: Implementing Better Project Measurements

### Start with the End in Mind: The Continuous Improvement Process

#### Measure What Matters

#### (Real) Skepticism and Meta-measurements

#### Measuring and Forecasting the Outside World

#### AI: The Most Important Project Ecosystem Measurement?

#### More Thinking, Fewer Projects, Bigger Wins

### Start Your Meta-Project

#### Examples of Meta-Projects Deliverables: Continuous Improvement

#### Develop an Initial Team

#### Assess the Current State of the Project Portfolio

#### Considerations for the Meta-Project Plan

#### The Pilot Project

#### Scaling to the Final Deliverable

### Organizational Challenges

#### Resistance to Change

#### Addressing Organizational Objections to Measurement

#### The Politics of Measurement

### Notes

## Chapter 12 --- A Call to Action for the Industry

### Call to Action for Project Software Vendors

#### Put Decisions at the Center

#### Deal in Uncertainties

#### Build the User-buyer-builder Federation

#### Be the Vendor That Measures Its Performance

#### Be Forward-Looking

### Call to Action for the Standard-Setting Bodies

### Call to Action for Consultants, Researchers, and Advisory Firms

### Big Future Projects

#### A Mars Mission

#### Stopping Hurricanes

#### The Meta-Project

### Notes

## Appendix 1 --- Analysis of Survey Responses on Project Management Practices

### Introduction and Data Overview

#### Success Metrics: Cost and Schedule Overrun Ratios

### Overview of Project Management Practices Reported in the Survey

#### Project Management Methodologies

#### Cost and Schedule Estimation Methods

#### Uncertainty and Risk Assessment Tools

#### Certifications

### Results

#### Project Management Methodologies

#### Cost and Schedule Estimation Methods

#### Uncertainty and Risk Assessment Tools

#### Certifications

#### Interpreting the (Mostly) Statistically Insignificant Results

## Appendix 2 --- Reference Class Data on Project Cost, Schedule, and Benefit Overruns

### Relevance of the Data and Reference Class Forecasting

### Using Historical Data to Improve Estimates -- An Example

### Notes

## Appendix 3 --- Selected Distributions

### Uniform

### Beta

### Beta PERT

### Triangular

### Binary

### Normal

### Lognormal

### Power Law

### Truncated Power Law

### Quantile-parameterized

### Gamma Poisson

### Stochastic Information Packet

## Appendix 4 --- Chapter 6 Calibration Question Answers

### Appendix Answers to Confidence Interval Questions

### Answers to True/False Questions

## Appendix 5 --- Measuring Biodiversity

### The Benefits of Biodiversity

### Variables to Measure for Biodiversity

### Notes
