562 lines
11 KiB
Markdown
562 lines
11 KiB
Markdown
---
|
|
title: How to Measure Anything in Project Management
|
|
tags:
|
|
- authorship/other
|
|
- type/media/book
|
|
- exclude-from-word-count
|
|
author: Douglas W. Hubbard & Dr. Alexander Budzier & 3
|
|
---
|
|
# How to Measure Anything in Project Management
|
|
|
|
## Foreword
|
|
|
|
## Preface
|
|
|
|
## Acknowledgments
|
|
|
|
## About the Authors
|
|
|
|
## Chapter 1 --- A World-scale Risk and a World-scale Opportunity
|
|
|
|
### The Size of Projects
|
|
|
|
### The Size of Project Problems
|
|
|
|
### Efforts to Fix Projects: The Emergence of Project Management
|
|
|
|
### A Path Forward: The Meta-Project
|
|
|
|
### Notes
|
|
|
|
## Chapter 2 --- A Measurement Primer for Project Management
|
|
|
|
### The Concept of Measurement
|
|
|
|
#### A Definition of Measurement
|
|
|
|
#### Definition
|
|
|
|
#### Measurement and Probabilities for Practical Decision-making
|
|
|
|
#### Are Scales Really Measurements?
|
|
|
|
### The Object of Measurement
|
|
|
|
#### What Do You See When You See More of It?
|
|
|
|
#### Why Do You Care?
|
|
|
|
### The Methods of Measurement
|
|
|
|
#### Statistical Significance: What's the Significance?
|
|
|
|
#### Small Samples Tell You More Than You Think
|
|
|
|
#### The Rule of Five
|
|
|
|
### Other Sources of Measurement Aversion
|
|
|
|
#### The Cost Objection
|
|
|
|
#### Measurements Change What Is Being Measured
|
|
|
|
#### Statistics Can Prove Anything
|
|
|
|
#### Ethical Objections to Measurement
|
|
|
|
### Notes
|
|
|
|
## Chapter 3 --- How We Know What Works
|
|
|
|
### Skepticism for Project Managers
|
|
|
|
#### The Analysis Placebo
|
|
|
|
#### The Problem of Feedback and Learning
|
|
|
|
### How to Test Methods
|
|
|
|
#### Controlled Experiments and Component Testing
|
|
|
|
#### Evaluating Sources
|
|
|
|
### The Performance of Quantitative Methods
|
|
|
|
#### Experts Versus Algorithms
|
|
|
|
#### The <span class="class_s9dd">Exsupero Ursus</span> Fallacy: Algorithm Aversion
|
|
|
|
#### Potential Reasons for <span class="class_s9dd">Exsupero Ursus</span>
|
|
|
|
### Improving the Human Expert
|
|
|
|
#### Calibrating the Expert
|
|
|
|
#### The Expert Consistency Component
|
|
|
|
#### Collaboration on Estimates
|
|
|
|
#### The Decomposition Component
|
|
|
|
### A Summary of Research on Other Project Planning and Management Methods
|
|
|
|
#### Reference Class Forecasting
|
|
|
|
#### Various Project Management Methods
|
|
|
|
#### The Performance of Monte Carlo Simulations
|
|
|
|
### Notes
|
|
|
|
## Chapter 4 --- The Project Decision Model: The Reason for Measurements
|
|
|
|
### Two Types of Project Measurements
|
|
|
|
#### Proto-purpose Discovery Measurements
|
|
|
|
#### Decision-driven Measurements
|
|
|
|
#### Unproductive Incentives vs. Measurements
|
|
|
|
### Decisions Before: Thinking Slow
|
|
|
|
#### Exploration vs. Exploitation
|
|
|
|
#### Tracking the Outside World
|
|
|
|
#### Choosing How to Run the Project
|
|
|
|
### How Models Indicate What to Measure
|
|
|
|
#### The Expected Value of Information: A Simple Introduction
|
|
|
|
#### The Measurement Inversion: Measuring the Wrong Things
|
|
|
|
#### The Value of Imperfect Measurements
|
|
|
|
### An Aspirational Model
|
|
|
|
#### The Rise of Digital Twins
|
|
|
|
#### Digital Twins in Project Management
|
|
|
|
#### A Practical Path Forward
|
|
|
|
### Notes
|
|
|
|
## Chapter 5 --- Project Uncertainty and Risk: A Primer
|
|
|
|
### Basic Concepts and Definitions
|
|
|
|
#### Uncertainty as a Probability Distribution
|
|
|
|
#### Risk: A Special Case of Uncertainty
|
|
|
|
#### Definitions for Uncertainty, Risk, and Their Measurements
|
|
|
|
### The Problem with Current Methods
|
|
|
|
#### Why Risk "Scores" Don't Work
|
|
|
|
#### How the Risk Matrix Makes Scores Worse
|
|
|
|
### A Quantitative Risk Model: Starting <span class="class_s9dd">Very</span> Simple
|
|
|
|
#### The One-for-One Substitution
|
|
|
|
#### Monte Carlo Mechanics: A Brief Introduction
|
|
|
|
### Supporting Decisions
|
|
|
|
#### A Return on Mitigation
|
|
|
|
#### How Much Risk Do You Tolerate?
|
|
|
|
#### Risk Versus Return: The Powerful Theory of Utility
|
|
|
|
### Simple Tools for Measuring Uncertainty and Risk
|
|
|
|
#### A First Estimate of a Discrete Probability
|
|
|
|
#### A First Estimate of a Continuous Probability
|
|
|
|
### Final Clarifications
|
|
|
|
#### Case Examples for What Probability Means
|
|
|
|
#### Uncertainty Versus Risk Versus Opportunity
|
|
|
|
#### Epistemic Versus Aleatory Uncertainty
|
|
|
|
#### Even More Ordinal Scales
|
|
|
|
#### Risk as Governance or Compliance
|
|
|
|
#### The Problem of "Black Swans"
|
|
|
|
#### Some Items That Aren't Really Risks
|
|
|
|
### More Improvements to Come
|
|
|
|
### Notes
|
|
|
|
## Chapter 6 --- Calibrated Subjective Probability Estimates
|
|
|
|
### Introduction to Subjective Probability
|
|
|
|
#### Two Extremes of Subjective Confidence
|
|
|
|
### Calibration Exercise
|
|
|
|
#### The Calibration Exercises
|
|
|
|
#### Evaluating Performance and Typical Results
|
|
|
|
### Improving Calibration
|
|
|
|
#### The Equivalent Bet
|
|
|
|
#### More Techniques
|
|
|
|
#### More Advanced Calibration Topics to Come
|
|
|
|
### The Effects of Calibration
|
|
|
|
### Conceptual Obstacles to Calibration
|
|
|
|
#### Conflating Uncertainty with Knowing Nothing
|
|
|
|
#### Hypotheses That Contradict the Data
|
|
|
|
#### Objections Based on the Philosophical Debate in Statistics
|
|
|
|
### Notes
|
|
|
|
## Chapter 7 --- Cost and Schedule Measurements
|
|
|
|
### The Big Plan Versus Iteration: Meta-measurements of Common Estimation Methods
|
|
|
|
### Top-down Estimations: Reference Class Forecasting
|
|
|
|
### Bottom-up Forecasting with Monte Carlo
|
|
|
|
#### A Deterministic View of Tasks
|
|
|
|
#### Probability Distributions for Project Tasks
|
|
|
|
#### Correlations
|
|
|
|
#### Multiple Prerequisites and Stochastic Critical Paths
|
|
|
|
#### Parade of Trades
|
|
|
|
### Comparing Top-Down and Bottom-Up: Case Examples
|
|
|
|
#### The Swedish Nuclear Waste Program
|
|
|
|
#### High-speed Rail
|
|
|
|
### How to Improve the Models
|
|
|
|
#### The Granularity of the Monte Carlo Model
|
|
|
|
#### Distributions and Biases
|
|
|
|
#### Correlations
|
|
|
|
#### Improving the RCF with Monte Carlo
|
|
|
|
### Notes
|
|
|
|
## Chapter 8 --- Betting on Benefits
|
|
|
|
### Meta-Measurements of Benefits
|
|
|
|
#### How Much Should Benefits Be to Justify a Project?
|
|
|
|
#### Why This May Be Optimistic
|
|
|
|
#### Why Measuring Benefits Is Rare
|
|
|
|
### Fermi Decompositions for Benefits
|
|
|
|
#### Introduction to Fermi
|
|
|
|
#### Some Example Decompositions
|
|
|
|
### Monetizing Benefits
|
|
|
|
#### Forecasts of Monetary Impacts
|
|
|
|
#### Preferences
|
|
|
|
#### Quantifying Preferences
|
|
|
|
#### The Use of Scores and Multiple Objectives
|
|
|
|
#### An Example of Challenging Benefit Measurement: Biodiversity
|
|
|
|
### Measuring What Matters in Projects
|
|
|
|
#### A (Slightly) More Realistic Information Value Calculation
|
|
|
|
#### The High Information Values for Projects
|
|
|
|
#### Getting Started on Measuring What Matters
|
|
|
|
### Considering Risk and Return
|
|
|
|
#### A Risk Neutral Decision-maker for Projects
|
|
|
|
#### Adding Utility Theory to Projects
|
|
|
|
#### Some Alternatives Within Utility Math
|
|
|
|
#### Are Executives Too Risk Averse for Projects?
|
|
|
|
#### The (Apparent) Utility Paradox
|
|
|
|
### A Framework and Its Consequences
|
|
|
|
#### Findings from Quantitative Analysis of Past Projects
|
|
|
|
#### How and When, Not Just Whether
|
|
|
|
#### Benefits Are Not Just for Project Approval Decisions
|
|
|
|
### Notes
|
|
|
|
## Chapter 9 --- Measuring Progress
|
|
|
|
### The Progress Problem
|
|
|
|
#### Simple Progress, Simple Interventions
|
|
|
|
### Earned Value Management
|
|
|
|
#### EVM Basics
|
|
|
|
#### The XRL Example
|
|
|
|
#### Recovery vs. Performance
|
|
|
|
#### Forecasting with EVM
|
|
|
|
### Progress in Information Projects
|
|
|
|
#### Waterfall
|
|
|
|
#### Agile and Measurement in Other Software Development Methods
|
|
|
|
#### Summarizing Software Metric Difficulties
|
|
|
|
### Four Stories and Lessons
|
|
|
|
#### Interfaces in a Global Bank Transformation
|
|
|
|
#### An Energy Project Front End
|
|
|
|
#### Construction Constraints
|
|
|
|
#### Testing as Software Checkpoints
|
|
|
|
#### Lessons
|
|
|
|
### The Remaining Project Simulation
|
|
|
|
#### Conditional Reference Class Forecasting (CRCF)
|
|
|
|
#### The Bottom-Up Simulation for the Remaining Project
|
|
|
|
#### Further Considerations for the RPA
|
|
|
|
### Notes
|
|
|
|
## Chapter 10 --- More Measurement Methods Made Easy
|
|
|
|
### Intuition for the Habitually Scientific
|
|
|
|
#### A Jelly Bean Example
|
|
|
|
#### A Little Probability Theory
|
|
|
|
#### Consequences of Probability Theory
|
|
|
|
#### Myths Exposed by Probability Theory
|
|
|
|
#### Significant Points About Statistical Significance
|
|
|
|
#### Basic Sampling Methods
|
|
|
|
#### The "Mathless" Table for Medians
|
|
|
|
#### Estimating a Population Proportion
|
|
|
|
### Project Cancellation Rates as a Function of Duration
|
|
|
|
#### Measuring Population Size
|
|
|
|
### Measuring Some Things by Knowing Other Things
|
|
|
|
#### Controlled Experiments
|
|
|
|
#### Regression
|
|
|
|
#### More Advanced Methods of Regression and Classification
|
|
|
|
### Estimating the Whole Distribution
|
|
|
|
### Summarizing Methods
|
|
|
|
#### Brainstorming a Measurement Approach
|
|
|
|
#### Data Gathering Methods
|
|
|
|
### A Review of Methods in This Chapter
|
|
|
|
### Notes on Surveys
|
|
|
|
### Notes
|
|
|
|
## Chapter 11 --- The Meta-Project: Implementing Better Project Measurements
|
|
|
|
### Start with the End in Mind: The Continuous Improvement Process
|
|
|
|
#### Measure What Matters
|
|
|
|
#### (Real) Skepticism and Meta-measurements
|
|
|
|
#### Measuring and Forecasting the Outside World
|
|
|
|
#### AI: The Most Important Project Ecosystem Measurement?
|
|
|
|
#### More Thinking, Fewer Projects, Bigger Wins
|
|
|
|
### Start Your Meta-Project
|
|
|
|
#### Examples of Meta-Projects Deliverables: Continuous Improvement
|
|
|
|
#### Develop an Initial Team
|
|
|
|
#### Assess the Current State of the Project Portfolio
|
|
|
|
#### Considerations for the Meta-Project Plan
|
|
|
|
#### The Pilot Project
|
|
|
|
#### Scaling to the Final Deliverable
|
|
|
|
### Organizational Challenges
|
|
|
|
#### Resistance to Change
|
|
|
|
#### Addressing Organizational Objections to Measurement
|
|
|
|
#### The Politics of Measurement
|
|
|
|
### Notes
|
|
|
|
## Chapter 12 --- A Call to Action for the Industry
|
|
|
|
### Call to Action for Project Software Vendors
|
|
|
|
#### Put Decisions at the Center
|
|
|
|
#### Deal in Uncertainties
|
|
|
|
#### Build the User-buyer-builder Federation
|
|
|
|
#### Be the Vendor That Measures Its Performance
|
|
|
|
#### Be Forward-Looking
|
|
|
|
### Call to Action for the Standard-Setting Bodies
|
|
|
|
### Call to Action for Consultants, Researchers, and Advisory Firms
|
|
|
|
### Big Future Projects
|
|
|
|
#### A Mars Mission
|
|
|
|
#### Stopping Hurricanes
|
|
|
|
#### The Meta-Project
|
|
|
|
### Notes
|
|
|
|
## Appendix 1 --- Analysis of Survey Responses on Project Management Practices
|
|
|
|
### Introduction and Data Overview
|
|
|
|
#### Success Metrics: Cost and Schedule Overrun Ratios
|
|
|
|
### Overview of Project Management Practices Reported in the Survey
|
|
|
|
#### Project Management Methodologies
|
|
|
|
#### Cost and Schedule Estimation Methods
|
|
|
|
#### Uncertainty and Risk Assessment Tools
|
|
|
|
#### Certifications
|
|
|
|
### Results
|
|
|
|
#### Project Management Methodologies
|
|
|
|
#### Cost and Schedule Estimation Methods
|
|
|
|
#### Uncertainty and Risk Assessment Tools
|
|
|
|
#### Certifications
|
|
|
|
#### Interpreting the (Mostly) Statistically Insignificant Results
|
|
|
|
## Appendix 2 --- Reference Class Data on Project Cost, Schedule, and Benefit Overruns
|
|
|
|
### Relevance of the Data and Reference Class Forecasting
|
|
|
|
### Using Historical Data to Improve Estimates -- An Example
|
|
|
|
### Notes
|
|
|
|
## Appendix 3 --- Selected Distributions
|
|
|
|
### Uniform
|
|
|
|
### Beta
|
|
|
|
### Beta PERT
|
|
|
|
### Triangular
|
|
|
|
### Binary
|
|
|
|
### Normal
|
|
|
|
### Lognormal
|
|
|
|
### Power Law
|
|
|
|
### Truncated Power Law
|
|
|
|
### Quantile-parameterized
|
|
|
|
### Gamma Poisson
|
|
|
|
### Stochastic Information Packet
|
|
|
|
## Appendix 4 --- Chapter 6 Calibration Question Answers
|
|
|
|
### Appendix Answers to Confidence Interval Questions
|
|
|
|
### Answers to True/False Questions
|
|
|
|
## Appendix 5 --- Measuring Biodiversity
|
|
|
|
### The Benefits of Biodiversity
|
|
|
|
### Variables to Measure for Biodiversity
|
|
|
|
### Notes
|