Estimation Beyond Points: Three-Point/PERT, Wideband Delphi, Work Breakdown and Reference Class Forecasting

Version note: The calculator and simulation in this post were built and run with .NET 10; the reference class forecasting part is based on Bent Flyvbjerg's 2006 paper From Nobel Prize to Project Management.

Make a Guess First

One story came out of planning poker with 8 points and a big question mark: matching rows without an ISBN by title. Selin wants to be surest about this slice of Institutional Orders in Bookstore, the online bookstore used as the shared example across this site's series, because some of the lists from Nevin Hanım's school have no ISBN at all. The team splits the "bulk order from a list" slice into nine work packages and writes a "most likely" duration in days for each. The total is 21 developer-days.

Selin's question is simple: "Shall we say 21 days?" Does adding up the most likely duration of every package give the most likely duration of the total? How likely do you think it is that all nine packages finish within their most likely duration: 50%, 20%, less? And think about this: if each package also got a "everything goes wrong" duration and you added those up, would that total be a realistic worst case?

Work Breakdown: Pieces You Can Estimate

The first step in estimating a big, uncertain piece of work is splitting it into pieces you can estimate. In project management this is the work breakdown structure, or WBS. For a software team the practical rules are: no piece should be bigger than a few days, the pieces together should cover the whole job, and the forgotten work from the first post, testing, review and deployment, should have its own line.

Splitting stories and work breakdown are not the same thing. In the Requirements Analysis series post "Story Mapping and Splitting Stories" we split stories into vertical slices that deliver value to the user. Work breakdown separates the work inside a story or slice for estimation; the pieces do not need to deliver value on their own. "Tests, review, deployment" is not a story, but it is a line in the estimate.

The Bookstore team's breakdown is: parsing the pasted list, ISBN validation and catalog lookup, matching rows without an ISBN by title, the validation result screen, the institutional discount rule, a stock check for large quantities, saving as a draft, submitting the order with a purchase order number, and finally tests, review and deployment.

Three-Point Estimates and PERT

Instead of one number, each piece gets three: optimistic (O), most likely (M) and pessimistic (P). O is the duration if everything goes well; M, in the most typical case; P, in a reasonably bad case. P is not "the end of the world"; think of it as a value exceeded roughly once in twenty attempts.

PERT, the Program Evaluation and Review Technique, was developed in 1958 by the US Navy's Special Projects Office together with Booz Allen Hamilton for the Polaris missile program. A piece's expected duration is (O + 4M + P) / 6 and its standard deviation (P − O) / 6. The formula is an approximation that assumes durations follow a beta distribution: the most likely value gets four times the weight and the span from O to P is taken as roughly six standard deviations.

When you add up pieces, the means add directly. Uncertainty does not: if the pieces are independent, the variances add and the standard deviation of the total is the square root of the summed variances. That assumes a package that goes badly is partly offset by another that goes well. This is where the important part is; we will talk shortly about why this assumption is dangerous.

Code: PERT, Simulation and the Outside View

We add the calculator to the series' console app. Next to the work packages, the same file holds the first estimate and actual duration of Bookstore's last eight epics for the reference class:

namespace Bookstore.Planning;

public sealed record WorkPackage(string Name, double Optimistic, double MostLikely, double Pessimistic)
{
    public double PertMean => (Optimistic + 4 * MostLikely + Pessimistic) / 6;
    public double PertSigma => (Pessimistic - Optimistic) / 6;

    // Sample from a triangular distribution (inverse CDF)
    public double Sample(Random rng)
    {
        double u = rng.NextDouble();
        (double a, double m, double b) = (Optimistic, MostLikely, Pessimistic);
        double split = (m - a) / (b - a);
        return u < split
            ? a + Math.Sqrt(u * (b - a) * (m - a))
            : b - Math.Sqrt((1 - u) * (b - a) * (b - m));
    }
}

public sealed record PastEpic(string Name, double EstimatedWeeks, double ActualWeeks)
{
    public double Ratio => ActualWeeks / EstimatedWeeks;
}

public static class Pert
{
    // The "bulk order from a list" slice, O / M / P in developer-days
    static readonly WorkPackage[] Packages =
    [
        new("Parse pasted list (ISBN-10/13)", 1, 2, 4),
        new("ISBN validation + catalog lookup", 1, 2, 3),
        new("Match rows without ISBN by title", 2, 4, 10),
        new("Validation result screen", 2, 3, 6),
        new("Institutional discount rule", 0.5, 1, 2),
        new("Stock check for large quantities", 1, 2, 5),
        new("Save as draft", 1, 2, 4),
        new("Submit with PO number + e-mail", 1, 2, 4),
        new("Tests, review, deployment", 2, 3, 6),
    ];

    // Outside view: Bookstore's last 8 epics, first estimate and actual
    static readonly PastEpic[] Epics =
    [
        new("CheckoutV2", 6, 9),
        new("Returns", 4, 6.2),
        new("CampaignEngine", 5, 8),
        new("MobileWeb", 8, 11),
        new("SearchRelevance", 3, 3.6),
        new("GiftCards", 3, 4.8),
        new("AdminReports", 2, 2.8),
        new("StockReservation", 2.5, 3),
    ];

    public static void Run()
    {
        Console.WriteLine($"{"Work package",-34} {"O",4} {"M",4} {"P",4} {"PERT",6}");
        foreach (WorkPackage w in Packages)
            Console.WriteLine($"{w.Name,-34} {w.Optimistic,4} {w.MostLikely,4} {w.Pessimistic,4} {w.PertMean,6:F2}");

        double sumMostLikely = Packages.Sum(w => w.MostLikely);
        double mean = Packages.Sum(w => w.PertMean);
        double sigma = Math.Sqrt(Packages.Sum(w => w.PertSigma * w.PertSigma)); // assumes independence
        Console.WriteLine();
        Console.WriteLine($"Sum of most likely   : {sumMostLikely:F1} days");
        Console.WriteLine($"Sum of pessimistic   : {Packages.Sum(w => w.Pessimistic):F1} days");
        Console.WriteLine($"PERT total           : {mean:F1} days, sigma {sigma:F1}");
        Console.WriteLine($"PERT 85% (normal)    : {mean + 1.04 * sigma:F1} days");

        const int Runs = 10_000;
        Random rng = new(2026);
        List<double> totals = [.. Enumerable.Range(0, Runs).Select(_ => Packages.Sum(w => w.Sample(rng))).Order()];
        Console.WriteLine();
        Console.WriteLine($"Simulation, {Runs:N0} runs (triangular):");
        foreach (int p in new[] { 10, 50, 85, 95 })
            Console.WriteLine($"  p{p,-2} {Stats.Percentile(totals, p),5:F1} days");
        Console.WriteLine($"  P(total <= sum of most likely) = {totals.Count(t => t <= sumMostLikely) * 100.0 / Runs:F1}%");

        // Reference class: actual / estimate ratio of past epics
        List<double> ratios = [.. Epics.Select(e => e.Ratio).Order()];
        Console.WriteLine();
        Console.WriteLine($"Reference class: {Epics.Length} past epics, actual / estimate");
        Console.WriteLine($"  ratios: {string.Join(" ", ratios.Select(r => r.ToString("F2")))}");
        Console.WriteLine($"  p50 uplift x{Stats.Percentile(ratios, 50):F2}, p80 uplift x{Stats.Percentile(ratios, 80):F2}");
        Console.WriteLine($"  sum of most likely x p80 uplift = {sumMostLikely * Stats.Percentile(ratios, 80):F1} days");
    }
}

On every run the simulation draws a duration for each package from a triangular distribution and adds up all nine. A triangular distribution uses O, M and P directly and is easy to compute; the Sample method turns a single random number into a duration with the inverse CDF. We give Random a fixed seed of 2026 so the output is reproducible. We add case "pert": Pert.Run(); break; to Program.cs and run it:

dotnet run --project src/Bookstore.Planning -- pert

Work package                          O    M    P   PERT
Parse pasted list (ISBN-10/13)        1    2    4   2.17
ISBN validation + catalog lookup      1    2    3   2.00
Match rows without ISBN by title      2    4   10   4.67
Validation result screen              2    3    6   3.33
Institutional discount rule         0.5    1    2   1.08
Stock check for large quantities      1    2    5   2.33
Save as draft                         1    2    4   2.17
Submit with PO number + e-mail        1    2    4   2.17
Tests, review, deployment             2    3    6   3.33

Sum of most likely   : 21.0 days
Sum of pessimistic   : 44.0 days
PERT total           : 23.2 days, sigma 2.0
PERT 85% (normal)    : 25.3 days

Simulation, 10,000 runs (triangular):
  p10  22.3 days
  p50  25.4 days
  p85  28.2 days
  p95  29.9 days
  P(total <= sum of most likely) = 3.1%

Reference class: 8 past epics, actual / estimate
  ratios: 1.20 1.20 1.38 1.40 1.50 1.55 1.60 1.60
  p50 uplift x1.40, p80 uplift x1.60
  sum of most likely x p80 uplift = 33.6 days

Reading the Output

The sum of the most likely durations is 21 days. Out of 10,000 possible futures, the simulation says the total stays under 21 days in only 3.1%. Telling Selin "21 days" would be a promise that holds in about one project out of thirty. The reason is the shape of the distributions: a duration cannot go below zero but has a long tail upwards. For the title matching package M = 4 and P = 10; when things go wrong you lose six days, when they go well you gain at most two. That asymmetry pushes in the same direction in all nine packages.

The other end is misleading too: the pessimistic durations add up to 44 days, but the simulation's 95th percentile is 29.9 days. All nine packages going badly at once is very unlikely; adding up the Ps is not a realistic worst case but a mathematical way of padding.

The PERT formula gives a mean of 23.2 days and a standard deviation of 2.0; assuming a normal distribution, the 85th percentile is 25.3 days. The simulation's median is 25.4 and its 85th percentile 28.2 days. The gap comes from the choice of distribution: PERT's beta approximation gives M four times the weight, while the triangular distribution's mean is (O + M + P) / 3, which is more pessimistic for right-skewed work. The team's own history tells which one is right; but the size of the gap is useful information in itself: the choice of formula alone moves the 85% date by three days.

A bigger problem is the independence assumption. If title matching turns out harder than expected, the validation screen gets more complicated and the tests take longer too. Packages get worse together. In that case the real distribution is wider than the simulation shows. The code does not model this; the simplest way to do so is to merge dependent packages into one and give it a wide O–P range.

This simulation does something different from the one in the Kanban series post "Service Level Expectation and Monte Carlo". The Monte Carlo there samples the team's historical throughput and the team estimates nothing; the method only holds for work that resembles the past. The simulation here samples the three-point estimates the team gave for this work. It can be used for new kinds of work without historical data, but because its input is still people's estimates, it cannot escape the optimism from the first post. The fix for that is to look from the outside.

Reference Class Forecasting: The Outside View

Against the planning fallacy, Kahneman and Tversky set what they called the outside view: estimating work not from the inside, imagining it step by step, but by looking at how similar work actually turned out. Bent Flyvbjerg applied the idea to large infrastructure projects and turned it into a method called reference class forecasting. In his 2006 paper he traces forecast errors to two causes: optimism bias, and strategic misrepresentation, deliberately understating cost and duration to get a project approved. In 2005 the American Planning Association recommended that planners use the method alongside traditional ones.

The method has three steps. First, pick a reference class of past work that is similar enough. Then derive the distribution of actual outcomes in that class; in software the most useful one is the ratio of actual duration to first estimate. Finally, place the new work in that distribution: apply the ratio for the confidence level you want as a multiplier on the first estimate.

The last block of the output does exactly this. Across Bookstore's last eight epics the actual / estimate ratio ranges from 1.20 to 1.60; the median is 1.40 and the 80th percentile 1.60. No epic finished before its estimate. Applying the 80% multiplier to the most likely total of 21 days gives 33.6 days. The inside view's 85th percentile was 28.2 days. The five-day gap suggests that, as in the past, the team may have missed something in this breakdown. A realistic answer gives both views together: "Estimating the tasks one by one, 28 days with 85% probability; accounting for how our past epics drifted, it can go up to 34 days."

The calculation in the first post, where we corrected Emre's answer with the team's feature history, was in fact a small example of this. The real difficulty is choosing the reference class. Choose it too wide, say every project in the company, and work unlike the Bookstore team's gets in. Choose it too narrow and you are left with two samples. For Institutional Orders the best class would be epics lasting a few sprints that depend on an external system or another team; the eight-epic list is not detailed enough for that, but it is far better than nothing.

Wideband Delphi and Expert Estimation

If the estimate comes from a single person on the team, that person's assumptions are its only source. The Delphi method was developed at the RAND Corporation in the 1950s to combine expert opinions: experts give anonymous estimates, a coordinator collects and summarizes them, and the experts estimate again after seeing the summary. In the 1970s Barry Boehm and John Farquhar added group discussion between rounds and called it Wideband Delphi; the method spread through Boehm's 1981 book Software Engineering Economics.

In practice it works like this. The coordinator explains the work and the scope. Each participant gives an anonymous estimate, along with their own list of work packages. The coordinator shows the estimates without names, and the discussion focuses on the assumptions behind the extremes: did someone miss a package entirely, did someone assume a different technology? Then comes a new anonymous round. After a few rounds the estimates usually converge; if they don't, something about the work itself is still unclear.

Planning poker clearly draws on Wideband Delphi: anonymous estimates, talking about the differences, a new round. The difference is scale and speed. Poker works at story level in minutes; Wideband Delphi is an hours-long session with work package lists for large, one-off work, such as an entire integration project. In a small team like Bookstore's, two developers' estimates are very few independent views; inviting someone from another team who has done a similar integration may be the step that adds most to the estimate.

Spikes and Time-Boxes: Learn What You Can't Estimate

Some work cannot be estimated because the team doesn't know yet what it will do. Title matching is such work: titles are spelled differently ("Suç ve Ceza" and "Suc ve Ceza"), the same title exists in several editions, sometimes the author's name is misspelled too. The range O = 2, P = 10 is the numeric form of that ignorance. You cannot narrow it by discussing; only by learning.

A spike is research work with a fixed time limit set in advance, done to reduce uncertainty. The Bookstore team plans a two-day spike: using the school lists gathered in the Requirements Analysis series, they will try a simple prototype that normalizes title and author and matches the rows without an ISBN against the catalog, and they will measure the automatic match rate. The output of the spike is not code but an answer: if the rate is above 90%, simple normalization is enough and the package drops to 3–4 days; if it is around 60%, the purchasing officer needs a screen of candidate books and the package grows.

The spike's time-box matters. When the two days are up, the team decides with whatever it has learned; a spike should not quietly turn into development that keeps growing. It is the small-scale version of the spiral model from the SDLC series post "Incremental, Iterative and Spiral Models: Surfacing Risk Early": tackle the most uncertain topic first, with a cheap experiment.

In Production

Record three-point estimates and actual durations. At the next estimate you can see how often the team's Ps get exceeded. If Ps are exceeded one time in five instead of one in twenty, the team is optimistic even in its pessimistic estimates and the ranges need to widen. Update the reference class table every time an epic finishes, too; the multiplier itself changes over time.

Don't confuse developer-days with calendar days. 28 developer-days is not 14 working days with two developers: meetings, support requests, reviews and other work get in the way. To turn developer-days into calendar time, either use the share of time the team has historically been able to give such work, or do the conversion at team level through velocity, as in the fourth post. Sprint capacity is covered in the Scrum series post "The Sprint and Sprint Planning: Why, What, How and the Sprint Goal".

Share the simulation result without reducing it to one number. The note to Selin should give the 50%, the 85% and the outside view together; it should also say which packages widen the range most, because those are the first candidates for a spike.

Common Pitfalls

• Adding up the most likely durations: even if each package's M is realistic, their sum almost never holds.
• Adding up the pessimistic durations: not a realistic worst case but padding too wide for anyone to use.
• Presenting one person's estimate as the team's: one person's assumptions come with one person's blind spots.
• Trusting "experience" without keeping historical data: without records, experience is made of remembered successes; delays get forgotten.
• Not questioning the independence assumption: packages that get worse together widen the range beyond what the formula shows.
• Running a spike without a time-box: learning work turns into real development unnoticed, and the estimate itself gets lost.

A Small Experiment

★☆☆ Split the next piece of work you will do into five to ten packages, write O, M and P for each and replace the Packages array with your own data. What is your probability of staying under the sum of the Ms?

★★☆ Add a second Sample method to WorkPackage that draws from PERT's beta distribution instead of the triangular one. Hint: for beta(α, β), α = 1 + 4(M − O)/(P − O) and β = 1 + 4(P − M)/(P − O); you can generate a beta sample as a ratio of two gamma samples. How different are the two distributions' 85th percentiles?

★★★ Model the dependency: add a rule that raises the duration of the validation screen and test packages by 30% whenever the "Match rows without ISBN by title" package lands in the upper half of its range. How far did the 85th percentile move? Use that gap to show the cost of the independence assumption.

We now have an estimate for one slice that looks both from the inside and from the outside. In the next post we return to the whole of Release 1: deriving a range from historical velocity, showing scope separately on a burn-up chart and deciding how scope flexes if March 1 stays fixed.