Organizations rarely get into trouble by approving one bad project. The trouble comes from running more projects than they can staff, funding rewrites that never reach an end, and never checking whether finished projects delivered what they promised. Then a portfolio scorecard is added to get control back, and the organization starts producing whatever the scorecard rewards.
This practice works above the single project. It covers choosing which projects to fund, running related projects as a program toward one outcome, deciding whether to evolve or replace a system, measuring the portfolio without ranking people, running a PMO that helps rather than polices, and confirming that the benefits arrived.
When to use this
- Use it when deciding which proposals to fund and in what order.
- Use it when several projects serve one business outcome and need shared sequencing, risks and release dates.
- Use it when someone proposes replacing a system rather than improving it.
- Use it when designing portfolio reporting, or when the existing reporting has started to drive behavior.
- Use it when setting up or reviewing a PMO.
- A single project with its own funding and no dependencies needs only the other practices. The selection and benefits steps still apply when it's proposed and when it closes.
Principles
Fund outcomes, then projects. Every project in the portfolio traces to a business outcome. A program is the set of projects that share one, run together so their dependencies and releases line up.
Capacity is the real budget. Starting more projects than the organization can staff doesn't speed anything up. Every project slows down, and the ones that matter most wait behind the ones that started first.
A measure used as a target stops measuring. Scorecards get exactly the behavior they reward, and the arithmetic can be correct the whole way through. Patterns invite conversation; scores invite gaming.
Every investment has an exit condition. Whether the plan is to evolve a system or replace it, say what done looks like and what evidence would make the organization stop and re-plan.
Prefer the smallest change that moves the business meaningfully forward. Small doesn't mean trivial. It means taking on no more disruption than the change requires, in steps that each leave the business better off.
Intervention follows evidence, not anxiety. A stable system deserves attention, not activity. Check whether it's quiet because it solved the problem or because nobody is looking, and change it only when the evidence says so.
The loop closes at leadership. Teams can build, validate and measure well, and the portfolio still stalls if nobody with authority is willing to be redirected by what the measures say.
The practice
Select and prioritize projects
- Require every proposal to arrive with its outcome measure, a quick estimate range at P70, its dependencies, and an expiration date.
- Score proposals against the same criteria every time. The scorecard is in Decision aids. Consistency matters more than the weights, because it lets leadership compare unlike projects.
- Compare the approved list with the people available to staff it, not with the budget. Approve up to capacity and no further. Everything else waits in Proposed or goes back to Definition.
- Re-rank at a fixed cadence unless an outcome or a constraint changes. I use quarterly at most, because re-ranking every month means nobody finishes anything.
- When a new project displaces one in progress, say which one, out loud. Pausing a project is a decision someone owns; letting it starve is not.
Run related projects as a program
- Name the shared outcome and one owner for it, usually the sponsor or business owner who funds the program.
- Map the dependencies between the projects, and sequence the work so each project's releases unblock the next rather than wait on it.
- Put every project in the program on one release calendar, so release clocks line up and a slip in one project is visible to the others by its D−5.
- Keep a program risk list for the risks that cross project boundaries: shared environments, shared people, integration points and external dates.
- Review the program's outcome measure, not just the projects' delivery status. Four projects on schedule can add up to a program that isn't moving its outcome.
Decide whether to evolve or replace a system
- Write the two options side by side. For replacement, list what freezes, what gets rediscovered from scratch, how long live traffic stays on the old system, and when the first value ships. For evolution, list the slices and what each one delivers.
- Find out whether the problem is in the logic or in the foundation underneath it. Evolution fixes logic. If the platform or data model underneath is being retired, incremental work on top of it may be wasted.
- By default, evolve in slices that each ship something useful, and give the evolution an exit condition that triggers re-evaluation.
- Approve a replacement only with a realistic picture of what the existing system does, a plan for getting there, the staffing to execute it, and a definition of done. "A proper rewrite" is an intention, not a plan.
- Never restart a replacement just because a better technology appeared mid-project. Each restart makes sense on its own, and the pattern is how projects run for years without shipping.
Check stable systems on purpose
- List the systems that have gone quiet: no defects, no requests, no changes.
- Check each one at a set interval for evidence that it still fits the business, such as usage, dependency health and whether the process it supports has changed.
- Leave it alone when the evidence is good. I've seen more damage done by changing working systems for the sake of activity than by leaving them alone. Schedule work only when the evidence says the fit is slipping.
Measure the portfolio without building a zoo
- Measure outcomes and delivery health at the project and portfolio level, never as individual rankings.
- Use each measure to start an investigation, not to settle a judgment. A rising defect count may mean the product got worse, or that reporting a defect finally got easy.
- Pair every engineering measure with the customer-side observation it's supposed to explain. A dashboard can improve while the customer experience stays exactly the same.
- Write down what each measure can't tell you, next to the measure. The table in Decision aids gives a starting set.
- Don't publish leaderboards, badges or composite health scores whose formula nobody can defend. A number that can be calculated is not the same as a conclusion that can be supported.
Run the PMO as a service
The PMO owns the practice, not the projects' decisions. It runs intake and the selection scorecard, maintains Project Mechanics standards and templates, coaches delivery leads, produces portfolio reporting, and turns closing lessons into updated checklists and estimating multipliers. Judge it by whether projects decide faster and more consistently, not by how many templates are filled in.
Confirm the benefits arrived
- For each project, carry its outcome measure, baseline and review date from the proposal into the portfolio record.
- Roll up the outcome reviews that each project's business owner holds after release.
- Compare the realized benefits with the proposals that justified the funding, and feed the gap back into selection. Proposals that habitually promise more than they deliver should get harder questions.
Decision aids
Selection scorecard
Score every proposal against the same questions.
| Criterion | The question to ask | Evidence required |
|---|---|---|
| Outcome | Which business measure moves, and by how much? | Measure, baseline, target |
| Fit | Which strategic goal does it serve? | A named goal, not "alignment" |
| Cost | What does it cost at P70? | Range and assumptions |
| Risk | What could stop it, and how likely is that? | Top three risks |
| Urgency | What does waiting a quarter cost? | A date or a penalty |
| Dependencies | What must happen first, and what waits on it? | Named projects or teams |
Evolve or replace?
| Evolve when | Replace when |
|---|---|
| The problem is in the logic, and the foundation is stable | The platform or data model underneath is being retired |
| The system carries live traffic that can't pause | Usage can move gradually to a new system |
| Each slice can ship something useful | Incremental change would only patch around a broken foundation |
| The team can describe the next three slices | The replacement has a plan, staffing, an exit condition and a definition of done |
Measures and their limits
| Measure | What it can show | What it can't show | Don't use it to |
|---|---|---|---|
| Cycle time | Where work waits | Whether the work mattered | Rank individuals |
| Change failure rate | How fragile releases are | Why they failed | Punish the team that reports honestly |
| Time to restore | How fast recovery happens | How often failure occurs | Set a target that discourages reporting |
| Escaped defects | Gaps in validation | Product quality on its own | Compare teams with different products |
| Outcome measure | Whether the business moved | Which change moved it | Credit or blame a single release |
Who owns what
| Decision or task | Owner | Advises | Informed |
|---|---|---|---|
| Portfolio selection and ranking | Leadership | PMO, sponsors | Delivery leads |
| A program's outcome | Sponsor | Business owners, PMO | Program teams |
| Program sequencing and shared calendar | PMO | Delivery leads | Sponsor |
| Evolve or replace | Sponsor | Engineering lead | Leadership |
| Portfolio measures and their definitions | PMO | Analytics | Leadership |
| Benefits review | Business owner | Analytics, PMO | Sponsor, leadership |
Signs it's going wrong
- More projects are in progress than there are teams to staff them. Everything is late in the same way.
- Every project is "high priority." The ranking has stopped meaning anything.
- People are being ranked, or given badges. The organization is about to get exactly the behavior those rewards describe.
- Dashboards keep improving while customer complaints stay the same. The measures and the experience have drifted apart.
- A replacement has been "almost done" for several quarters with no exit condition. It's restarting quietly.
- Nobody reviews benefits after delivery. The next round of proposals will promise too much in the same way.
- Stable systems are churned by changes nobody asked for, or nobody has looked at a quiet system in years.
- The PMO is judged on template compliance. It has become an auditor instead of a service.
Related terms
Program, Portfolio, PMO, Benefits Realization, Exit Condition, Outcome, Business Owner, Delivery Lead, Definition of Done, Confidence Level, Release Clock, Authority, Sponsor
Where this came from
- The Engineering Zoo: When the Dashboard Gets Better but the Zoo Does Not: scorecards that get what they reward, and pairing measures with the customer's experience.
- Dave's Top Ten: Git Stats You Should Never Track: patterns invite conversation, scores invite gaming.
- Evolution over Revolution: A Pragmatic Approach: evolving versus replacing, exit conditions, and the cost of repeated restarts.
- The Shark Stops Evolving When It's Done: attention versus activity for stable systems.
- Tactical Excellence, Strategic Truth: Who Checks Whom: the loop that stalls when leadership won't be redirected by the evidence.