Predictive planning often fails in large organizations dealing with knowledge work. Unlike physical labor, cognitive tasks are difficult to plan because of uncertainty. The farther out we plan, the more uncertainty increases. There are four main types of uncertainty, often referred to as VUCA:
- Volatility – The constant, increasing rate of change.
- Uncertainty – Our inability to predict the future.
- Complexity – Hidden connections between things.
- Ambiguity – The lack of clarity about what’s happening and why.
While these conditions rarely affect physical labor where tasks are repeatable and predictable, they are a constant factor in knowledge work. For example, I know how long it takes to pour concrete or machine metal parts. But in developing new technology within complex systems, the complexity is often revealed as the work progresses.
Breaking work into smaller, valuable batches reduces complexity, allowing for shorter-term planning. This minimizes ambiguity, volatility, and the risk of canceling projects after significant investment. A distant fixed scope tied to a far-off deadline will inevitably change multiple times. Long-term, detailed planning is wasteful because plans become obsolete with each shift.
Instead, short-term plans combined with a high-level roadmap are far more effective for knowledge work. We accept uncertainty as a given, allowing work itself to reveal the truth—rather than trying to force-fit reality into a static plan. Comprehensive planning for knowledge work is as accurate as predicting the weather three months out.
Implications
If we embrace uncertainty, our plans become low-detail and short-range, with checkpoints for reviewing progress. By completing small batches of work, we can adjust the roadmap as we learn more, which helps avoid delays, rework, and loss of motivation.
Long-term planning is wasteful because it operates under the false assumption that we can control the future. Planning large batches creates delays, reduces urgency, and makes it harder to pivot, leading to increased time to market and lost profit. As a result, organizations become less flexible and more prone to failure when faced with change.
In my experience, I’ve never seen a project come in on time, on budget, and on scope without changes along the way—especially when the timeline stretched for months. The very existence of change requests reveals the failure of comprehensive planning.
Cascade Effects of Long-Range Plannning
When we engage in long range planning, a cascade effect happens that causes a host of other problems. Assuming we start long range planning:
- Large batch work. We end up creating a large batch of work. We know from experience in managing processes that large batches of work flow through systems more slowly and with higher levels of defects. A large, complex plan creates uncertainty through its complexity.
- Delays. The large batch of work moves more slowly, so the waste of delays is created.
- Lost sense of urgency. As things slow down, everyone begins to learn helplessness against the slow motion. A sense of urgency is replaced by complacency.
- Longer time to market. The time between us having an idea and getting that idea in front of customers expands in exponential correlation with the increase in complexity.
- Lost profit. We lose money because what could have been valuable to us is being held up by all of the other connected work upon which it is dependent.
- Wide turn radius. As all of this starts to happen, we find that our organization’s ability to pivot in the face of needed change is greatly reduced.
- Brittle organization. Long range plans that are large and comprehensive require strong control systems to handle the massive scope, far future dates, delays, lack of urgency, longer time to market, lost money and value. Heavily centralized organizations tend to break as a unit when faced with catastrophic failure rather than flexing and limiting damage to a small, few areas.
The very existence of change requests not to handle new ideas but to repair plans after learning new things demonstrates an inability of comprehensive planning to handle mid-stream learning and change. If we were able to successfully plan and execute work, we would not create status meetings, status reports, and change requests. We would not make war rooms for releases. We would not have known issues when we release work. These things exist because planning in this fashion doesn’t work for cognitive work. It was originally adopted for physical labor, and that’s where it works.
Beyond Theory to Reality
In one example, a team I managed opted for one-month sprints against my advice. At the end of the iteration, not much was completed. The longer timeframe reduced urgency, leading to procrastination. When we switched to shorter sprints, work was finished, and retrospectives were far more productive.
In another case, a large program spent nine months writing requirements for a new product. But in the end, a scrum team took the initiative, built the product in one sprint, and everyone agreed it was sufficient. This highlights the futility of long-term planning when doing actual work reveals the truth.
There are endless case studies in history of long-range plans going awry. I thought about googling a bunch of them up and listing them here, but it’s like shooting fish in a barrel. I thought about talking about past long-range planning projects I had managed or helped manage in the past, but instead I could just skip all of that and say all of them had endless change requests, and change requests are wasteful overhead that is handled automatically by iterative, incremental development.
Where are the stories of short-term plans going badly? There are few, because short term, small plans that yield bad results are easily scrapped after learning the truth and replaced by new, small plans. There is no big explosion that becomes worthy of telling a story. There wasn’t enough risk in the short-range plan to cause an explosion.
To those who still believe in comprehensive planning: create your plan and imagine you will bet $1,000 that you’ll hit the scope, schedule, and budget without changes. No moving milestones, no extra funding, no change requests. You’re already shaking your head because you know that’s not how it works.
Ries, E. (2011). The Lean Startup: How today’s entrepreneurs use continuous innovation to create radically successful businesses. Crown Business
Reinertsen, D. G. (2009). The principles of product development flow: Second generation lean product development. Celeritas Publishing.
Duke, A. (2018). Thinking in bets: Making smarter decisions when you don’t have all the facts. Portfolio/Penguin.