Now don’t get me wrong. I plan and I estimate. But reality unfolding always has a way to humble me. It’s not that I was wrong to try to guess; I am wrong when I believe it too much and sometimes even use the prediction/plan/estimates to shield myself from reality based feedback.

Recently, I was watching an interesting argument about the ability to estimate and do planning with Agile/Scrum. This topic can get people very animated and emotional. “Yes you can, if you know how to estimate”. “No you can’t because its the future. Estimates become predictions and then you beat the team to make the guess turn true.” Below are some ideas I wrote about this. If you don’t want to read it, my short answer is like that famous quote “the planning matters but the plan doesn’t”. Even shorter; it doesn’t matter.

The key is a core belief that software development is more a process of discovery rather than an assembly process where the solution is known. In scientific discovery, say to cure cancer, it would be absurd to get MS Project out and Gantt chart out end day. On the other hand if software were like a manufacturing assembly process it would be reasonable to give a lead time for delivery with very good accuracy.

What Agile Scrum does is acknowledge the reality that software product development is on the spectrum pretty far toward discovery. Waterfall on the other hand believes software development is basically assembly. All you need is a perfect specification (an order) and you can get a perfect product on a specific predicted day (Unless someone screws up).

What makes Agile Scrum work to meet a schedule is not because of points and velocity. It’s because the team is focused on constantly reprioritizing to make sure the most valuable thing is built to done next. This way, if you want to release value in a domain in 3 months you know you can delivered the most value possible. There is no sense of done from a product perspective. There is always more; it’s just a matter of stopping when there is nothing very valuable left to do.

So the Agile/Scrum way to think about time estimates is that we will have something in a month (or 3 months) and I know it will be the best we can do and I know it will all be working software. Will it be the ultimate final solution to that domain? I don’t know. Can we deliver on a specific day? Yes. What will be in scope? Whatever you say is most valuable.

So by keeping the software always working, allowing for constant reprioritization for value, and allowing for discovery instead of specing out the perfect solution, we can hit a date with something pretty good most of the time. This shifts mindset to product development and away from fixed start fixed stop project mentality. And you don’t need time estimated for fixed scope. Often this is a very slow cultural change in a transforming company.

Now, keeping the software always working is hard. It’s even harder when your team is not the only one working on it. I recall the feeling I had when people started talking about this idea. It was nuts. Previously, we would build something once to get through the acceptance test once. If every time we checked in code, the software still had to work, we needed to work much much differently. We had to know how to build something quick and crude but not sloppy. We needed to pick up out toys and clean up. We needed to re-engineer legacy code by isolating the code with unit tests before we “fixed” it. We needed to build and test much much smaller. We could not have a gap from building and testing. And we had to do all this when the company was only really rewarding new functionality on a fixed schedule.

When you have a mindset that is project based you are open to many anti-patterns. One is placing too much value in plans/estimates. If you are thinking that the estimates will give you the ability to predict set scope on set time you will probably be pretty disappointed or be blaming a lot of people . It’s not because we are bad estimators or you have a bad team. It’s because even with perfect information the future is unknown. And most of the time we have way less than perfect information. Because of this Agile mindset places a greater focus on being adaptive than predictive. The most valuable reason for estimation is for the team to take care not to try to do something too big. We estimate what we are about to do when its ready to be done. Little’s Law tells us that if we work on things too big we will get less done. The highest performance comes when the team uses estimates to keep the throughput of value high. This is way more important than trying to hit fixed scope on a schedule. Again, this is so because you are keeping the software always working. If the team is not keeping the software shippable than all bets are off. You may as well use Gantt charts.

If we keep the software working, we relentlessly and rationally reprioritize, and we keep the team focused, then you will always have the most valuable system that can be shipped on any date you want. This makes the schedule irrelevant; or at least not important. Release decisions are driven by external factors like customer urgency or competitive pressure verses internal pressure like running out of budget or bonus time.

After many years of building systems, I don’t think that if you had good estimators you can predict fixed scope and schedule. And I will go a bit further, most systems that do come in with scope and schedule have very low value. So the Agile/Scrum way is to release the most value you can at the right time.

Often this mind-set shift can be awkward when you are in a company in a half-state transformation. The budgeting and performance management is still project centric. You get money to deliver scope. You get bonus for meeting objectives. This is funding as if there is no future after the project. It’s like that product or those customers won’t still be there and need more. This leads to many anti-patterns; at least to constantly re-building teams to deliver this quarter’s scope. This is at odds with a more Agile/Scrum model that funds capacity to impact in an area and trusts teams to make the highest impact with the most value. This latter model is focused on impact versus following the plan (delivering on time aka estimating right).

Sorry for all the words, what are your thoughts?