Is estimation generally a worthwhile endeavor? I really used think so. I actually promoted multiple levels of estimation in an effort narrow the “cone of uncertainty”. I’m not sure I’m proud of that. However, for about a year now, I’ve begun to question the legitimate reasons for doing significant estimation (at any level). I feel that the need for the activity is far and few in between (for example having to deliver something by a hard date or the business will collapse) and the activity of estimation should be minimal, opportunistic and spread throughout the development process. It also seems to me that estimates should not drive business priorities because estimates are generally very unreliable, but for some reason it generally seems to be the most important factor at times. Estimation (in my opinion) is an attempt to treat symptoms presented by software delivery challenges. Here are just a few reasons why I think estimation is largely a waste and is ineffective in many instances.
- Estimates focus on the amount of effort that will be expended to complete a work item. It is usually a single number and it is usually wrong. Ranges are better, but unfortunately can cause even more confusion (the wider they are).
- Except you are developing the same type of product in the exact type of way with the same people, your historical data estimates from previous projects is probably of very little use. Even if you compare projects by “complexity”, you’d need to ensure that all conditions for the executing of the previous project will be the same for new project. Not.
- Even if you are developing the same type of product in the exact same manner with the exact same people, we are assuming that software development is deterministic, and unfortunately its not. Things happen, sorry.
- If it just so happens, you’re the best estimator and use all the techniques we are aware and hence get an estimate range that is actual right, you still have the schedule (calendar) to figure. For example, if you estimate an initiative will take 40 – 80 hours is that 1 – 2 weeks on the calender or 1 -2 months? How do you account for holidays, sick time, emergency client issues, meetings etc etc?
- Let’s just suppose that you get both the estimate and the schedule right, now you have to explain to multiple stakeholders why a 80 hour initiative will take a month (and not two weeks). You sure you really want to do that?
So how then do you plan for a release? How do you commit to a plan? I think these are the wrong questions driven by the wrong set of of goals. If you are arbitrarily planning to stuff features into a time window based on a plan, you will most likely fail because your plan won’t execute as you want it. I guarantee it. Ask yourself this: Has any of your software development delivery releases ever gone to plan i.e. the original plan? I’ve been around software and software delivery (in some capacity) for close to two decades and I have never seen a software development plan be completed as originally intended. Things are either removed from the plan or resources are asked to give an “arm and a leg” in order to make the original plan. So if we know that our plans are most going to change, why do we spend so much effort estimating and planning and then seeing the plans fail time and time again? Albert Einstein defines insanity as “doing the same thing over and over again and expecting different results’. Since our results are generally the same, why do we continue practicing them? Our goals need to change. Instead of focusing on “how much can be delivered”, we need to focus on “what has to be delivered for the organization to be successful”. Based on this, I will offer up a set of steps for delivering the most value in a given release.
1. Determine what has to be delivered
The most important thing is identifying what features provide the business with competitive advantage. Whether its keeping existing customers happy, automating time consuming manual processes or gaining additional market share, determining what needs to be in product by the time it is released for these goals to be met is key and of the utmost importance. At the beginning of a release cycle, work with your customers and its representatives (such as your product manager) to prioritize the list of requirements (or features or whatever you use at the highest level) designated for the release.
2. Divide and conquer
Split the development team in 1..N groups with all constituencies required for development present in each group and assign the highest ranking requirements to each group. Limit work-in-progress (another upcoming blog posting).
3. Identify simplest solution that works
Instead of having the teams “estimate” effort, have them focus on working with the customer to determining the simplest solution that addresses the gap. Over-engineering is a malaise in the software development. We love the grandiose and hence by default (and possibly unknowingly), tend to oscillate towards delivering the most expensive solution possible instead of providing the simplest solution that works.
4. Develop and deliver iteratively
Once the minimal solution has been identified, develop it iteratively with each iteration being meaningful i.e. each iteration delivers actual functionality that can be used by the end user. Deliver solution once completed. An interesting observation of this step is that based on how the teams are progressing concurrently, it will become apparent as to whether the entire backlog of business requirements can be completed by the end of the release. You won’t need estimates to tell you this, trust me.
5. Start all over
Once an initiative is completed, start at step 1 again.
While I wouldn’t claim that this sequence is perfect for all situations and does not require tweaking to handle different scenarios, I believe it does provide a framework under which development organizations can execute in a lot of situations. It commits to delivering the most important items while leaving room for business priorities to be adjusted in an ever changing environment. Priorities are adjusted based on feedback from the development team and customers. Hours are not spent estimating and planning for 12 months of development. Ridiculous commitments that lead to unbearable working conditions are not made. Everyone ends up being happier for it and business goals are met.