This is a follow up post to How Long Will It Take. I often read that we need to know the “all in cost” before embarking on an initiative. I’m not sure at what level this rule applies at i.e. the program, product or feature level etc etc. I do however disagree with that position. I agree that sometimes its needed for decision making re how valuable are your estimates.
Over a decade ago, I was tasked with the programming language/technology platform conversion of a large supply chain system. It was a change we needed to make in order to remain relevant and stay in business. Both the platform and language were brand new to all of us and we had very little knowledge as to what the conversion would really entail for this complex system. Yet this was something we just had to do or we’d be out of business, remember?
I was asked to estimate how long it would take to complete this assignment and I tried my best (with a lot of preliminary research, long discussions with the company that was retiring the language/platform and searches for other companies who were doing the same thing) but at the end of the day the upper bound of my estimate range was still too low. I will accept that I could have possibly tried harder or used better estimating methods but given how much was not known, I question how valuable the estimates would have been anyway. Would they have really reduced uncertainty? Would the reasonable uncertainty range have provided much value? Did we need an estimate to decide to port the solution suite? I daresay that even if I had come in under, it would have been a result of luck or pushing the team to the brink and not the estimating procedure (but no one makes their teams make sure they come in at the estimate, right?)
In full disclosure, several months into the conversion, we did become good at projecting how long things would take. But this was a result of a ton of experimentation and learning that we had acquired after a couple of months of working through things. We now had historical data and experience that we could reference when asked “how long”. Based on my experience, it seems quite obvious that we often need to experiment and learn first and then and only then can we attempt to estimate effectively (if we still need too). It would seem that there are those are uncomfortable with the notion of getting paid to learn or discover. I think this gets to the heart of some of the debate surrounding #NoEstimates i.e. a disagreement about the nature of the work we do or how much do we really know at the onset. I, based on my experience, believe that learning and discovery reside at the core of product development. It’s knowledge work. Maybe that will change with time, we’ll see.
“How long will it take?” is a question that gets asked often in many spheres of life. Sometimes, we are able to answer immediately because “it” is something that we’ve done time and again. For example, if you ask me how long it will take me to prepare jollof rice, I can answer that immediately. Sometimes, there is a delay in our answer because we don’t know but know someone who does. For example, if you ask me how long it will take to make biryani, I’ll need to go look up a recipe and figure what the recipe says about how long it (should) takes.
But what happens when we don’t have or can’t get to the information? What happens when the range of answers is so wide that its of little value? What happens when we simply don’t know. For example, as a music writer and composer if you ask me how long it will take me to write and compose a new piece, I won’t have a good answer for you because I really don’t know. It could be just me but in talking to friends who are involved in creative-type activities, I’ve come to observe that its hard to determine how long a creative act will take (except you are copying someone/something else).
So how do creatives get things done? How do I end up finishing a piece? How did you get your term papers done? How do you finish writing a book? Deadlines. Deadlines are what keep this creative in check. Even when I finish early, its probably because there was a deadline of some sort. Without deadlines, I would probably write a single piece for six or seven months. But when a friend says they need a song for their wedding in the summer, I know I need to constrain myself.
I have experienced software development as being highly creative endeavor. I imagine that some feel and have experienced differently. In my professional career, I have tried to avoid situations where there is not a lot of creativity and hence I find that it is often very challenging to answer the “how long will take” question in a valuable way when it arises. So I usually ask for a deadline instead – as a side note, very often there is no deadline, it just happens that someone has a need to know how long just like I want to know when an Arsenal player will come back from injury – and then the team works towards that deadline. Working towards a deadline has some interesting side-effects as well but that is beyond the scope of this post.
But just like with most things in life, please use deadlines responsibly. Understand that a deadline that is too aggressive can kill creativity and your product. And while we’re at it, please don’t turn an estimate into a deadline. Thanks!
As I have mentioned in previous posts, it’s critical to understand the nature of the work that one is engaged in or significantly dependent on to ensure that everyone (including both C-level executives and developers) have their needs met.
My personal experience with product development is that it is characterized by discovery and learning that occurs throughout the product development cycle. In my experience, and it may be unique to me and the teams I’ve worked with, we knew the least about what needed to be done at the beginning in spite of all the conversations that happened upfront. Enlightenment occurred as the work progressed and then at some point there was that eureka moment where it all came together for a particular item. I’ve even experienced this when working on the second system. This has not been the case, when I’ve been engaged in highly-repetitive work. So if you are domains in which you do highly-repetitive work or you are required to do majority of the learning upfront or where the scope once defined doesn’t change significantly i.e. it is more fixed than not, this post is also probably not for you.
I wondered aloud on Twitter a couple weeks as to why there are so many cost overruns in product development regardless of the sector. I find it hard to believe that all these projects (many which should be executed as products) did not use mature estimation approaches but if I’m wrong, I’d love to be corrected. A thought leader in the software development industry who I respect suggested that “a good/great Agile team can can deploy software at any given time”. The implication being that a good/great Agile team can stop when budget has been met. I totally agree because I have experienced this firsthand, however I also observed that when we stopped, the “finished” product was not exactly what had been conceptualized on paper simply because as we developed, we learned new information and adjusted accordingly. This is not to imply that no features from the original plan were in the finished product but rather that some features were in and others were not but possibly replaced with other features. This isn’t a bad thing.
So for Agile teams (at least), our estimate is not (or is it should not be?) tied to a fixed scope. How could it be except we ignore the nature of our work and insist that we stick to some plan we came up with when we knew the least about what we were doing? In other words, our estimates don’t tell us specifically what we’re going to get with a certain cost. But we know that already, don’t we? Do the people we’re working with understand that? Do they understand the concept of variable scope?
So then, what did the original estimate REALLY tell us? Well it depends on what question we were trying to answer in order to reduce uncertainty. I’m hopeful that no one is contracting work out based solely on cost estimates so let’s assume that we were uncertain as to whether we should proceed and the cost estimate was what we needed to determine if we should proceed or not. The estimate we received made us more certain that we should proceed but now that we have the end result, how valuable was that estimate? I’ll let you the reader decide that. Or maybe we were uncertain as to what the budget should be for an approved initiative and so we needed a cost estimate to determine that. Now that we’ve met budget, how valuable was the estimate? Once again, I’ll let you the reader make that call.
But here’s an idea on budget. What if we have a measures for both ROI and value/gain (and we can know these things going in because we can measure anything, right?) could we not come up with how much we should consider spending i.e. our potential budget? It’s pretty simple algebra (who thought we’d be talking #NoEstimates and high school algebra?) The question now becomes “who should I entrust with my money” because you definitely don’t want to someone to mismanage that.
Just a thought.
Estimates are supposed to inform decision making by reducing uncertainty. This also applies to the types of estimates developers are asked to provide. Because estimates (not guesses and guesstimates) require some effort and rigor, its only fair to ensure that the product of the developer estimation process actually reduces uncertainty to the point that it actually justified going through the estimation exercise.
If you were to do an analysis of the estimation activities that your team(s) perform, would you find that they are truly used to reduce uncertainty? Does their impact on decision making justify the effort spent in actually doing them? Do you ask yourselves “what question does this answer and how does this estimate reduce uncertainty or can we use some other criteria for decision making?”. What would go wrong if you didn’t do them? Why are we doing them in the first place?
Are your estimates just a proxy for communicating something else such as a deadline or fixed budget?
Could the exercise just be a ritual or tradition that may satisfy/address some human need. Human needs are extremely important, but could it be met some other way? I’m a ardent supporter of Arsenal Football Club and every time a player is injured I want the estimate on when the player will be back. It gives me some comfort to know when a player will be coming back but that’s about it. I don’t use the estimate for anything else.
So do yourself a favor, take some time to see if your (developer) estimates are truly valuable. It may seem like this should be common sense but as Voltaire put it:
Common sense is not so common.
I’ve spent a good portion of this morning filling out multiple brackets for different pools before the start of the NCAA Men’s College Basketball tournament. I couldn’t but help notice that the programs asked me to guess my picks. I’ve posted previously on the difference between a guess, guesstimate and estimate and I wondered if this had any relevance to filling out brackets..
A guess would be making a pick without any information on the teams playing. A guesstimate would be making a pick based on some existing knowledge of college basketball and the teams playing. An estimate would be making picks based on some degree of rigor which would possibly include a study of the teams, their seasons and the matchup between the teams. Sports analysts make estimate type picks.
So what kind of pick did I make? Well given that I follow mens’s college basketball a little bit, I suppose my picks are more guesstimates than guesses. I really didn’t do any analysis, I just picked based on some pre-existing knowledge of some of the teams and tournaments past. But I’m pretty certain that some of my team members who are in this pool are simply guessing. This reminded me of something that happened recently where a business owner provided a one page list of features that they wanted add to their system and asked me how long I thought it would take. I guesstimated that it would take at least a couple of months. I consider this to be a guesstimate (and not a guess) because I used knowledge gained from my experience developing software systems. I couldn’t see how a one page listing of features would not take as least months but for all I know, it could be years, then again, maybe it will be weeks.
Reflecting on this a bit more, it occurred to me that many of the answers (to questions that get asked in the day-to-day ongoings of a software development team) that the teams I have been a part of have provided as estimates were really either guesses (because we have no idea or knowledge to work with) or guesstimates (based on some residual knowledge that we have). It’s sort of like being asked for juice and providing flavored water instead. Even though the drink tastes good, its still not juice.
So what does this have to with #NoEstimates? Well it has been suggested that some within the #NoEstimates movement believe that estimates are guesses. I don’t believe estimates are guesses but have observed that guesses (and guesstimates) are very often provided as (if they are) estimates. So how do we address this? I see two options that I do not believe are mutually exclusive (a) actually provide true estimates (b) remove the need for estimates in the first place. Personally, I’m interested in both options.
Enjoy the tourney!
The use of analogies when discussing software development is quite common and has rung true with #NoEstimates. We know that an analogy is a construct of our languages that allows us to explain “a thing” by using something else that is similar and familiar.
In the case of #NoEstimates, analogies have used activities such as building a house or fixing a car. The point being that the individual who wants these activities performed for them needs to know the cost upfront. So while these analogies may be good, I still have a problem with them (and yes I understand that no analogy is perfect) for the simple fact that (a) my experience informs me that the nature of these activities is different enough from that of developing software and (b) I think there are better analogies.
Before my mechanic fixes my car, he performs a thorough assessment of the vehicle until he determines what the problem is and how it is going to be addressed and then he provides the cost (it’s not really an estimate). When I had my basement finished, it was also a very similar process of defining upfront what was going to be done and how it was going to be done and then getting the quote. Don’t get me wrong, we can approach software development in exactly the same way we approach fixing a car or building a house. We can try to get to that level of detail upfront and in some cases, maybe its the right thing to do. But with all the knowledge and experience we now have, is this what our general practice should be?
In my experience, non-software people (the people we like to call the “business”) already consider software development to be just like building a bridge, finishing a basement or repairing a car and use the same analogies we use (or taught them). When we (re)use the same analogies, we validate and reinforce their thoughts and ideas. In my experience, this is rarely helpful. I prefer to use analogies that highlight the continuous discovery, learning and experimentation that occurs during software development that I am involved with such as that of developing a recipe (food), a pharmaceutical drug or new school curriculum.
At the heart of this is the debate regarding the nature of the work we do. Everyone is largely influenced by their own experience. If your experience is like that of building a house or fixing car, then using those analogies might be the best you can do. If on the other hand, your experiences are like mine, then try to use analogies that highlight the fact that we will be learning (a lot) as we go along. I think it will benefit everyone involved.