Predictability in Software Development – Part III

In Part I and Part II of this series, I challenged the popular perspective on predictability in software (product) development and suggested that if an organization needs to respond rapidly to the changes in its ecosystem, it needs to value adaptability over predictability at least “in the large”.  However if innovation is not critical, then valuing predictability over adaptability may be the better thing to do.

What role (if any) does predictability play in product development?  This is especially important because – as mentioned in the first post of this series – many people in positions of significant authority demand predictability from their teams.

Even though it would seem that I’ve spoken against predictability, I do believe that there is often a need for teams to be predictable however I just consider this need to be predictable in the small.  Teams that are predictable in the small consistently complete about the same amount of work over a series of short time periods. Candidly, “short time periods” is dependent on your strategic and operational approaches to addressing business opportunities.  As a rule of thumb, however, I recommend that ‘small’ be 4 weeks or less.

For example, let’s say that a team forecasts the completion of ten items each iteration over the course of four two-week iterations and then has the following results:

Forecasted Delivered
Iteration 1 10 8
Iteration 2 10 11
Iteration 3 10 9
Iteration 4 10 9

Even though the team never delivered exactly ten items, we can see that they completed about the same amount of work in each iteration.  A basic understanding of variation will help us understand that the team exhibits predictability.

But wait a second, this seems to be all about outputs and I thought we were all about outcomes?  Actually we care about both outcomes and outputs and even though we value outcomes over outputs, we know that our outputs are needed for our desired outcomes to be realized.  So, understanding how much work we can complete in the small can help us determine what outcomes can be achieved in the small.

What practices can an organization and their teams adopt in order to help them be predictable in the small? Here are three:

Bite Small, Chew Fast

As a team, focus on value adding items that can be completed in a couple of days. Use techniques such as INVEST (for user stories) to decompose large initiatives into small chunks of value.

Less Is More

Take an essentialist approach to how much work the team takes on.  Less is often more. Minimize the amount of work that is progress.  Team members with different skills should collaborate on work items with the goal of finishing the work items as quickly as is possible.

Minimize Process Loss

Ivan Steiner came up with the equation AP = PP  – PL; that is, the actual productivity of a group equals its potential productivity minus process losses.  Process losses are those things that prevent our teams from being as productive as they could be.  They dampen the good we could potentially do.

Teams need to take the time to reflect on the things that are negatively impacting their performance and then address those things with both rigor and discipline.

(To be fair, these three things are good for any Agile team to pay attention to regardless of whether they have predictability demands or not)

The Conclusion of the Matter

And yet with all this talk about predictability, I’d be remiss if I didn’t make observations about predictability and the potential unintended consequences that arise from pursuing it in an unbridled manner:

  1. Describing knowledge work teams (especially product development teams) as factories or engines needs to be done with care.  Software development teams are not machines that are executing repetitive or pre-programmed activities.  Innovation means discovery.
  2. Just because a team is predictable doesn’t mean they are high performing or that they are delivering value. (Subject of a future post).
  3. Predictability, even in the small, is often at odds with innovation and creativity.  If you are challenging your teams to be creative and innovative and also demanding high levels of predictability from them, something will have to give.  Eventually.

So after three posts on predictability, where have we landed?  Are we adaptable in the large? Are we predictable in the small?  If you’re part of a real Agile organization, then adaptability and predictability are characteristics that your organizational operating model needs to support.  Its critical that leaders provide their teams with clarity on the business landscape and guidance how they need to balance the attributes of adaptability and predictability in software (product) development.

Advertisements

Velocity is NOT about Productivity

I happen to participate in a couple of Lean and Agile subject matter groups on LinkedIn. This is my way of learning and sharing.  In my opinion, any serious Agile practitioner should join and participate in such types of groups (no it’s doesn’t have to be on LinkedIn). I personally learn more from these groups than from workshops and certifications classes.

I’ve observed over the past year that not a week or two goes by without someone having a question around Agile velocity on one of these discussion groups.  Interestingly enough, their questions are never about how to use velocity for forecasting and/or planning. No, the questions are always about how to increase or improve their team velocity.  As an example, check out this very recent velocity conversation.

Why are so many of the velocity questions focused primarily on productivity?

Unfortunately, many of these well-intentioned posters find themselves in organizations where velocity is considered to be a primary measure of productivity.  Not only do organizational leaders use velocity as a measure of productivity, we find some Agile practitioners (sometimes unintentionally) doing so as well.  I’ve seen postings on LinkedIn where teams are congratulated for increasing their velocity.  I’m saddened and disappointed that in 2016 we’re still talking about velocity in this way.  I’m not sure what got us here but instead of simply complaining, I am compelled to share my thoughts in the hopes that it may help someone or some organization now or in the future.  Paraphrasing Peter Block, start with the room you’re currently in.

First of all, let’s agree on what velocity in the context of Agile software development is. Operational definitions are extremely important and I think defining velocity will do us some good.  Velocity can simply be defined as the:

Sum of effort estimates completed in an iteration (see the Agile Alliance reference for more information of velocity).

Any other usage of the term in an Agile software development context is a redefinition of the term.  Agile velocity doesn’t refer to how fast a team works.  It’s not even the count of items completed in an iteration (that could be throughput). It how much of our effort estimates did we complete in an iteration.  No more, no less.  That simple.

As the Agile Alliance article makes clear, velocity is primarily a planning instrument. Carefully read the article as it provides a pretty good explanation of the purpose of velocity and what it should be used for.  Pay attention to the common pitfalls as well.

(However I must point out that many Agile teams effectively forecast and plan without using velocity.  Stated differently, velocity is not a requirement for forecasting and planning.)

But maybe you still think that we should be able to use velocity to gauge how productive our team is.  Maybe the information presented so far is not convincing enough for you. In that case, let’s take a moment to dissect what it means for a team to increase its velocity.

If velocity is the sum of our effort estimates, increasing our velocity would mean that we have to increase the sum of our effort estimates completed.  On the surface, it may seem that this would be easy to do.  All a team has to do is increase the number of items it completes and their velocity will increase.  However, completing more items doesn’t assure us of a velocity increase because we can’t assume that our estimates are the same or larger than our estimates for previous iterations.  In fact they could be smaller and our velocity could actually decrease and yet it would be a good thing!

For example, let’s assume a team had a velocity of 20 points in Iteration 1, which meant they completed 4 stories each estimated at 5 points.  Then in Iteration 2, they had a velocity of 18 points after completing 6 stories, which were estimated at 3 points each. Were they less productive in Iteration 2 because their velocity went down?  What if they completed 7 stories in Iteration 3 and each of those stories has an estimate of 1 point? Are they still less productive?  How do we know?

Don’t forget that velocity is calculated using “effort estimates”.  Estimates are subject to many factors that I won’t get into in this article and yet its important to remind ourselves that each iteration the team is solving problems they have never solved before. The team effort estimates could go up due uncertainty and complexity and it would have nothing to do with how productive they are.

Determining how productive a team is based on how much of their “effort estimates” they complete totally misses the point.  I think Deming would refer to this as management malpractice.  It’s that bad.

And yet teams can easily increase their velocity.  In fact, it’s not hard at all.  All they need to do is make their effort estimates larger.  I’ve seen many teams do this in my career. I’m not sure your organization really wants that.  Riffing off of Goldratt:

Tell me how you’ll measure me and I’ll show you how I’ll behave.

So encouraging teams to increase their velocity is simply something organizations that are intentionally adopting Agile should not be doing especially if they don’t want their teams to game the system.  I will go further to suggest that using velocity to determine predictability (in the small) is in serious danger of missing the point of velocity as well. There are there better ways to do this and I will be addressing that point as I finish my series on predictability.  If you missed the first two posts, check out Part I and Part II.

After all this talk on velocity, I am moved to remind us that the primary measure of progress for an Agile team (and organization) is the amount of software that has been delivered to users and that the users find valuable. All other measures are secondary and tertiary.  Velocity can be used to help forecast and plan (and there are possibly better alternatives to it) but let’s not use velocity as a measure of productivity.  If you must use velocity, use it responsibly.

Predictability in Software Development – Part II

My previous post asserted that predictability at the business or organizational level is having teams that consistently deliver solutions that enable desired business outcomes. These organizations and business are predictable in the large.  Predictable organizations have the ability to change course even as the ecosystem they are a part of changes.  Ergo,  my second assertion:

Your company values adaptability over predictability 

This is not to suggest that predictability is not important.  In fact, it is quite the opposite. In the ever-changing business landscape that now confronts us, organizations are required to adapt in order to survive.  In order to be predictable, organizations must be able to adapt.

(I will concede that it is possible that your business cares about predictability more than it cares about anything else.  I just don’t interact with a lot of those organizations these days.)  

Organizations that possess a high degree of adaptability have the capacity to change course in a largely frictionless manner.  They have the characteristic of responding to changes in the ecosystem and changing direction as needed, effectively and efficiently.

An organization can only adapt as quickly as its component parts can adapt.  More specifically, a product organization’s ability to adapt is a function of their individual product development teams ability to adapt.

Agile places tremendous value on teams being able to adapt as change emerges.   In fact the Agile Manifesto states that we value:

Responding to change over following a plan

and backs that up  with following the principle of:

Welcome changing requirements, even late in development. Agile processes harness change for the customer’s competitive advantage value number

Many Agile practices enable teams be in position to respond to change in a productive manner.  Here are a couple of them (in case you’re not familiar with Agile practices):

  • Focusing on the “what” over the “how”
  • Working in small increments of value (days not weeks or months worth of work)
  • Short delivery cycles; weeks over months
  • Stakeholder feedback on a regular cadence
  • Pair-programming/test-driven development
  • Continuously running automated test suites that validate product suite
  • Feature teams developing features
  • (I’m sure you can add more to this list)
  • Etc etc etc

Teams that leverage all these practices (and more) are in a position to respond to change very effectively.  I find however that many organizations that claim to “be Agile” are only partially committed to a few of these practices and then wonder why it’s so difficult for them to change direction.  Coincidentally, these organizations seem to also believe that Agile is about delivering software faster.  (If that’s you, it’s possible that you’ve been misled).  To top it off, these organizations often respond to their inability to adapt by instituting heavy processes and protocols that are focused on ensuring that they won’t have to adapt!

Here is a thought exercise for you: If at this very moment, your team was asked to stop working on a particular initiative and release it into production so that your customer could start using it and the team could start working on a different mission, would your team be able to do it?  How many days, weeks or months would it take for your team to be able to release a product increment at this very moment?  Would you be able to release anything at all?

If an organization desires to be predictable in the large, its teams need to be able to effectively adapt in the small.  When teams holistically adopt Agile-based approaches to software development, they place themselves in a position where they can respond to change in a predictable way.  Could your organization and teams benefit from this ability?

But wait, we’re not done yet.  There is the little matter of predictability in the small. My next post will be about that.  Stay tuned!

Predictability in Software Development – Part I

Recently, I’ve been part of few conversations regarding predictability and commitments.  It is the opinion of some that Agile teams should be able to make and keep their Sprint commitments as it pertains to outputs (points, stories, card counts).  It has also been suggested that measuring a team’s consistency on delivering on committed deliverables (outputs) is a good metric.  After all, the thing our businesses values the most is predictability.

Here is the challenge I have with these assertions.  They are based on the premise that the team should know enough about how they will address problems such that they can make a highly confident prediction on what they will complete before they actually start the work i.e. pull it into a Sprint (or do something equivalent).  Is this the case for the problems your teams solve? Is your team highly certain on how to solve problems (both functionally and technically) before they actually start solving them?

The Chinese restaurant around the corner from my house is very good at telling me how long it will take for my vegetable fried rice order to be ready because they do that multiple times a day.  My barber is very good at telling me how long it will take to cut my hair because he’s done it for me for the last five or so years (and I haven’t changed my style).  The folks who change my oil are pretty good at letting me know when to come and pick up my car.  They’ve been changing my oil for almost eight years.  Are your teams solving the same (or similar) problems from Sprint to Sprint?

I often find teams spending a ton of time (days) upfront trying to determine what needs to be done so that they can make and meet their commitments.  If they discover that they will not be able to meet their commitments,they take shortcuts (specifically in code quality) to ensure the commitment is met. When they don’t meet their commitments, they begin to look for someone on the team to blame.  If they can’t find someone on their team, they look for someone on some other team.  Are these the behaviors you want on your team(s) and in your organization?  To be fair, with a heavy dose of upfront planning, a set of stable requirements, stable technology, stable team (and a bunch of other stable factors), you may be able to achieve such predictability in a non-damaging manner.  What are you willing to pay to reduce uncertainty to a point where your team always deliver output targets?

I believe some of the mass movement to Kanban is as a result of the commitment pressure uninformed leaders place on their teams.  The distortion of Scrum – Dark Scrum – has led teams to look for alternatives. Unfortunately, this is the wrong reason to make process changes but people will do what they need to do to feel safe (and in the process mask real problems).

My tone so far probably leaves you thinking that I believe we need to punt on predictability in its entirety in software development and just let things evolve as they may.  That’s not the case. However I do strongly believe that it’s critical that we (especially business leaders) appreciate the uniqueness of software development and the challenges around predictability that it presents.

What does predictability really mean in software development?  Is it that a team consistently completes a certain number of points or stories in a given Sprint?  Would that make them predictable?  Possibly, but that’s not my opinion.

Agile is concerned with managing the unpredictable nature of software development and delivering on business outcomes ergo Agile teams that are “predictable” are teams that consistently deliver solutions that enable desired business outcomes.  My next post will look into this further.

Does Your Agile Process Mimic a Production Line ?

Disclaimer: This post is for individuals, teams and organizations genuinely interested in Agile.

I am noticing a certain trend in software development where teams make their software development process mimic a production line by treating each software development activity as a separate station.  Their “boards” make every activity a separate column.   A number of teams do this in the name of implementing the Kanban Method while others do it because it represents what they are used to.  Why is this the case?

Two reasons immediately come to mind (I’m sure you can think of more):

  • Work items take more than a few days to complete
  • Work items are acted upon in a sequential manner

If your work items consistently take more than a few days to complete, what actions is the team taking to address this?  Have you asked yourselves why?  Is the nature of the work such that working software cannot be completed in a few days?  Everything has to take a few weeks or months?  Is the team focused on too much stuff at the same time?  Maybe everyone is working on their own unique items and is not working together (collaborating) to complete items in a timely manner?  It is a known fact that the less work we have in progress, the more we get done and the less time it takes. In spite of this known fact, many organizations/teams/individuals fail woefully at limiting work in progress.

I still find teams that believe software development has to be sequential (in the small) just like a production line.  Many have described this as mini-waterfall.   Each role on the team represents its own “workstation” and for the work item to be completed, it needs to pass through all the workstations via hand-offs.  A classic indicator of this is a team where the testers complain that they have no time to test in the Sprint (another reason why teams abandon time-boxes for Proto-Kanban).  For example, a user story would “flow” in this manner:

Analysis -> Design -> Code -> Test -> Deploy

(The fact of the matter is that if it takes a few days to complete a user story, these activities would overlap to the point that making them explicit would provide little value.)

What if instead of having work move in a sequential manner we have all the roles performing their activities on the user story concurrently?    What if the work item was the center of the universe with multiple actors acting on it at the same time?  What could happen then?

There are some major implications with this approach.  For starters, it would mean we focus multiple team members and roles on as few work items as is possible.  A business analyst, engineer and tester (for example) all working together at the same time on the same thing.  If you don’t think this is possible, let’s have a chat.  Or perform a Google search.  Or ask an Agile practitioner.

This is a fundamental shift in the way we traditionally approach how we work in software development. (In the first Kanban system I ever designed, it was a rule that more than one person should be working on a item that was in development.  Team members and partners in the other departments initially found it strange as they were not used this but eventually came to appreciate this approach as it enabled us to deliver software more effectively.)

The feedback I receive from people when I bring topics like this up is that things are working just fine and hence they have no reason to do anything different.  No reason to decrease the time it takes to complete user stories. No reason to change the way they work on their work items.   In my opinion, these people have totally missed an important point about Agile (and Lean) and that is the relentless commitment to continuous improvement. The commitment to finding better ways to do our jobs even when the current ways seems to work well.  The commitment to being the best we can be. These individuals, teams and organizations are completely missing a critical component of the Agile mindset. 

So does your Agile process have to mimic a production line?

What’s Wrong With Scrum – Part III

Earlier this year, I started writing on the limitations of Scrum as a result of articles I had come across.  In my first article, I was critical of the individuals (largely Scrum Masters) responsible for guiding Scrum adoption in organizations. In my second article, I took a look at the system (and those who create them – leaders and managers) and wondered aloud as to whether the organizational constructs actually supported Scrum being effectively leveraged in our organizations.

It is possible, readers came away from my first two postings with the impression I believe nothing is wrong with Scrum –  it’s perfect – and dubbed me a Scrum zealot of some sort.  Well in case you did, maybe this last post will leave you with a different impression.  Maybe.

Scrum is defined as: a framework within which people can address complex adaptive problems, while productively and creatively delivering products of the highest possible value.  This definition would seem to imply that as long as a team is in the business of delivering products, Scrum should be a fit.  Is it really as simple as that?

I rarely hear productive dialogue around the scenarios when and where Scrum (the process framework itself) isn’t a good fit for a team developing products, so I’ll attempt to start some dialogue here.  In my opinion, for Scrum to be a fit, the holistic nature of the work the team is doing needs to be in alignment.  When could the nature of the work not be a fit?  I’m going to attempt to identify these situations via the lens of the VUCA framework.  VUCA stands for volatility, uncertainty, complexity and ambiguity.

Volatility

Volatility refers to how much change is occurring.  In the context of Scrum, I see this as how does the rate of change impact how we plan.  Let’s consider Team A that gets to their Sprint Review and identify they did not deliver what they had planned because they had to deal with new work that emerged during the Sprint (not because they planned poorly which many teams do!).  Or maybe every Sprint there is a discussion around canceling the Sprint because the Sprint Goals keep changing due to emerging market factors.  Maybe emergency customer requests or defects have to be addressed in order to keep major accounts. The nature of their work is so volatile, they cannot plan any further out than maybe a day or two.  In this case, Scrum may not be an appropriate fit.

Uncertainty

Uncertainty is focused on the degree of predictability we have.  Scrum works well for handling unpredictability around what will satisfy the customer because the framework requires customer engagement throughout the process. On the other hand, Scrum expects certainty around what the team will be working on in any given Sprint and the goals they have.  If we cannot arrive at a level of certainty around what exactly we will be working on in a Sprint (maybe due to volatility), Scrum may not be a fit.

Conversely, if there is little uncertainty about what needs to be delivered such that (for example) customer engagement has little impact on delivering a work product that meets the desired goals or if the team just needs to plow through a laundry list of items and ordering (or prioritization) is not particularly important or if potentially releasable increments are not needed during the duration of development, then Scrum may not be valuable as a framework for delivering products.

Complexity

Agile frameworks and methods are designed to deal with the complexity involved in the development and delivery of products.  But what if your work is not complex?  What if it’s simple or complicated?   While complicated work is difficult/hard, we know the steps we need to take and the “end is largely known from the beginning”.  Simple work is like complicated work except we don’t have the ‘difficulty’ property present.  In either case (with simple or complicated work), the amount of discovery that occurs while we are doing the work is minimal and hence the benefits of Scrum itself is probably minimal.  If we don’t need to inspect and adapt, then the events Scrum provides for this are probably overhead.

Ambiguity

Ambiguity has been described as the presence of “unknown unknowns”.  It seems to me “unknown unknowns” are always present except in the simplest of endeavors.  The question we need to answer is this: how much of our work is characterized by ‘unknown unknowns’?  It’s my personal opinion that Agile (and hence its frameworks like Scrum) work best in situations where the work has a balance of ‘known knowns’, ‘known unknowns’ and ‘unknown unknowns’.  When ‘unknown unknowns’ is the predominant characteristic of the work, there may be better frameworks for managing such work.

So yes, it’s quite possible, that the nature of a team’s work may be such that the Scrum framework is not the best framework for managing the work. We can only come to this conclusion when we take the time to objectively profile our work.  Objectivity is key here and worthy of another post.

It’s at this juncture I need to remind us that Scrum (and Agile by extension) are not THE goal.  Your organization has business goals that are extremely important.  Your team has hopefully been asked to own making a contribution towards these goals.  Agile (and its frameworks and methods) may enable you achieve those goals in a manner that is fulfilling and rewarding and yet it is important that we continuously assess, with a critical eye, the frameworks and methods we have chosen and make sure they remain a fit.

I find many teams and organizations give up on Scrum because it makes them uncomfortable as it shines a light on both team and organizational dysfunctions, NOT because the nature of their work is not a fit.  To make matters worse, they then distort Scrum by redefining it to meet how they would like to work.  When we give up on Scrum or any other frameworks for the wrong reasons, we have placed a self-imposed cap on what we could have accomplished.  Our new approach doesn’t really solve our problems; it just introduces a brand new set.  Isn’t that a crying shame?

I hope these three articles on Scrum provide you with thinking tools that help you choose ways to work that are both fulfilling and extremely effective.

Agile vs Waterfall. Why?

Every week it seems like I read, see or hear someone positioning Agile as some sort of opposite of Waterfall.  I understand why people do this – it’s a shortcut of sorts.  However I feel it can be extremely shortsighted and a potential conversation-killer, especially when people in positions of authority or of significant influence are the ones doing these comparisons.  It’s even worse, in my opinion, when Agile practitioners do it.

Waterfall, as a software development approach, is largely a straw-man.  If you don’t believe me, read the source that seems to have started it all. And yet, even if you believe Waterfall is real, recognize that it is primarily (if not solely) focused on the process aspects of developing and delivering software.

Agile, on the other hand, and in my opinion, is a cultural approach (future posts will explore this) to software development.  This culture has been adopted outside of software development.  Agile is not only concerned with process.  It is concerned with both the people and the practices involved in software development.

So while Agile is about incremental and iterative software development (which is why its often pitted as an opposite of the straw-man that is Waterfall), that’s not where it ends. Agile informs how we organize to address our software development challenges.  Agile addresses the importance of engaging those who will use our solutions during the development process (as does Royce’s paper by the way).  Agile emphasizes paying attention to our technical practices and improving as we progress.

So instead of comparing Waterfall with Agile – which reminds me of people who compare the United States with Africa (thinking Africa is a country) and I’ve met quite a few of those people in my day – let’s have meaningful conversations around the aspects of the way we work that we’d like to improve.

What’s Wrong With Scrum – Part Deux

Five months (and a bunch of Agile thoughts) later, its time to continue to address the challenges organizations have with Scrum.

In my last post on this subject, I focused on the Scrum Master role, the current certification model and the failure of people who are in supposed Agile change agent positions to really dig deep and own their craft.  I got a bit of feedback from some people indicating I was a bit too harsh.  I’m sorry if I offended anyone.  The remainder of this post may also be offensive to some. Please proceed with caution.

This post focuses on the situation where the existing organizational culture not being a fit for Scrum (and Agile).  The values of each subculture are not shared or complementary and the behaviors are at odds with each other.

Many organizations that want to “go” Agile (by way of Scrum for example) don’t realize that they are potentially embarking on a cultural change that the organization may not be prepared for.   A lack of awareness and unpreparedness eventually causes conflict and failure. I’ll present (as examples) three Agile cultural values that are often at odds with the prevailing cultural values in many organizations.

Planners Plan, Doers Do

I’ve shared my thoughts on the role of the manager in knowledge work and human-based systems.  You can check out all my posts on this here.  Many (if not most) managers have been informally trained to be managers from the school of Taylorism with the belief that managers “plan” and the non-managers “do”.  A manager is only successful if she or he is good at telling people exactly what they should do, how they should do it and ensuring it gets done.

Scrum teams are self-managed. It’s been my experience and observation that many organizations find themselves at odds with the concept of self-managed teams because it challenges the conventional role of a manager in their organization.

The organization has little need for Scrum Master’s who focus on helping establish self-organizing teams.  Managers want someone who will answer to them and “drive the team to success”. But in order to have the moniker of an “Agile organization”, they label these individuals as Scrum Masters.  It doesn’t take long for Scrum to implode in such environments.

Get Off My Lawn

Many organizations struggle with establishing cross-functional teams that work across functional silos.  The development of products is often more than just a ‘tech org’ endeavor and yet I often hear people (in and outside of technology) suggest that Scrum (and Agile) is just for technology.  As a result of this, silos are encouraged and reinforced.  People spend hours debating roles and responsibilities and then developing RACI matrixes to govern their interactions.  Working as a cross-functional team throughout the process of developing products simply doesn’t occur.  Instead, work is handed off from department to department and complex processes are developed to manage these handoffs.  If your organization is not committed to cross-functional teams and believes deeply in silos, Scrum has a little chance of helping the organization change.

It’s Just Complicated (Not Complex)

Many individuals in organizations will agree (on paper) that product development is inherently complex and yet their behaviors would indicate that they view software development as being a “complicated”.  What’s the difference between complicated and complex?  The answer to that is probably worthy of it’s own post however I will try to illustrate with simple examples.

A jigsaw puzzle of 1000 pieces is complicated.  Even though it’s difficult and may take hours to finish, there is still only one outcome that indicates that we have solved the puzzle. With study we can determine the best way to solve the puzzle and possibly automate its solution.  On the other hand, a soccer match is complex.  There are many interactions that are occurring and multiple outcomes are possible.

Product development is complex.  Scrum as a framework realizes this and has “rules” and events that are intended to guide work that is occurring in a complex system.  Organizations that spend a lot of time creating plans and then attempt to follow them as closely as possible are treating product development as if it’s complicated.  Organizations that focus primarily on efficiency (over effectiveness) are essentially considering the creation of products to be a predictable and repeatable process.  Scrum will struggle in environments where this is the case.

Culture Matters…

These are just three examples of where the Scrum subculture could be in conflict with an organizations pre-existing subculture. I actually expect such conflicts to exist.  The question is this: Is your organization ready to embark on a cultural change journey or is it wedded to its current culture?  If there is no intention for meaningful change, then it’s very likely that there is nothing with Scrum. On the other hand, there may be something wrong with your organization.  I’ll let you decide that.

What’s Wrong With Scrum?

What’s wrong with Scrum? In my opinion, nothing really. That’s right, nothing.

But Scrum isn’t perfect. Okay, show me what is, please.

But Scrum is missing some key properties. Okay, show me what has ALL the key properties you believe are required for successful software delivery.

But Scrum doesn’t work in MANY situations. You just might be right and I just might agree with you. But does that mean that something is wrong with Scrum? I think not – and I’m not a Scrum defender (even though it seems like I’m becoming one these days).

There are at least a three reasons that I can quickly think of as to why Scrum will NOT work in a given context (I assume you’ll be able to add to the list):

  1. The nature of the work is not a fit for Scrum.
  2. The organizational culture is not a fit for Scrum.
  3. The individual(s) responsible for guiding Scrum adoption are lacking in skill and ability.

In a very unusual turn for me, I want to explore bullet point 3 first, that is, I want to focus on the individual(s). I rarely do this but in this case I believe it to be important, so with that being said, let’s talk about the Scrum Master role.

The Scrum Master Role

The Scrum Guide says this about the Scrum Master:

The Scrum Master is responsible for ensuring Scrum is understood and enacted. Scrum Masters do this by ensuring that the Scrum Team adheres to Scrum theory, practices, and rules.

The Scrum Master is a servant-leader for the Scrum Team. The Scrum Master helps those outside the Scrum Team understand which of their interactions with the Scrum Team are helpful and which aren’t. The Scrum Master helps everyone change these interactions to maximize the value created by the Scrum Team.

But actually doesn’t stop there, it goes on to say the following:

Scrum Master Service to the Product Owner

The Scrum Master serves the Product Owner in several ways, including:

  • Finding techniques for effective Product Backlog management;
  • Helping the Scrum Team understand the need for clear and concise Product Backlog items;
  • Understanding product planning in an empirical environment;
  • Ensuring the Product Owner knows how to arrange the Product Backlog to maximize value;
  • Understanding and practicing agility; and,
  • Facilitating Scrum events as requested or needed.

Scrum Master Service to the Development Team

The Scrum Master serves the Development Team in several ways, including:

  • Coaching the Development Team in self-organization and cross-functionality;
  • Helping the Development Team to create high-value products;
  • Removing impediments to the Development Team’s progress;
  • Facilitating Scrum events as requested or needed; and,
  • Coaching the Development Team in organizational environments in which Scrum is not yet fully adopted and understood.

Scrum Master Service to the Organization

The Scrum Master serves the organization in several ways, including:

  • Leading and coaching the organization in its Scrum adoption;
  • Planning Scrum implementations within the organization;
  • Helping employees and stakeholders understand and enact Scrum and empirical product development;
  • Causing change that increases the productivity of the Scrum Team; and,
  • Working with other Scrum Masters to increase the effectiveness of the application of Scrum in the organization.

I took the liberty to copy directly from the Scrum Guide because I consider this to be important stuff.  (Another good resource is the 42 tasks).

A careful review of this role description hopefully makes it obvious to the reader the vital nature of the Scrum Master role in Scrum.  If you currently serve in the role of a Scrum Master, what does your self-assessment look like taking into consideration the Dunning-Kruger effect? Be honest with yourself. Did you understand what empirical product development was (before you googled it right now)?

I posit that one of the big challenges to Scrum in the industry is the inability of many Scrum Masters working in different organizations to be effective. I might be wrong about this but after working with a lot of Scrum Masters, interviewing many more and participating in online discussions, I believe there is some truth to this.

So why is this the case? There are few reasons in my opinion and yet today I just want to look at three that focus largely (surprisingly) on the individual. (There are organizational factors that contribute to the low degree of skill and ability as well, but those will be addressed in a subsequent post.)

 

The Scrum Master Certification

For a handsome fee and a two-day training session, you can become, wait for it, a Certified Scrum Master. The Scrum Alliance says that: As a CSM, you will be able to fill the role of Scrum Master or Scrum team member.

I am yet to understand how a 2-day training session equips anyone to effectively perform the services required of a Scrum Master (except they were already a practitioner).  I would love for someone to bring clarity around this statement. People tell me that they are going to take the class so they can act in the role. Recruiters present candidates that are CSM’s. Managers look to hire CSM candidates. The services are entrusted into the hands of people who are not qualified to perform the role (but they are certified).

The way the CSM certification seems to have been positioned leads people to believe that just taking the class qualifies them for the role and that there is really nothing much to it. This may not be the intent of the Scrum Alliance, but it is definitely (at least) an unintended consequence. Your CSM trainer might even be good enough to stress what is needed to be really effective and yet I’m not sure that makes a difference to many people. I suppose I could be wrong. My interactions would suggest that I’m not.

As Rob Myers and I discussed the other day:

 

Shallow Commitment to Craft

What’s sad about this point is that I encounter many Scrum Master’s that are more committed to enforcing myths in Scrum (as I pointed out in my last post) than they are to improving their craft and growing in their knowledge. They are victims of a “little learning”. As I asked in my last post, when is the last time you as a Scrum Master reviewed the Scrum Guide? Do you know that the Scrum Master doesn’t actually have to attend the Daily Scrum? How about the Agile Manifesto? What’s the last “people coaching” book you read? How about the last dialogues you participated in on a Scrum forum? How much time are you investing in growing your ability?

Scrum Masters that do not continually evolve their capability actually retard the adoption of Agile in the organization. Facilitating events is only the tip of the iceberg. There is more to the role than meets the eye.  Go back and read the role description again.

If you’re really serious about being a Scrum Master then the Certified Scrum Professional certification may be something you would want to look into.

 

Poor Examples

Albert Einstein is attributed with saying the following:

Setting an example is not the main means of influencing others, it is the only means.

I find that many people feel they can be effective Scrum Masters because of the examples of other Scrum Masters they see around them on a daily basis. Examples that are extremely poor. Examples that suggest that one can perform the role after a two-day training session. Examples that suggest that the role is only about facilitating meetings and removing impediments. Examples that suggest that the role is just another flavor of a project manager (or release manager or…well you get the point).

I’ve focused on the Scrum Master role here because Scrum is talked about quite a bit. I do need to point out however that these observations are not exclusive to the domain of Scrum. I find them quite applicable to Agile (as a value system) and any of its methods and frameworks that are being used today. Hence, this post is not just focused on Scrum and the Scrum Master role.  Its scope includes all Agile Team Leader-type roles.

At the beginning of this post, I presented 3 reasons why Scrum may not work in a particular context. Interestingly enough, an ineffective Scrum Master will not be able to identify reasons 1 and 2 presented above. It disastrous having people who don’t have the requisite skill guide Agile adoption. I’ve encountered many certified Scrum Masters who want to make every thing they do fit into the Scrum framework. For them, they have a hammer and everything is a nail. The inability to take a step back and look at things through the eyes of the system is severely lacking. Sometimes, Scrum is not initially a fit and it’s in the best interest of the teams and organizations to evolve so that Scrum does become a fit. At other times, something else (instead of Scrum) needs to leveraged. The ability to discern and then influence appropriately is critical and yet I find those skills to be significantly lacking in many Agile Team Leaders.

So for Scrum Masters or for those intending to be Scrum Masters (or similar Agile Team Leader roles), take a few minutes to understand the significance of the contribution you are supposed to be making within your organization. Identify where you have gaps and make a conscious effort to address them. Find someone in your organization or the Agile community who can be your coach or guide. Join Twitter and follow some Agile thought leaders. Join one of the Agile groups online. Engage in conversation, debate and learning.

Don’t be the reason people say something is wrong Scrum or Agile.

The Traditions of Men

Ye leave the commandment of God, and hold fast the tradition of men.  Mark 7:8 (ASV)

I never in my wildest dreams would have thought I would ever write a post  “defending” Scrum especially considering that I’m not a Scrum advocate (ask the folks I work with, I think they’ll confirm that 🙂 ). Then again, Scrum definitely doesn’t need me to come to its defense and yet I’m compelled to share a few thoughts down – at least for posterity’s sake.

Every week I come across a post indicating why Scrum did not work for a team and how they switched (or evolved as some describe it) to something else.  (I come across similar posts on diets and soccer formations as well). First of all, I’m happy that the teams found something that worked for them.  I hope that if they wanted to work in Agile-based manner, their new approach allows for that.  After all, “we are uncovering better ways…” aren’t we?

However, I find it irresponsible for these people (and other Agile thought leaders) to blame Scrum for certain things that occur especially when these things are really not Scrum things.  It’s important to remember that Scrum is a framework that needs to be filled out if software is actually going to be developed and delivered.  Scrum, via the Scrum Guide, doesn’t provide much (explicit) guidance on how to actually develop and deliver software.  It provides a framework for managing the software development process.

Before you begin to debate this with me, let’s simply agree that the Scrum Guide is the official definition for Scrum.  In my opinion, any other text on Scrum is the author(s) description of how the framework was implemented and what additional techniques or practices were found beneficial as the framework was leveraged. Unfortunately, some of these techniques (promoted by Agile and Scrum thought leaders) have over time become characterized and adopted as Scrum rules when they are really techniques used with Scrum.  They are traditions.  Let me provide a few examples.

Estimation

Scrum currently says nothing about how estimation should be done and yet it would seem that Story Points estimation is considered THE Scrum estimation approach.

Delivery Cadence

Scrum requires that a “potentially releasable product increment” be created by the end of the defined time-box.  I don’t see anything in this requirement that prevents a team from releasing sooner if both they and their customer can handle it.  I also find this supposed limitation to be a bit hilarious.  I mean, it wasn’t it but a few years ago that many organizations were struggling with delivering software more than one time a year!

User Stories

Is this in the Scrum Guide?

Daily Scrum

These days it’s become quite chic to state that instead of focusing on the individuals doing the work, we should focus on the work itself.  In other words, focus on the baton, not the runner.  The criticism of the Daily Scrum then is that every day, it’s the same (paraphrased) three questions that are asked:

  1. What did I do yesterday?
  2. What do I intend to do today?
  3. What impediments do I possibly have?

Unfortunately, these three questions (while represented in the Scrum Guide) are removed from the larger context for which they are a part of. Here are the first two lines of this section in the Scrum Guide:

The Daily Scrum is a 15-minute time-boxed event for the Development Team to synchronize activities and create a plan for the next 24 hours. This is done by inspecting the work since the last Daily Scrum and forecasting the work that could be done before the next one.

In fact, if you are interested, take the time to read that section (and the entire Guide) on your own.  Tell me where it requires you to “stand up”!

These are a few examples of things I often read people say “Scrum made them do…”  So is this a post to convince you to use Scrum?  Nope.  First of all, I don’t believe it’s always a fit for the type of work a development team might be doing.  Secondly, context matters and depending on your context (of which I have no idea of), something else might be better. However, I will challenge you to read for yourself what the rules of Scrum are so you can separate those rules from the traditions of men and make informed decisions.  Study to show thyself approved.

When I have these conversations in social settings, its suggested that the Scrum Guide has changed over the years.  When did change become a bad thing?  What happened to inspect and adapt?  Show me a method or framework that doesn’t evolve and I’ll show you a dead method or a dead framework.  The real problem is that many (maybe even most) Scrum Masters that I interact with don’t evolve beyond what they may have learned in their 2-day CSM course 7 years ago.  Ask them the last time they actually looked at the Scrum Guide or how many Scrum books they have read. The answer is often quite disturbing and disappointing.  (By the way this point extends beyond Scrum and into Agile (as an approach to software delivery).  The cargo-cult behavior of people who promote “inspect and adapt” can be quite disappointing).  Don’t even get me started on managers who try to impose Scrum (or any other Agile approach for that matter) in their organizations without having a clue.

Scrum is a framework that your team can leverage (if it’s a fit) to develop and deliver products in an Agile manner. It’s not perfect because nothing really is. There is no silver bullet and that includes your new approach as well.