Cause and Effect – Is It Really That Simple?

So this post has been on my mind for a while, but something happened at work today….

It would seem to me that it’s human nature to explain occurrences in the simplest manner possible, often in the form of linear “cause and effect” relationships.  You know, after you kick a ball, you realize the that the kick (cause) led the ball to move (effect) and so we are always looking for that “kick” that led something to happen.  For example, why did we find fewer defects this sprint?  Well, it’s because we went from a 3 week sprint to a 2 week sprint.

Is it really that simple?  I don’t think so.  I’m reminded of the phrase “correlation does not imply causation“.  In the domain of systems thinking (and every one responsible for the system should have a basic understanding of systems thinking), we realize that there are often many variables at play in the system.

Take the “fewer defects” example I mentioned earlier.  While it is true that the sprint duration was shortened, is it possible that other conditions changed as well?  Was the team now a bit more familiar with the work?  Was their less pressure because they committed to less in that particular sprint? Was this work less complex than the work before? Was technical expertise that had been missing before made available?  What about the conditions we are not aware of? I mean, we can’t be aware of everything can we?

The additional challenge we have is that we rarely have the luxury of repeating our experiments.  Every time we try something new, the conditions are different.  How many times do we get the opportunity to develop the same feature repeatedly just to capture how effective a change is?  I assume rarely, if ever.  Our laboratories don’t allow us to reset and run the exact same experiments over and over.

So where does this leave us? (Especially those of us who consider ourselves to be expert problem solvers!).  Does this imply that we cannot attribute cause to the things that happen around us?  I think we need to be a bit more respectful of the complexity of the problems we often face.  We need to understand that the underlying structures and conditions continually impact one another.  To use systems thinking parlance, structures reinforce and balance one another. There are often many factors at play and we should be careful latching onto a single one as THE (only) cause.  This complicates things for us in some regards, but put us in a position where we can actually dissolve problems.

It is rarely as simple as we often make it out to be.

PS.  Some links to my posts on systems thinking.

Have You Seen The System Lately

Appearances Can Be Deceiving

Advertisements

Is Your Oven The Right Temperature?

A large number of organizations attempt to to have their software developments go “Agile” for a variety of reasons.  The sheer number of discussions in Agile discussion groups is a reflection of this craze. Many of these attempts fail and fail miserably.

A careful read of the Agile Manifesto makes it clear (to me) that the signatories anticipated a certain type of environment  for Agile software development to thrive in.  Consider the following principles:

  • Principle 4: Business people and developers must work together daily throughout the project.
  • Principle 5: Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.
  • Principle 8: Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.
  • Principle 11: The best architectures, requirements, and designs emerge from self-organizing teams.

Each principle implies a lot about the type of ecosystem in which Agile software development has a chance of being successful. Many organizations that are trying to do or be Agile are only aware of a set of codified practices and are oblivous to the principles that I just mentioned.  Unfortunately, the Agile manifesto itself doesn’t explicitly address the ecosystem needed for Agile to have a chance of being successful.  Organizations that are interested in Agile software development need to actually look to other sources for information on how to cultivate such an environment. Sources such as Argyris, Deming, Drucker, Weinberg, Ackoff, Senge, Hackman, Pink, Mintzberg (to list a few) are a great place to start for anyone who is interested. If you are a manager and most of these names are not familar to you, I would strongly suggest that you look them up.  In fact, if you get nothing else out of this post, I hope you just start a reading list.

The fact, however, is that many (if not most) large organizations have systemic structures (politics, hierarchy, funding models, appraisal systems, control systems etc etc) in place that are at odds with the structures needed for Agile software development to actually succeed.  Several thought leaders in the Agile space have largely given up on “the large organization” and have suggested that it’s waste of human potential trying to create the conditions for Agile to succeed in such environments.  Others have fused other methods such as Lean Thinking and Theory of Constraints to create a more holistic approach.  I’ve especially found the work of Bob Marshall (@flowchainsensei) and Rightshifting to be particular motivating for me in this regard.  Let’s take a look at few examples of organizational structures typically found in the large organization and the challenges they present to Agile.

Political Structures

The larger an organization, the more of an influencer “politics” is.  People protect their territory (which often includes other people) at the expense of the overall good of the organization.  Transparency is lost as a result of folks continually manuvering. Alliances form beind the scenes and nothing gets done.   In a highly political charged environment, Agile will struggle.

Funding Structures

If your organizations sees technology as an auxillary function (a cost center) that provides utility services then technology will be funded in a such a way.  Very often that funding leads to the execution of projects.  I’ve written in the past about projects and products so I won’t rehash that here.  But, in a nutshell, if your organization delivers projects, it may be challenging (or impossible) for your organization to be Agile.  (I realize that the manifesto uses the word, project. Don’t let that mislead you.)  On the other hand, if your organization sees technology as a strategic lever with the understanding that product development is primarily an exercise in discovery, then Agile may work for you as you’ll find that you work be doing projects.

Accounting Structures

This is related to the previous structure but it still warrants being called out explicitly.  Cost accounting or throughput accounting?  Your accounting structure can impact your ability to truly work in a Agile way. What accounting structures do you have in place?

Hierarchical Structures

Traditional organizational structures are largely Taylorist in nature. Managers (and this includes all titles involved in the management) are brought in to ensure that people (very well educated and compensated people mind you) are doing the right things.  I’ve also written about this recently and so I won’t repeat what I said.  I’m saddened by how many managers are unaware of the role of the manager in an organization.  Your organization will struggle with Agile if the organizations view on hierarchy and management remains Tayloristic.  If you don’t plan on making a true commitment to enable self-directed teams (see Hackman and Kimball) and network-based structures, then Agile is probably not for you.

Decision Making Structures

Heavily centralized or heavily decentralized ? What is prevalent in your organization? Do people have to run everything up the chain of command for approval before they can actually do their jobs?  Is information centralized in a “special few” (think Director of Agile Coaching) roles in the organization and nothing happens if they are not involved?  Is leadership at all levels promoted or does your organization just pay lip service to such an idea.  Are ideas valuable based on the title of the individual who came up with the idea or are all ideas considered equally?  If your organization doesn’t make a concerted effort to push information out to as many people as is possible to enable quick decision making, Agile may not be for you.

It has been said that “culture eats strategy for breakfast”.   A study of the Toyota Production System reveals how important the culture of an organization is.  I believe the same to be true for any organization interested in the Agile thing. Having teams try to do it on their own is (a) a local optimization and (b) not Agile.  An organization interested in Agile (and continuous improvement in general) needs to make a commitment to changing it’s existing structures if it realizes that its structures impede its ability to “do Agile”.

Your can have the perfect cake batter, but if you don’t bake your cake at the right temperature, you won’t have a cake.  Is your oven (organization) at the right temperature?  If no, what are you doing about it?  Can you do anything about it? Better yet, should you do anything about it?

Have You Seen The System Lately?

Have you seen the system?  Let’s start with a little story..

A father goes into his children’s bedroom to wake them up and get them ready for school. As he pulls the covers off of his daughter, he notices that her eyes seem a little cloudy.  He finds a thermometer and checks his daughter’s temperature.  It reads 101.7 F, she definitely has a fever.  His daughter’s school has a very clear policy that children should be fever free for 24 hours before being brought in to school.  This means Dad is going to have to call his employer and let them know that he won’t be able to make it in because he has a sick child he needs to take care of.  Seems pretty straightforward doesn’t it?  But is it?

Dad’s employer keeps track of how often their employees call in to say that they will not be able to make it in for one reason or the other.  Employees are dinged for not “showing up” and it affects their yearly raise and bonus.  It just so happens that Dad is one ding away from some significant repercussions.  On the other hand, the company does nothing if an employee comes in to work but has to leave because of a situation.  As long as you show up (even if its for just one hour) you are good.

Dad being fully aware of his situation at work, decides to take his daughter in to school fully expecting to be called to come and pick her up in just a few hours. In his mind it’s a win-win situation because he’ll still be able to take care of his daughter and not get dinged for not showing up to work.  What he doesn’t know is that his daughter will go on to spread her sickness to a couple of the other kids in her class.  He also doesn’t realize that the parents of these children work for organizations with the same “call in” policy as his and are also going to bring in their sick children to school.  His decision to take his daughter in to school has just triggered weeks of kids being sick in the class. In fact, his daughter will fall sick again a few weeks from now.

So what’s the system in this story?  Can you identify it?  Many of us might be quick to judge the father harshly for taking his daughter in to school but when you consider the structure(s) that influenced his behavior, you realize that there are a set of interconnected elements working together even though they may not be obvious to the different actors involved.

Wikipedia describes system thinking as:

…the process of understanding how things, regarded as systems, influence one another within a whole.

Dr Deming defined a system as:

…network of interdependent components that work together to try to accomplish the aim of the system

Peter Senge describes as system as:

webs of interdependence.

Russ Ackoff on the system:

A system is more than the sum of its parts; it is an indivisible whole. It loses its essential properties when it is taken apart. The elements of a system may themselves be systems, and every system may be part of a larger system.

and Ackoff advises that we should

…focus on the interactions of the parts rather than their behavior taken separately.

A team struggling with quality thinks that better or more QA people need to be added to the team or the iteration needs to be longer.  An organization struggling with product delivery has its development team go on an Agile transformation.  Confusion exists around what people should be doing, so an exhaustive writeup needs to be done to clarify this for everyone.  An employee is underperforming so they must be slacking and are put on a performance improvement plan.  There is no shortage of reactive analytical responses that show up in the  workplace on a day to day basis.  Yet these responses really don’t solve anything over the long-haul.  They are at best, cheap band-aids.  We are solving problems with the same mindset that we used to create them (Albert Einstein).  We fail to identify the components and the interactions between these components within the system.

When we take a systems view to addressing problems, we make an attempt to see as much of the whole picture as is possible.  We realize that our development team is part of a large product delivery organization.  We understand that the employee struggling with performance problems just had a new baby and is not getting much sleep at night but is on our most complex project.  We realize that product management is making commitments that are leading our developers to cut corners ultimately leading to poor quality.  A systems thinking view to problem solving causes us to look beyond where we “think” the problems are to identifying the different components and the interactions between these components that is actually leading to our problems.

Let’s be humble enough to realize that we may never see the whole system but at least we’re trying to.  Don’t be deceived, taking a systems view is by no means the natural thing to do because we are taught (from an early age) to break things up and address the individual components independently. I meet many people who say that they are systems thinkers but really cannot see beyond their nose in problem dissolving.  They just don’t know how to do it because they haven’t learned how to.  I’m still learning.  If you haven’t read the works of Deming, Senge, Ackoff and Weinberg (for example), you may want to start there.  We need to learn how to become system thinkers if we truly intend to dissolve our system problems.

Thanks to Bob Marshall (@flowchainsensei) for inspiring this post.

Appearances Can Be Deceiving

I’m a Theory Y guy.  I may be näive in my outlook but I’m just a Theory Y guy.  I believe that most (not many) people want to do a good job wherever they are.  So what do we do when we observe that someone is not doing a good job?

Let’s say that Chidi (a guy I know) is having a difficult time at work, he’s struggling to meet expectations of his co-workers. Even though it seems his trying really hard, he’s struggling to deliver.  What could be the problem?  How can we help him? Well if we view his performance through the lens of Theory Y,  could we identify the reasons why he is not doing so?  I can think of at least 3 (I’m sure you can add more):

  1. Chidi doesn’t know HOW to do a good job – competence.
  2. Chidi doesn’t know WHAT a good job is – understanding.
  3. Chidi knows how to do a good job but the environment is preventing him from doing so – system.

Let’s explore these a bit.

COMPETENCE

Competence means the ability to do something successfully.  Does Chidi have the right skill for the job?  Was he given the appropriate education to allow to be competent at the job? If this is a new position, is a competent mentor mentoring him?  How is the organization taking deliberate steps in improving his competence?

UNDERSTANDING

Does Chidi understand the expectations of the job that the organization has given him?  Have they been made clear to him?  Was it discussed with anyone?  Who is available to answer his questions if he has any?  Was he told to “figure it out on his own”?  Have the outcomes expected from him been connected to the strategic vision of the organization? Does he understand the strategic vision?

SYSTEM

Do the structures (Senge) in place enable Chidi to succeed? Is the system working in his favor or against him?  Is he doing his best in a poorly designed system?  Have problems with the system been identified?  Is anyone (management) fixing the system?  Is management tampering (Deming) with the system e.g. adding more process?  Does management even know and understand the system?

In my experience as a “manager”, competence and understanding often prevent people from excelling at their jobs.  However, I have found that the system is the biggest inhibitor to people’s success.  It was Dr. Deming, who suggested that 85% of a workers effectiveness is determined by the system.  In addition, Lewin’s formula reminds us that the behavior of an individual is dependent on the person AND their environment.  This is another reason why performance appraisals can be largely ineffective.

At the beginning of this post, you may have identified  yourself as a Theory Y type person.  Is your espoused theory the same as your theory-in-use?  When people or teams have challenges, how do you look to help them?  How does your ladder of inference inform you? Do you just see the tip of the iceberg and act on it or do you go deeper to truly understand why an individual or team is challenged?

At the end of the day, it’s very likely that Chidi is really not the problem, after all, he really wants to do a good job. You see my friends, appearances can be deceiving.