A large number of organizations attempt to to have their software developments go “Agile” for a variety of reasons. The sheer number of discussions in Agile discussion groups is a reflection of this craze. Many of these attempts fail and fail miserably.
A careful read of the Agile Manifesto makes it clear (to me) that the signatories anticipated a certain type of environment for Agile software development to thrive in. Consider the following principles:
- Principle 4: Business people and developers must work together daily throughout the project.
- Principle 5: Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.
- Principle 8: Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.
- Principle 11: The best architectures, requirements, and designs emerge from self-organizing teams.
Each principle implies a lot about the type of ecosystem in which Agile software development has a chance of being successful. Many organizations that are trying to do or be Agile are only aware of a set of codified practices and are oblivous to the principles that I just mentioned. Unfortunately, the Agile manifesto itself doesn’t explicitly address the ecosystem needed for Agile to have a chance of being successful. Organizations that are interested in Agile software development need to actually look to other sources for information on how to cultivate such an environment. Sources such as Argyris, Deming, Drucker, Weinberg, Ackoff, Senge, Hackman, Pink, Mintzberg (to list a few) are a great place to start for anyone who is interested. If you are a manager and most of these names are not familar to you, I would strongly suggest that you look them up. In fact, if you get nothing else out of this post, I hope you just start a reading list.
The fact, however, is that many (if not most) large organizations have systemic structures (politics, hierarchy, funding models, appraisal systems, control systems etc etc) in place that are at odds with the structures needed for Agile software development to actually succeed. Several thought leaders in the Agile space have largely given up on “the large organization” and have suggested that it’s waste of human potential trying to create the conditions for Agile to succeed in such environments. Others have fused other methods such as Lean Thinking and Theory of Constraints to create a more holistic approach. I’ve especially found the work of Bob Marshall (@flowchainsensei) and Rightshifting to be particular motivating for me in this regard. Let’s take a look at few examples of organizational structures typically found in the large organization and the challenges they present to Agile.
The larger an organization, the more of an influencer “politics” is. People protect their territory (which often includes other people) at the expense of the overall good of the organization. Transparency is lost as a result of folks continually manuvering. Alliances form beind the scenes and nothing gets done. In a highly political charged environment, Agile will struggle.
If your organizations sees technology as an auxillary function (a cost center) that provides utility services then technology will be funded in a such a way. Very often that funding leads to the execution of projects. I’ve written in the past about projects and products so I won’t rehash that here. But, in a nutshell, if your organization delivers projects, it may be challenging (or impossible) for your organization to be Agile. (I realize that the manifesto uses the word, project. Don’t let that mislead you.) On the other hand, if your organization sees technology as a strategic lever with the understanding that product development is primarily an exercise in discovery, then Agile may work for you as you’ll find that you work be doing projects.
This is related to the previous structure but it still warrants being called out explicitly. Cost accounting or throughput accounting? Your accounting structure can impact your ability to truly work in a Agile way. What accounting structures do you have in place?
Traditional organizational structures are largely Taylorist in nature. Managers (and this includes all titles involved in the management) are brought in to ensure that people (very well educated and compensated people mind you) are doing the right things. I’ve also written about this recently and so I won’t repeat what I said. I’m saddened by how many managers are unaware of the role of the manager in an organization. Your organization will struggle with Agile if the organizations view on hierarchy and management remains Tayloristic. If you don’t plan on making a true commitment to enable self-directed teams (see Hackman and Kimball) and network-based structures, then Agile is probably not for you.
Decision Making Structures
Heavily centralized or heavily decentralized ? What is prevalent in your organization? Do people have to run everything up the chain of command for approval before they can actually do their jobs? Is information centralized in a “special few” (think Director of Agile Coaching) roles in the organization and nothing happens if they are not involved? Is leadership at all levels promoted or does your organization just pay lip service to such an idea. Are ideas valuable based on the title of the individual who came up with the idea or are all ideas considered equally? If your organization doesn’t make a concerted effort to push information out to as many people as is possible to enable quick decision making, Agile may not be for you.
It has been said that “culture eats strategy for breakfast”. A study of the Toyota Production System reveals how important the culture of an organization is. I believe the same to be true for any organization interested in the Agile thing. Having teams try to do it on their own is (a) a local optimization and (b) not Agile. An organization interested in Agile (and continuous improvement in general) needs to make a commitment to changing it’s existing structures if it realizes that its structures impede its ability to “do Agile”.
Your can have the perfect cake batter, but if you don’t bake your cake at the right temperature, you won’t have a cake. Is your oven (organization) at the right temperature? If no, what are you doing about it? Can you do anything about it? Better yet, should you do anything about it?
I can barely hear what you are saying.
Yes, your actions are the truest indicator of what you believe; a revelation of the theories that lead to your behavior.
Based on the work of Chris Argyris and Donald Schon, we understand that we all have our espoused theories, for example, let’s say I believe that teams should be allowed to make tool decisions, but then I turnaround and make the decision for the team. My behavior actually reveals the theories that govern how I act. It exposes our mental models which we are often unaware of – yes very often we have no clue. Examples of the difference between espoused theory and theory-in-use abound. Ever heard “there is no i in team” followed by action that says “who was responsible for this “? Or how about, “let’s empower the team to fix their problems” followed by action that says “just give me control and I will fix the problems of the team”?
Unfortunately, many in leadership positions are unaware of the fact that how they say they will/would act doesn’t match how they actually act. Not discussing this inconsistency leads to a huge tax on trust within the organization. “Leaders” are perceived as being deceitful and conniving. The erosion of trust impacts the ability of the organization to deliver and meet its goals. It’s always a pity to see an organization working against itself.
Identifying differences comes from taking the time to reflect on one’s behavior and soliciting feedback from others and comparing the observations and feedback with our espoused theories. This doesn’t happen on accident, it needs to be intentional. Differences should lead to creative tension (Senge). The absence of creative tension implies lip service to an espoused theory; we are saying all the right things just to get by and not because that’s how we want to act or behave.
Remember, our actions speak so loud, people can barely hear what we’re saying (with our mouth).
Every day, I encounter people who are in the business of trying to change people. Don’t worry, I’m guilty too. Organizations design around the notion of changing people but you already know what I think of performance appraisals. We have deceived ourselves into thinking that we can change others. I beg to differ. The best you can do is influence and present information. The decision to change lies with the individual, not you.
I think we’re better off paying attention to Lewin’s equation which tells us that behavior is a function of a person and their environment. Instead of trying to change individuals, change their environment (system). Look for the organizational impediments and dysfunctions that limit the abilities of the individuals in your organization. I am pretty sure there are a couple you (manager/leader) can identify pretty quickly once you ask. Create an environment that enables the people within it to succeed. But a word of caution, in order to effectively do this, you must have empathy for people.
So what are you really trying to change?
Have you seen the system? Let’s start with a little story..
A father goes into his children’s bedroom to wake them up and get them ready for school. As he pulls the covers off of his daughter, he notices that her eyes seem a little cloudy. He finds a thermometer and checks his daughter’s temperature. It reads 101.7 F, she definitely has a fever. His daughter’s school has a very clear policy that children should be fever free for 24 hours before being brought in to school. This means Dad is going to have to call his employer and let them know that he won’t be able to make it in because he has a sick child he needs to take care of. Seems pretty straightforward doesn’t it? But is it?
Dad’s employer keeps track of how often their employees call in to say that they will not be able to make it in for one reason or the other. Employees are dinged for not “showing up” and it affects their yearly raise and bonus. It just so happens that Dad is one ding away from some significant repercussions. On the other hand, the company does nothing if an employee comes in to work but has to leave because of a situation. As long as you show up (even if its for just one hour) you are good.
Dad being fully aware of his situation at work, decides to take his daughter in to school fully expecting to be called to come and pick her up in just a few hours. In his mind it’s a win-win situation because he’ll still be able to take care of his daughter and not get dinged for not showing up to work. What he doesn’t know is that his daughter will go on to spread her sickness to a couple of the other kids in her class. He also doesn’t realize that the parents of these children work for organizations with the same “call in” policy as his and are also going to bring in their sick children to school. His decision to take his daughter in to school has just triggered weeks of kids being sick in the class. In fact, his daughter will fall sick again a few weeks from now.
So what’s the system in this story? Can you identify it? Many of us might be quick to judge the father harshly for taking his daughter in to school but when you consider the structure(s) that influenced his behavior, you realize that there are a set of interconnected elements working together even though they may not be obvious to the different actors involved.
Wikipedia describes system thinking as:
…the process of understanding how things, regarded as systems, influence one another within a whole.
Dr Deming defined a system as:
…network of interdependent components that work together to try to accomplish the aim of the system
Peter Senge describes as system as:
webs of interdependence.
Russ Ackoff on the system:
A system is more than the sum of its parts; it is an indivisible whole. It loses its essential properties when it is taken apart. The elements of a system may themselves be systems, and every system may be part of a larger system.
and Ackoff advises that we should
…focus on the interactions of the parts rather than their behavior taken separately.
A team struggling with quality thinks that better or more QA people need to be added to the team or the iteration needs to be longer. An organization struggling with product delivery has its development team go on an Agile transformation. Confusion exists around what people should be doing, so an exhaustive writeup needs to be done to clarify this for everyone. An employee is underperforming so they must be slacking and are put on a performance improvement plan. There is no shortage of reactive analytical responses that show up in the workplace on a day to day basis. Yet these responses really don’t solve anything over the long-haul. They are at best, cheap band-aids. We are solving problems with the same mindset that we used to create them (Albert Einstein). We fail to identify the components and the interactions between these components within the system.
When we take a systems view to addressing problems, we make an attempt to see as much of the whole picture as is possible. We realize that our development team is part of a large product delivery organization. We understand that the employee struggling with performance problems just had a new baby and is not getting much sleep at night but is on our most complex project. We realize that product management is making commitments that are leading our developers to cut corners ultimately leading to poor quality. A systems thinking view to problem solving causes us to look beyond where we “think” the problems are to identifying the different components and the interactions between these components that is actually leading to our problems.
Let’s be humble enough to realize that we may never see the whole system but at least we’re trying to. Don’t be deceived, taking a systems view is by no means the natural thing to do because we are taught (from an early age) to break things up and address the individual components independently. I meet many people who say that they are systems thinkers but really cannot see beyond their nose in problem dissolving. They just don’t know how to do it because they haven’t learned how to. I’m still learning. If you haven’t read the works of Deming, Senge, Ackoff and Weinberg (for example), you may want to start there. We need to learn how to become system thinkers if we truly intend to dissolve our system problems.
Thanks to Bob Marshall (@flowchainsensei) for inspiring this post.
When you plan an event, what are you more concerned about? The outputs or the outcomes? At face value, these may seem to be the same thing, but are they?
Think about a soccer team involved a soccer match. The outcome the team desires is to win the match (although if your Chelsea and your playing away it may be to draw the match). In order to do that, the team needs to score goals - that is its output. Outputs lead to outcomes but in of themselves are not outcomes. I think its very important to distinguish between the two. Why? Because chasing outputs takes our eye off the outcomes we desire.
For the purpose of this post, outcome is defined as “the e
Our analytical way of viewing the world would have us focus on outputs. We like to break things down into its component parts and specify them in a neat and tidy way. Outputs are easier to measure. I was involved for over a decade in the design and architecture of a relatively large Supply Chain solution and have practically spent most of career (till date) designing and building platforms. I’ve also spent a lot of time on teams where we developed processes to help manage our knowledge work organization and have experienced that process design is often done through the lens of the analytic mindset. Remember, this is how we are taught in school. It’s our default way of learning. We devise solutions via decomposition. Even though conversations start with outcomes in mind, the process eventually gravitates towards outputs. Think of all the processes in your organization. Aren’t they by and large outputs trying to ensure certain outcomes?
Focusing on outputs instead of outcomes can prevent us from making the necessary adjustments required by our desired outcomes. Outputs becomes proxy variables for outcomes. What happens if you go to the gym every day (output) but don’t weigh yourself (for example) to see if you’re actually losing weight (outcome)?
Even in the wonderful world of Lean and Agile we fall in love with practices (outputs) and forget that the goal is to deliver software that delights the people that use it (outcome). Organizations fall in love with productivity (output) metrics and pay little or no attention to effectivity (outcome) metrics. It is critical to take time to reflect on how an organization is tracking towards its stated outcomes. Unfortunately many organizations spend more time auditing output compliance than they do outcome attainment.
I am not suggesting that outputs are bad and that we trash them. Rather I am reminding us that what we often truly care about is an outcome. We should value outcomes over outputs. Our work should guided by the outcomes we seek. Our teams should have a clear understanding of these outcomes. As we assess our progress towards our outcomes, we should change our outputs to help us get there.
Ok, stop. I mean it. Stop this moment.
Is a manager supposed to manage people? NO.
You, dear manager, are supposed to focus on creating an environment that enables people to excel at their job. I’m not making this up. Take some time to read the work of Deming and Senge, you’ll see that I’m not off my rocker. Then again, as a manager, how come you haven’t read their work? Oh, that’s right, you were probably just promoted into a management position and asked to figure it out on your own. Don’t worry, that’s what happened to me as well. There is hope for us.
I’m sorry you’ve been misled and that the prevailing management theory in use is still from the Stone Ages. I’m sorry, very sorry. I have a confession. I was also brought up (or is it brainwashed) to think and act that way as well. Yes, I do find myself at war with myself at times as the flesh wants to revert back to “old way”. So no, I am not judging you. I’m challenging myself as I challenge you.
Trust me, there is a better way. A way that makes work more enjoyable for everyone. I’ve experienced it. I need for others to experience it. While we’re at it, let’s stop talking about “empowering people” as well. I think we can do better. It doesn’t reflect well on us. It suggests we have power to give others power. What’s that? Hubris? I used to use that word all the time. I’ve stopped. I think we all need to stop. And since we’re stopping the use of words, why don’t we quickly add “resource” to that list. But this could all be wishful thinking on my part. Old habits die hard, don’t they?
But I got sidetracked there, the point of this post is to remind us (managers) of what our focus should be on if we really want to make a difference in our organizations. Focus on the system.
I’m a Theory Y guy. I may be näive in my outlook but I’m just a Theory Y guy. I believe that most (not many) people want to do a good job wherever they are. So what do we do when we observe that someon is not doing a good job?
Let’s say that Chidi (a guy I know) is having a difficult time at work, he’s struggling to meet expectations of his co-workers. Even though it seems his trying really hard, he’s struggling to deliver. What could be the problem? How can we help him? Well if we view his performance through the lens of Theory Y, could we identify the reasons why he is not doing so? I can think of at least 3 (I’m sure you can add more):
- Chidi doesn’t know HOW to do a good job – competence.
- Chidi doesn’t know WHAT a good job is – understanding.
- Chidi knows how to do a good job but the environment is preventing him from doing so – system.
Let’s explore these a bit.
Competence means the ability to do something successfully. Does Chidi have the right skill for the job? Was he given the appropriate education to allow to be competent at the job? If this is a new position, is a competent mentor mentoring him? How is the organization taking deliberate steps in improving his competence?
Does Chidi understand the expectations of the job that the organization has given him? Have they been made clear to him? Was it discussed with anyone? Who is available to answer his questions if he has any? Was he told to “figure it out on his own”? Have the outcomes expected from him been connected to the strategic vision of the organization? Does he understand the strategic vision?
Do the structures (Senge) in place enable Chidi to succeed? Is the system working in his favor or against him? Is he doing his best in a poorly designed system? Have problems with the system been identified? Is anyone (management) fixing the system? Is management tampering (Deming) with the system e.g. adding more process? Does management even know and understand the system?
In my experience as a “manager”, competence and understanding often prevent people from excelling at their jobs. However, I have found that the system is the biggest inhibitor to people’s success. It was Dr. Deming, who suggested that 85% of a workers effectiveness is determined by the system. In addition, Lewin’s formula reminds us that the behavior of an individual is dependent on the person AND their environment. This is another reason why performance appraisals can be largely ineffective.
At the beginning of this post, you may have identified yourself as a Theory Y type person. Is your espoused theory the same as your theory-in-use? When people or teams have challenges, how do you look to help them? How does your ladder of inference inform you? Do you just see the tip of the iceberg and act on it or do you go deeper to truly understand why an individual or team is challenged?
At the end of the day, it’s very likely that Chidi is really not the problem, after all, he really wants to do a good job. You see my friends, appearances can be deceiving.