On a more serious note, I have learned a lot this year via various avenues and people.  I’ve been motivated to improve the quality of life within my field.  That makes the upcoming year even more exciting when I consider the possibilities that exist out there.  For all of you who have taught me something directly or indirectly, thanks.  I’ll see you all on the flip side.

Happy New Year.

I’m not discouraging looking outside the field of software development for radical ideas as I don’t believe the field will evolve if we don’t do so.  Rather, I’m suggesting that we become more accountable in the application of these borrowed ideas.  Let’s understand that our domain is unique in its own way and grow it in a responsible manner.

1. Technical competency
2. T-shaped skill set
3. Personality types

Technical competency should go without saying.  We need people who can do the job and do it well.  Even if they can’t do exactly what is needed, they should be capable of learning and subsequently doing what is needed.  However, beyond just being technically competent, they also need to be driven to improve technically and must have pride in their work.

T-shaped people can also be referred too as being “poly-skilled”.  In everyday life, most of us are poly-skilled.  We can cook, clean, read, drive and the list goes on and on.  In knowledge work, these are individuals who can do more than one thing.  For example, program in C# and write user stories or perform some DBA-type functions and also write automated tests.  In essence, these are people that are not one trick ponies. Teams that scale effectively are usually comprised of T-shaped team members.

The last, but most important thing, in my current opinion is looking for different personality types to complement each other on the team.  I know of teams that have prospective new hires take personality tests and while I don’t know if I’m in favor of that yet, I can definitely see how doing so could be valuable.  Having a range of personalities on a team naturally provides for variety of opinions which is always a good thing.  For all the bullish people on the team, having a couple of bears balances things out. For all the outspoken individuals, a couple of reserved team members may help the group actually think things through

This was intended to be short and discussion provoking.  If you have others things you look out for, please share.

Is there anything to learn from the Colts debacle? Possibly. Minimize silos and specialization as much as is practically possible. Create a culture of cross-functionality.

American Football is a sport that thrives on having specialists. It is rare to see a player actually play more than one position consistently except in special defensive packages and/or on special teams. In fact, player height and weight can be radically different depending on what position is being played. Offensive lineman are much bigger than tight ends.  Defensive backs and safeties are not very tall. Because injuries are common place, teams have backup specialists for each position. When a starter is hurt, especially in a key position like quarterback, it can have serious ramifications on the success of the team. (Did I mention the Colts are yet to win a game.)

Let’s look at another sport for comparison. Football (or soccer as it is referred to State-side) is another sport with defined positions. However, strictly speaking, there is really only one specialist – the goalkeeper. Field players generally can play different positions and be quite decent at them. Arsenal (my favorite club team) had to deploy Alex Song, usually a defensive midfielder, into a center back position due to injury for a couple of matches. Barcelona has done the same thing with Javier Mascherano. But beyond that, and this is even more exciting, during the course of a game, players change positions all the time. Fullbacks “overlap” into a winger position, midfielders make runs into a striking position (a technique Barcelona has perfected) or drop back to cover a fullback that has pushed up. Teams like Barcelona and Arsenal are renowned for the fluidity and movement of their players as they change positions on the field. The ability of the players to be cross-functionality allows quick adaption even during the course of a game.

I see software development being more like soccer than football.  The Agile movement made popular again, the notion of “cross-functional teams”. Instead of having a bunch silo-ed specialists on a team, adaptability and flexibility comes from having what one could call a “team of generalists”. A group of people with the ability to wear multiple hats and perform different roles. In fact, I believe it was for this reason that some Agile methodologies use only the “developer” title for members of the development team. A “team of generalists” can respond to team changes such as vacations, departures or even the simple need to swarm on a given initiative in much dynamic way than a “team of specialists”.

Beyond just having generalists on a team from a skill perspective, it’s also important to have generalists from a domain knowledge perspective. All too often, teams will have generalists from a skill perspective only and still promote specialization in aspects of the application.  For example, Susie is the only developer who works on the invoice module.  Well what happens if Susie is out?  While specialization may seem more efficient, it reduces the ability of the team to respond to change and change is certain.  Remember, that in knowledge work, the concept of a backup is more of a fallacy than a truth.  If you are not actually doing the work, then you don’t know how to do it.

So how do we develop cross-functional teams? It’s critical to establish the culture of cross-functionality especially considering that it goes counter to prevailing reward systems that esteems specialization. Teams need to be educated differently. Practices and techniques like pair-programming, rotating work assignments, cross-training, wikis, documentation, collaborative design sessions should become part and parcel of way of life for the team.

As an exercise, take inventory on both who knows what and who can do what on your team. This will show how cross functional your team really is and where the team is exposed.  Fill those gaps, you don’t want to end up like the Indianapolis Colts.

I actually believe that estimation is an activity that should be done as infrequently as is possible and when done, should be extremely light-weight.  I know this might be a bit controversial, especially for my traditionalists friends, but hear me out.  I don’t see estimation as a much of value-add to the product development process in most cases.  Estimates in the large ultimately create undue pressure (not challenges) and try to impose order on an inherently complex systems.  Unfortunately, we don’t learn and keep trying and keep failing. Einstein had a description for such behavior.

This post is going to look at estimation in the case where a “Fixed Date” is not in place but we need to get a sense of the effort required in order to determine what should be done next or as input into a high-level release plan.  A “Fixed Date” means that a solution much be delivered or live by a certain date.  In the absence of this constraint,  I suggest a completely different approach to estimating. There is a lot of effort (possibly overhead) in planning for a Fixed Date so it must be worth it.

The smaller the work item, the better we generally are at estimating.  The smallest work item is usually a “task” and so if you are looking for an accurate estimate that will translates to effort (cost) and schedule (delivery), then a user story/requirement/whatever should be decomposed into all its composite tasks and every single task should be taken into consideration. I have made the mistake in the past of not having every different role explicitly specify and estimate their tasks.  For example, instead of having a business analyst, developer and tester estimate the same line item, each role estimate their tasks separately.

However, when beginning a new initiative (such as adding a new module) referred to as “themes” in agile speak and “requirements” in other contexts, the tasks are pretty obscure, so what do you do when you’re required to give an estimate?  Well you could go through the process of decomposing down to tasks but as mentioned above, except there is a “Fixed Date” for delivery, I wouldn’t do that.  I would start with T-Shirt sizes.

T-Shirt sizing provides an avenue to quickly categorize and classify themes into buckets without expending too much effort.  Now these sizes need to tie back to a common unit of measure.  In my experience, non-product development folks may not understand things like “story points” (if you are using user stories) and so hours/days may need to be used i.e. develop estimates in the language currently spoken in your environment.  I believe in using ranges in estimates given the uncertainty that is product development and to account for the cone of uncertainty.  So an example (in hours) that I just used recently could be:

• XS:    1 – 100 hours
• S:       100 – 500 hours
• M:     500 – 1000 hours
• L:       1000 – 2000 hours
• XL:    2000 – 5000 hours
• XXL: 5000 – ?

The key is to use a scale that works in your environment.  Maybe all your requirements are smaller i.e. under 1000 hours, then XL would be 1000 – ?.  The disadvantage with using hours/days is that most product developers are tempted to want to breakdown themes into tasks in order to provide what they consider “better” estimates.  A way to mitigate this, is to find a baseline item to work with, such as previously completed initiative (historical data) or another item within the list of themes for which there is more “confidence” about its estimate. Another key point to emphasize is that we’re just attempting to identify at a high-level how much work we think we have.  By no means should concrete plans be made on such estimates.

A question that I often get asked, is ‘who’ do I think should provide these “high-level” estimates, especially on teams with different roles involved.  This is a little tricky because different roles bring different perspectives to the table.  However, my general answer is that the senior technical people who understand the domain should be the primary estimators at this level and it should ideally be more than 2 people.  Leverage the wisdom of the crowd.  Use the Wideband Delphi method.  When discrete tasks are being estimated, get everyone involved.

Assuming the “cost of delay” for a theme has been determined, the result of the estimation process should help drive the selection of the factor should determine.

What do you think?  How do you (if you) estimate large requirements or themes?

Borrowing Peter Drucker’s definitions for the two words, effectiveness means “doing the right things” and efficiency means “doing things right”.  Note that both things are inherently mutually exclusive.  If I am driving to work in the wrong direction, I’m being efficient (by driving since by walking I would never get there) but not effective because I’ll never get to work. However, if I take the scenic route to work adding another hour to my commute, then I’m being effective (I will arrive at work) but probably not that efficient. Generally speaking, one always wants to be effective, but being efficient could be considered optional.  So if this is the case, which is more important in software development?

I grew up in Nigeria and spent a good portion of my childhood cutting grass and thick foliage with either a cutlass or machete. 
It was a highly effective way of cutting the grass as it got the job done. It also left me very hungry afterwards!   But I remember having to spend days cutting grass given the size of portions I had.  Interestingly enough, on the university campus that I grew up on, the landscaping team also used cutlasses to the cut the grass of all the fields on campus.  It would take days for them to complete, and areas like our football (soccer for my American friends) fields would be last on the list causing us to go weeks without playing at times.  For those who know me well, you definitely realize how tragic this must have been for me.

Then the university made an investment in riding movers and all of a sudden, it no longer took weeks to cut the grass, it was all done in a matter days.  The university leadership realized that even though the right thing was being done (effectiveness), it wasn’t being done in the right way (efficient) given the size of the campus. Introducing riding mowers allowed the grass to be cut quickly and consistently, improving the overall aesthetics and beauty of the campus.  It also opened up the doors for the landscaping team to do other things besides just cut grass! Most importantly, it allowed yours truly to play football whenever I wanted to because the fields were always in tip-top shape. The landscaping team was now both effective and efficient.

It’s easy to understand why being effective is important – we all want to do the right thing and I would suggest that a lot of our energies go towards effectiveness (especially when agile practices are employed).  It’s much easier to see ineffectiveness.  Your “customer” will tell you. Efficiency on the other hand can be a little more subtle – it’s harder to see, harder to challenge and harder to understand.  If things are working (effective), in my experience, efficiency takes a back seat.  The classic “if it ain’t broke why fix it ?” approach to things. Yet, efficiency allows us to scale our operations providing us with the opportunity to do other things including living more fulfilling lives!  We are inherently making our company more profitable because we are possibly getting more done in the same amount of time as before. Remember, time is money.  For example, manually  executing a hundreds of test cases may be very effective but is probably inefficient and so to improve efficiency, we decide to automate tests allowing testes to focus on exploratory scenarios or help with requirements gathering.  We don’t need cross-functional teams to be effective.  We introduce cross-functional teams because makes all the roles/functions needed to deliver solutions readily available to us.  It reduces communications challenges and allows us to respond to each other in minutes instead of days.  It improves the overall efficiency of the team.  The same is true for co-located teams.  That being said, it’s important to note that improving efficiency can also be an expensive undertaking. It’s important to a have “sense” or “feel” for what the gain in efficiency is really going to provide and what the objectives are.

Is it okay to be inefficient in order to be effective?  Sure.  I’m familiar with at least two cases (I imagine there are others) such as having multiple people on a team researching a critical production issue or not having members on the team with siloed responsibilities such as “application developer”, ”database developer” or ”UI developer”.    We’ve given up some efficiency in order to be effective by allowing everyone to do some UI development for example.  

We should understand that in figuring out the “unknown” as we often do in software development, we will not be as efficient as we could be if we knew exactly what we were doing.  (Think about all the hallway discussions that occur or the back and forth that we end up realizing was not necessary after the fact).  We are conducting a bunch of experiments, coming up with new recipes.  That being said, it’s not an excuse to accept inefficiency as the status quo.  Practices such as TDD, using user stories to gather requirements and just-in-time planning, using dynamic languages to name a few, provide efficient approaches to working with uncertainty and vagueness.

Let me suggest that both effectiveness and efficiency are equally important in software development (except when what is being developed and delivered is of little consequence – why are you working there?) and software teams should focus on continuous improvement in both areas.

Photo of my very own cutlass

Riding Mower photo from Jeff Harbert

Why this article is important is that it directly addresses myths that surround database development (or developing with stored procedures) that are so commonly held.  Let me be clear, I cut my teeth developing with programming languages that run outside of the database such as Pascal, Fortran, C, C++, Java, C#, Ruby and Python so I have no reason to promote PL/SQL or T-SQL but in this age of mis-information, it’s important that the truth be told.

Developers should have at their disposal all the tools needed to effectively build applications regardless of the languages being used. However, I’ve been around teams where there are two types of developers: application developers and database developers that are depended on to deliver solutions.  Curiously however, software development is partitioned in such a way that no one type of developer can actually deliver a solution from top to bottom.  For example, application developers don’t implement logic or database developers don’t develop user interfaces (or Windows services or file outputs).   So in the effort to get the best of both worlds, the team becomes highly inefficient and partially hamstrung.

Figure out what tools you want you to use and just get good at them.  Understand the contexts where they work best and leverage them.  Use the proper software development practices across the board.  Have software developers armed with the tools to deliver solutions in their entirety.  If some part of software development is a black box to you, go figure out, ask questions.  Just don’t drink the Kool-Aid.

Determining the cost of doing business can be done at multiple levels within an organization but I want to focus precisely on the software development aspect of things and speak to my technical brothers and sisters in our beloved community because I fear we’re giving a lot of blanket advice without asking people to determine if doing something is worth it from a business perspective.  Why do I think so?  Well I interview engineers all the time who rattle off a list of best practices believing that they should always be used and they quote thought leaders in our space all the time as a basis for this. If you don’t do practice X, you can’t be agile and I won’t work with you.

Suppose I asked you to build me a boat, you would want (I hope!) to know things such as:

• Why I need a boat
• What I would use the boat for
• How the boat impacts my business
• Where I would use the boat
• How quickly I needed the boat
• How long I would use\need the boat
• Etc Etc

From these questions, you would understand what type of boat I actually needed e.g. toy boat, canoe or a cruiser and the importance of the boat to my overall strategy.

So why is this is critical?  Because your development process will be shaped based on the type of boat I need.  Granted, there may be some cross-cutting development practices involved regardless of what is being delivered, but as a guy who seen firsthand the development process for houses built from clay as opposed to those built from cement, I’m positive there will be some significant differences.  There just have to be.  The practices are never exactly the same.

So coming back to software development, its important that the development practices we become religious about remain contextual. Saying that all code needs 100% code coverage (for example), just seems irresponsible to me without saying when (if ever) that is most likely to be true.  Another pretty common edict now is that all software development need be test-driven at all levels i.e. both internally and externally.  It is critical that the technology team clearly understands the role the software solution plays as part of the overall business strategy i.e. how it impacts the economic bottom line.

An organizations most strategic technology solutions will have maintenance costs over a period of years (assuming they remain strategic)  so we want to do as much as we can to keep those future costs down.  In these cases, it may be prudent to incur more upfront cost to avoid future expenses.  However, for a utility solution that is just helping an organization meet an immediate but short-term need, we may go a different route.  In fact, we may just throw together something over a weekend and call it done.

Understanding the requirements for a solution goes beyond just determining what needs to delivered.  It includes asking probing questions that lead us to make the right investments on how the solution solution should be developed because we understand its strategic placement within our portfolio of solutions.  Don’t forget, there is always an opportunity cost out there. Make sure it is really worth it.

Merriam Websters defines brutal as: very bad or unpleasant.  A brutal fact is something that is both very real and not terribly nice.  Being unpleasant is obviously context bound and relative to an organizations particular situation.  My brutal facts are not my next door neighbors brutal facts.  Examples of brutal facts in software development could be consistently delivering software into production much later than originally planned, having builds fail every time code is checked in or not having a technically competent team.

Truth be told, many organizations defer, downplay or outright avoid identifying brutal facts because the exercise of doing so is both self-revealing and self-indicting.  Who really wants to talk about what’s going wrong?  The teams I’ve worked on have never had a problem talking about the good stuff we were doing, but talking about our mishaps never shared the same enthusiasm.  This unpleasantness is due largely in part to the fact that identifying brutal facts is often conducted as a highly subjective exercise with no concrete data to go off of.  When meaningful data is not collected, its challenging to even determine what the brutal facts are let alone possibly address them. Because of our “ladders of inference”, we need data that we can all try to agree on.  If you truly want to identify brutal facts you need to collect information and track/measure important indicators.  You need to be an organization that is desperate to learn.  Unfortunately, too many organizations measure stuff that doesn’t really matter, but that’s a topic for another day.   Revisiting the example of functionality being delivered late, if a team had collected issues/blocks/impediments during the development cycle, identifying the brutal facts possibly becomes a simple activity of reviewing the impediments.  The review could reveal that the development team doesn’t have the access it needs to the end users or subject matter experts for clarifications on how the system should behave.  Combining information like this with techniques such as the 5 Why’s can make identifying brutal facts a bit easier.

That being said, you don’t get brownie points for identifying brutal facts, no my friend, no gold stars for you.  Brutal facts need to be addressed.  The word confront carries with it what could be considered a ”negative connotation” but I appreciate the choice of word here because it conveys an attitude of an active and aggressive attack on brutal facts.  That being said, it’s so very easy to address brutal facts incorrectly within an organization.  An improperly addressed brutal fact can actually do more harm than good in same way that trying to kill a fly with a hammer is an idea that can leave you with a busted thumb and broken glass all over the house.  In fact, you may be better off leaving the brutal fact as is if the only alternative you have is to address it incorrectly.

Why do we fail do address brutal facts effectively when they are identified?  I would suggest that this is due (at least in part) to our traditional analytical problem solving approaches i.e. looking at the brutal fact in isolation and not seeing it as part of a potentially larger problem.  A classic example of this is the persistent notion that the way to solve the software delivery problem suggested earlier is by adding more developers.  However, adding more developers would obviously just worsen things.   Unfortunately many organizations struggle to look at things in a holistic manner because they do not have the necessary eduction, training and skill required to address issues holistically within the organization.  This where I believe an understanding of Systems Thinking can significantly help teams and organizations address brutal facts in a holistic manner.  A book recommendation for that would be Peter Senge’s: The Fifth Discipline: The Art & Practice of the Learning Organization.   Systems thinkings in of itself is not a panacea – nothing in isolation is - but looking at the whole system hopefully gets us moving in the right direction.  For the team that does not have the access they need to SMEs, the organization could make a decision that SMEs need to set out an hour each day to address questions from the development team.  That could be a holistic solution.

What brutal facts plague your organization? Are you up to the challenge to confront them? I hope you are!

For starters, LSSC11 was the first conference I’ve been to (maybe ever?) where there were no sacred cows touted, no best practices elicited and no meaningless debates between method X versus method Y.  I believe we’re coming to a place where we’re starting to accept as a community – yet a minority in my opinion – that there is no “silver bullet” but that the right combination of principles from differing sources can possibly yield great results.  We’re accepting that our environments are complex and are leveraging methods that exploit this fact instead of fight it.   We are discussing how to make people “matter more” because we realize (even more) that its people that really make a difference.  Finally, continuous learning is paramount; LSSC11 had an academic undercurrent to it which I’ve not observed at any other conference.  The list of reference material I need to read has doubled in size!

To summarize, I sense a movement that is saying very loudly and clearly “enough already” to some of the traditional ways of getting things done and is looking to embrace different ways (some that have been possibly rejected or ignored for years) as we move forward.  This is really exciting, make sure you don’t miss out!