Why Agile Fails? This is an interesting question that has perplexed me over the years. I have worked at a number of software shops and many have jumped on the bandwagon of Agile. However, most have failed, and lingering question remains Why Agile Fails? I have spent some time, and given this some thought as to why this might be. Based on my experiences with the different attempts at implementing Agile, this is what I have come up with.
One of the big reasons Why Agile Fails is because of how most shops approach it, from the very beginning. Most shops seems to start with the same approach. They hire a bunch of consultants to come in and do Agile training. However, the problem is that training is more or less not valuable (a.k.a. useless). The trainer(s) will stand up there and spout a bunch of Agile theory. Then what happens is a lot of people will take that theory as if it was Biblical (or whatever religious text appeals to you) truth. They begin the Agile journey in a very dogmatic way. However, that is not really what Agile is about. It’s a framework, a set of guidelines, and each team needs to explore and find out what works for them and what does not. The Agile theory provides a bunch of suggestions, which you may or may not choose to follow.
Another big reason Why Agile Fails is that management teams are full of fear, and are unable or unwilling to let go of what they perceive as their control. However, the problem with that is that most managers really have no idea of how to lead a technical project to success. Most ascended the managerial ranks very quickly and have lost touch with technology a very long time ago. They really need to learn to take a step back and let the technical people do what they do best. This one actually amuses me very much. I have been on projects where the team started out doing Agile, and then half way through some director or some VP or some whatever has a panic attack, and decides we are going to switch back to waterfall—talk about wasting time!
Yet another reason Why Agile Fails is that key people never quite able to shed their waterfall ways. This can be particularly devastating when the scrum master behaves more like a project manager than an actual scrum master. There is also the inability or unwillingness to change scope within an iteration. With Agile there are three levers to pull: 1) time, 2) resources and 3) scope. Time really can’t be changed. Your iteration is your iteration. Resources can’t really be changed in course of a single iteration. That is really only possible over the course of multiple iterations. So you are really only left with scope as something you can tweak in an iteration. But yet so many simply refuse to, thus you are left with short duration waterfall in an Agile wrapper.
While on the topic of hold overs from waterfall the desire to over-document things (or even everything) definitely speaks to Why Agile Fails. Adding on additional documentation and/or administrative related activities can effectively kill productivity. When you find yourself spending half or even a quarter of your day filling in Wiki pages, you know you have crossed the line of useful documentation. It is much better to wait till the end of the project, and then document what really matters. It will be very clear at that point. Do not attempt to do heavy documentation in each sprint. It just does not add that much value.
There are also some specific Agile suggestions the when taken to an extreme can facilitate Why Agile Fails. For example, the over-use of always attempting to reach consensus can lead to group think. It sounds like a great idea, but in practical terms it just doesn’t work. People just stop engaging in honest debate for a variety of reason (eg: fear, exhaustion, laziness, apathy, etc…). Moreover, the idea that estimates can be done with little or no thought, works out to be a total disaster. There are orders of magnitude in difference between the 10 second vs. the 10 minute estimate. I agree that estimates should be done quickly, but doing them for the first time in a meeting with all kinds of pressure is just a recipe for bad estimates.
Another big blunder that answers Why Agile Fails is the idea that everyone needs to be present for every meeting could end up crushing valuable support. This is especially true for the business people. I have seen annoying, overly persistent scrum masters causing the project sponsor to just walk away and totally lose interest, because it’s just not worth their time anymore.
However, for me the most important response to Why Agile Fails is the lack of long-term planning. Too many times I have seen a complete lack of any kind of design & planning. This only results in a lot of low quality code, and when the low quality code is questioned the answer is “this is Agile we will refactor later”. I am sorry, but it is ludicrous to think that is going to work out. That bad code, the stuff that was slapped together to fit in an iteration, has a bad habit of lingering on and on. This is just a recipe for technical debt, and let me tell you I have seen technical debt kill more projects and products than anything else. For some reason people don’t really talk much about technical debt, but more often than not it’s the root cause for failure. There needs to be some kind of big picture planning and design–there just has to be!
One of the best defenses against Why Agile Fails is to always focus on the problem that Agile is trying to solve. I am not talking about the silly build a paper airplane kind of problem. I am talking about the getting more lines of code written problem. Let’s face it, the main selling point for Agile is the promise of getting more code written and faster. Numerous studies have shown that its lines of code that slow down the development and release of most products. So if we all approach the implementation of Agile with that in mind, then we will be able to avoid all the missteps, pitfalls and blunders that I mentioned above. But yet I see these same mistakes being made again and again—gee go figure?