Welcome to EMC Consulting Blogs

This post presents the results of a selection process described in the previous post to find an agile process management tool with which to run our project.

I'm not going to go over old ground but to make this post capable of standing on its own, here is a very brief explanation. We evaluated a shortlist of tools that met our cost and technology constraints. We scored each tool against 11 criteria that described our project management process. We weighted each of the criteria based on how important they were to our team on our project.

To see the full list of tools, the explanations of the criteria and our weightings, have a look at the previous post (http://blogs.conchango.com/gavyndowst/archive/2009/08/04/reviewing-agile-process-management-tools.aspx).

Now on to the results and we'll start with the overall figures. Both the weighting adjusted and non-adjusted figures are represented but, as you can see, the correlation is pretty close, ScrumNinja being the notable exception as one of the main areas it drops points on has a low weighting for us (team management).

 

The two winners, of those tested, are Version One and Agile on Demand. These two products very much sit on their own in the upper tier of products that do everything. They score very highly because they simply tick every box. This comprehensiveness does make them quite complicated to setup and use though. If you are running several projects with numerous teams and multiple releases then I'd say these tools are a good place to start. The effort spent getting them set up will be rewarded. Almost making it into this upper tier is XPLive which is also very full featured. It drops points mainly in the project view stakes. It doesn't have a fully interactive story or task board and the burndown charts were not the best. It is considerably cheaper than the other two in this tier though and handles story, task and testing management well.

SilverCatalyst and Scrumninja are similar concepts and, that they scored so similarly is not surprising. They don't do anywhere near as much as the upper tier tools in terms of allowing multiple levels of project, releases, iterations, epics, stories, tasks, tests etc. but they are no where near as complicated either. They do the core items very effectively and intuitively with a high focus on a familiar card based story/task board view of the project.

AgileBuddy, ProjectCards and Agilo probably make up the next grouping and all focus on the backlog management areas of the process. They don't provide much, if anything, in the way of story and task boards or burndown charts.

Finally, sitting sort of on its own is XPlanner. This is an open source project and offers much already and promises much more. Currently as with the three in the previous group it's focused on the core backlog and task management areas. Those things that can be considered as improved interfaces such as task boards, burndowns are not there yet. It's one to keep an eye on though due to its price and its open source nature makes it potentially integratable with other tools.

So to our choice and the one we deemed right for our project and our team. SilverCatalyst. It does what we need and it does it efficiently with a highly intuitive interface and at the individual task level it stacks up well against the highest scoring Version One as can be seen below.

 

It also comes as an on-site version in addition to the hosted offering that was tested which promises integration with thirdparty tools allowing enhanced bug tracking, IDE integration etc. We may or may not go for this option as the hosted solution ticks a very big box for us with regards remote working but, if we do I'll certainly blog on the ease of integration, or otherwise.

 

I recently read an interesting article on the New Yorker's website, here*, about how rules, accepted convention and how doing something outside convention can yield results. It is an article that spawned a number of thoughts or, more accurately, realisations of occasions in my career where the messages in the article are really pertinent. The first example was triggered by one of several anecdotal references in the article, that of Doug Lenat and the Traveller Trillion Credit Squadron tournament which I will briefly summarise for your convenience and to better clarify my point though I'd still recommend you to read the whole of the New Yorker article.

In 1981, Doug Lenat was a computer scientist at Stanford University and decided to enter the Traveller Trillion Credit Squadron tournament. Run in San Mateo, this was a war game governed by a comprehensive rulebook which gave participants a trillion credits to design a fleet of warships to do battle with other contestants' fleets. Battles occurred over the course of a weekend in a large hall, described by Lenat thus, "Imagine this enormous auditorium area with tables, and at each table people are paired off. The winners go on and advance. The losers get eliminated, and the field gets smaller and smaller, and the audience gets larger and larger."

Lenat had developed an artificial intelligence program called Eurisko into which he fed the tournament rulebook. Lenat gave Eurisko no advice or help, not being a war gamer himself he had none to give and so, unencumbered by human interference, Eurisko set to work. Running for about ten hours a night on one hundred computers at Xerox Parc, it took about a month for Eurisko to find an answer (this was 1981 remember!).

Compared to the normal human designed fleet that featured an array of ships of varying sizes balancing attack, defence and mobility Eurisko's decision was radical. "The program came up with a strategy of spending the trillion on an astronomical number of small ships like P.T. boats, with powerful weapons but absolutely no defense and no mobility," Lenat said. "They just sat there. Basically, if they were hit once they would sink. And what happened is that the enemy would take its shots, and every one of those shots would sink our ships. But it didn't matter, because we had so many." Lenat, or rather Eurisko won the tournament easily that year.

Next year the rules were changed to outlaw static gun platforms by introducing an enforced 'agility' measure that counted toward success. Provided with the new rulebook, Eurisko started work again. "What Eurisko did was say that if any of our ships got damaged it would sink itself?and that would raise fleet agility back up again," Lenat said. Eurisko won again.

In this competition, Eurisko was an underdog. All the other gamers had extensive knowledge of military history and strategy. They were skilled at gaming. Eurisko knew nothing but the rules and it had no common sense and made no interpretation of the rules based on its own experiences. It was this lack of understanding of the conventions of the game and even life itself that was Eurisko's advantage. "Eurisko was exposing the fact that any finite set of rules is going to be a very incomplete approximation of reality," Lenat explained. "What the other entrants were doing was filling in the holes in the rules with real-world, realistic answers. But Eurisko didn't have that kind of preconception, partly because it didn't know enough about the world." So it found solutions that were, as Lenat freely admits, "socially horrifying": send a thousand defenseless and immobile ships into battle; sink your own ships the moment they get damaged.

For a summary, that was quite long, wasn't it? Hope you're still with me! The essence of the anecdote for me is the interplay between experience, interpretation of rules and the forming of conventions. When we are given a set of rules that define a process, we fill in the gaps, we interpret those rules based on our current knowledge and experience and that then defines how we proceed. If this interpretation of the rules is followed repeatedly it becomes a convention. Once something's a convention it becomes accepted as being the rules and that's where the problem occurs because people lose sight of the original rules, and most importantly, the true meaning of them.

This is a problem that I've seen in many large corporate IT departments with large, old policies. Over time, the policies have been interpreted and the interpretation has become convention. The convention has become so often practiced that it is now assumed to be the policy. When you attempt to implement something new, a technology or an approach (bearing in mind the speed at which IT moves, new is a fairly common concept) it creates all sorts of problems because it doesn't meet policy.

That's not true though because it's not being judged against policy. It's being judged against the accepted policy which is actually the convention that has been built around an interpretation of the policy that was made long before this technology or approach existed. What should happen but doesn't is the rules that define the original policy need to be picked up and looked through by someone unencumbered with the experience of the convention and with knowledge of the new approach or technology who can determine if the policy is truly broken or not.

We need to follow the rules, not the convention.

 

---

* http://www.newyorker.com/reporting/2009/05/11/090511fa_fact_gladwell for those reading this in print.

Minimising the Bug Debt - A question of size

I think most people would agree that on a real world development project of any scale, bugs will exist. Apply all the best practices in the world in the shape of unit testing and automated functional testing but there will be bugs.

Minimising the size of the debt that is allowed to accrue is essential to staying within a single iteration of release. A useful way to achieve this is to keep everything else small. It is already widely accepted that quality is improved if the functional tasks themselves are broken down into small discrete items. A very small functional requirement with a few defined acceptance criteria is far easier to get right than a large complicated requirement with 10s of acceptance criteria.

Extending this philosophy, keeping the sprints themselves smaller is also advantageous. I will personally never run a project with sprints longer than three weeks again and wherever possible I will aim for two weeks.

The obvious advantage of smaller sprints is that less code can be developed in any one sprint and consequently fewer bugs can arise in the first place. Of course, this is offset by the fact that you have less time to find and resolve those bugs and if you’re still writing bad code then it probably won’t help! Another benefit though should help and that is that a shorter sprint encourages the team (development and business) to break larger functional tasks down into smaller ones and, as mentioned earlier, this is an acknowledged way of improving quality.

Other advantages of smaller sprints are more frequent opportunities for retrospectives and, consequently, opportunities to modify anything that’s not working, more frequent releases to the customer and a greater number of total sprints in which to react to changes from these releases.

In summary then, just as KISS (Keep It Simple, Stupid) has become an acronym for development and code design, it seems it could be employed to mean “Keep It Small, Stupid” too!

This post is the last of those that I had pre-written in another form and was to be the last of my blog posts on the testing in Agile subject. These things are never fixed though and a comment on a previous post has triggered another area of thought. So, probably after Christmas now, I will post something on the topic of 'Functional testing vs. unit testing - the right tool in the right place'.

Have a nice break all.

Musings on the meaning of "potentially shippable"

One of the mandates of Scrum, and a mainstay of Agile processes in general is the concept that every sprint or development iteration should produce potentially shippable code. In essence this is taken to mean that the day after any sprint is completed the fruits of the teams labour can be released immediately. Release could be to a UAT environment for final fit-for-purpose testing, or directly to a production environment but this is an extraneous detail. The key point is that there’s an expectation that the developed product will have no significant bugs and require no further development before being moved on to the next stage of project implementation.

The question that has been occupying my thoughts for some time now having been through a number of Agile development project is whether this premise is actually achievable in the real world. Having failed to reach a definitive answer on my own I raised this as a discussion point at the recent Scrum Gathering in Minneapolis which provided some useful experiences and opinions on which to draw. This helped lead me towards a conclusion which is not so much an answer as a redefinition of the premise.

Potentially shippable code is code that is never more than a single iteration away from release.

In taking this approach it is possible to maintain agile development practices without significantly impacting the release as soon as it’s complete enough mantra. As a sprint backlog is worked through, code is developed to a high quality, tested at a unit level, functionally tested and bugs uncovered. In my experience, these bugs are generally rated by criticality and then handled in one of two ways. Regardless of the number of criticality levels, there is a dividing line with bugs sitting above that line being fixed immediately, within the current sprint, and bugs falling below that line being queued up for later attention. It is these queued bugs or “bug debt” that this final, “snagging” sprint provides the time to resolve.

In subsequent posts I will post some thoughts on how to both minimise and manage the inevitable bug debt.