Enterprise Agile Testing Part I : Introduction

The idea of "Enterprise Agile Testing" has been in my head for several months now, result of what I have learned at Guidewire and based on my previous XP experience at ThoughtWorks. I am planning on a proposal to Agile 2008 on this topic. Before I can choose my proposal topic, I need to write everything down first, kind of like a project engagement.

Actually, project engagement is not a bad analogy. I must define what my proposal is about and what it not about --- what is out of scope if you will. My approach is to write a series of blog posts, each cover a specific topic and look back to see what I end up with when I am done. If I approach it as if I like writing a book or even like that agile 2007 paper, I might never finish it.

The Enterprise in Agile Testing

Enterprise here means large scale software development. The large scale can come about through a large code base or a large team. Here I am ignoring the controversial topic of whether or not large code base or large team are problems that should be avoided in the first place. They exist, I just want to point out two things that result from them with regarding testing in such an environment.

First, with a large code base, a tester cannot clearly hold onto the code's design in his or her head, let alone the intention of the test. Instead, agile testing in enterprise environments requires a comprehensive testing framework. This framework must do more than what JUnit does out-of-the box, so that anyone (including you) can come back to a test at any time and understand it.

Second, with large team, it is pretty much impossible to ensure everyone is aware of how important testing is. This is not to say that you should give up on a large team's continuous improvement on treating testing seriously and writing better tests, but I have found out that the line of "Zero Test breakage" is extremely hard to hold. As a result, some middle ground must be reached between complete awareness and total ignorance. Only in this way is it possible to see results in testing improvement efforts.

Agile

Since Rob and Jim pulled me away from the EJB madness and introduced me to the wonderful world of XP in 2001, "Enterprise" has slowly restored its place in my vocabulary. At the same time, the term "Agile" is getting closer and closer to my list of red flag words.

Agile here refers to the situation where the code is constantly under changes. This can happen because the requirements keep changing, or the development is done iteratively through story driven development. Constant change makes the ability to write concise tests more important. Additionally, the tests status of project must be treated as more than a binary state if testing is to keep pace with development. In this way, the development would not be paralyzed because there will almost always a test broken here and there, and the turn-around time for testing the checked-in code is not as short as 10 minutes.

Testing

So you have a large project code base that you keep changing, with a team of members with mixed skills. You need to ensure that the code (including the tests) you write is of high enough quality so that two moths from now you can still read them, understand what they do, understand why they do that, and change them. At the same time, you want to give others the time and tools to adjust to test infected development and hopefully test-driven development eventually.

Content and Structure

So, the above is the introduction. The items that are in my mind are as following, I'll update the links as I post them.

• Test utilities like assertion, builder
• TestBase with annotations for test environment configuration
• ToolsHarness, a continuous integration server that treats tests individually
• Active and stable branch, localizing the damage
  

Team Estimation Game - By Steve Bockman

Well, Steve didn't have time to put it up his website, so I figure I'll write up what I have learned here.

I ran into Steve during the agile 2007 conference so I invited him to do the session for us at BayXP. Last October, Steve came down and hosted a session of the Team Estimation Game. This is the first time I learned it I really liked it.

Given a set of defined stories, the purpose of this game is to come to the consensus about the relative estimation of the work to be done for each story. If you don't have a set of stories to try it out, you can use the sample produced by Steve here.

So once you have all the story cards, you first estimate their complexity relative to each other:

1. Place Story Cards in pile on table.


2. First player places top card on playing surface.


3. Next player places top card on playing surface relative to first card.


4. Next player can either:
   * Play top card from pile, or
   * Move a card on the playing surface, or
   * Pass
   
5. Repeat Step 4 until
   a) no more cards remain in pile, and
   b) no player wishes to move a card
   

Here is an example of the result:


After the first stage is finished, you can now go on to assign points to them so that you can track your team velocity:

6) As a team, choose estimation units and values.

With the agreed units:

7) Place an estimate at the top of each column.
8) Change estimates until all team members agree.

We did it during the meeting and it went very smoothly. The interesting thing is that Steve told us the result is not too much different from the ones that he has conducted elsewhere with other groups.

I am definitely going to try it on my next story estimation session.

References:

* Planning poker, which is another popular and useful estimation game.
* Steve's slides: http://www.shaneduan.com/estimation/Team_Estimation_Game.ppt
* Planning Poker Cards you can buy here.