fix svn tagging in sourceforge

I have been having tagging problem with Subversion in Sourceforge. It took me several days to realize that this is not going to be resolved by itself. Once again, Google helped me found the solution:

You need to point your working directory to a different repository URL, through command like:

svn switch --relocate https://svn.sourceforge.net/svnroot/cotta/trunk https://cotta.svn.sourceforge.net/svnroot/cotta/trunk

You just need to replace 'cotta' to the unix name of your project.

So Much for Java Generics

I always thought that Java generics is supported through compiler is a bit weird, and after 5 hours of email and spike, I finally know why.

Don't get me wrong, I love generics. Even though with TDD I can catch my mistakes most of the time, but with generics I can see my mistakes even before I run the tests. I won't say no to that.

Anyway, it all started with me starting a prove-of-concept project with another friend. So I am setting up libraries and figured I'll use jMock's constraint framework, after I convert it to support generics. Hence started my journey.

I checked out jmock code and was surprised to find out that there is a jmock2 directory. By the document that has been written, it is a way to stay with jMock style but don't lose the type safety. Cool! What is more, the constraint framework has been extracted into something called hamcrest.

Through email, Nat pointed me to the project in sourceforge (duh): http://sourceforge.net/projects/hamcrest. It looks very promising, even have the adapter for JUnit3, JUnit 4, and TestNG. It is apparent the intention of this project.

After I calmed down with all the excitement, I noticed one thing strange: The Matcher interface, the one interface that matching framework is evaluating matching logic, takes the "Object" type, rather than the generic type. What this means is that all the matching instance will have to do a type cast.

"That was silly", thought I, "Surely they know better than this". So I modified the interface in hamcrest, ran all the tests and proudly submitted my patch to Joe.

Turns out that the issue is not hamcrest itself, but jMock.

If you have two methods of the same name that you want to mock, say
public void test(int value);
public void test(String value);

And you provide the expectation
mock.expect("test").with(eq(1))

Upon the invocation of a.test("arg"), it will try to find the match by checking the matcher returned by eq(1) as well as that returned by eq("arg"). When comparing to eq(1) it will fail because from the instance there is no way you can figure out what the generic type is, and invoking by force will cause a class cast exception because there is a class cast injected by the compiler.

So in the above invocation, instead of having something like "unexpected invocation error", you end up with a weird ClassCastException error.

Bloody hell.

SCRUM at Guidewire

A month ago, I have joined guidwire, a local start-up building insurance products.

At guidwire, we are using SCRUM. So every new hire will get a book of "Agile Software Development with SCRUM". Somehow I missed it but I found it soon enough and got a copy right away.

I don't know why I never got a chance to read this book and I am really glad that I did. SCRUM is very much like the project management part of what I have been doing for the last several years. This includes the details like format of SCRUM meeting (daily stand-ups), feature backlog (master story list), goal driven team development.

On google video, there is a presentation done by Ken himself: http://video.google.com/videoplay?docid=-7230144396191025011&q=ken+scrum.
In it, he talked about SCRUM and XP complements each other. I didn't understand it before, and now I think it is really true.

Typing Contest : Dvorak vs. QWERT

After switching to Dvorak for almost a year, I decided to switch back to QERTY, for two reasons.

• I was pairing at least 6 hours a day and was driving everyone mad with the layout switching
• I wanted to see how fast I was with QWERTY.

Before I switched back, I timed myself in typing out the page "Dvorak Follow-Up". Now that I am fluent with QWERTY again (I should have done that earlier but I was too lazy), I just typed the page again using QWERTY.

And the result is...(drum...)...

DVORAK!

Time used with Dvorak: 9:28minutes.
Time used with QERWT: 9:55 minutes

I cannot say that I am not surprised. After all, I have been typing QWERTY almost my entire life and I only used Dvorak for a year. But here you go.

On the other side, 30 second is very small. I could just be tired tonight.

I also finished the "Joy and Pain with Fitnesse" in 29:30 minutes using QWERTY. If I ever switch to Dvorak again, I might do another contest.

Scrum et al. at Google Vedio

http://video.google.com/videoplay?docid=-7230144396191025011

I thought it was interesting on the part that SCRUM works with XP. I always thought SCRUM was too much on the management side, and now I think I know why.

Too bad most of the guys on agilechina cannot see it. Copyright issue I suppose.

Code Snippet, with Color!

I did it!

Now build master can grab code snippet from another file and apply syntax highlighting using syntax.

Next step is to figure out a way to specify the snippet validations.

I am dead tired right now and I have a 9am meeting tomorrow. So I just updated two (actually one and half) pages to show it off.

• http://buildmaster.rubyforge.org/getting-started.html
• http://buildmaster.rubyforge.org/docs/tag-references.html

I also fixed the document source and history link at the end of each page so you should be able to see the source file now.

Last Day as ThoughtWorker

Last day as ThoughtWorker, writing this blog that I have been putting off until the last minute, I find myself lost in thoughts in all the things that I have experienced since I joined ThoughtWorks 30 months ago.

It has been a nothing short of life-changing experience. As a matter of fact, it is with these changes that I have decided that in order for me to push even further, I will have to go out and look for good opportunities that can help me, rather than waiting for them to happen. As I going through the projects, my expectation on what I can do on the project has changed dramatically.

When I first joined, I was just happy if I can merrily write my tests, be happy about the 40-hour week and that the CEO is not trying to cash in one million dollars every quarter when the company stock is not going anywhere (like some company that I have worked for). Now on my last day, I have found myself comfortable coaching agile software development and influencing project management. I have experienced all the things a senior developer can do in a project, and even have had a short taste of being an IM and PM. I have been to China, twice, to lead projects. I have been on more and more discussion and presentations that involve more than "TDD" or "patterns". I am even pulling an effort with several others right now to write a book that we hope can be published.

Like my fellow Borlanders showing me the way to be a good developer, my fellow ThoughtWorkers have helped me understand what it takes to have a success project, besides technical execellence. I created this blog for my experience in ThoughtWorks. I have grown fond of it over the time and I am going to keep it up for my experience in applying agile practices.

So to wrap it up:

First several books that I read as a ThoughtWorker:

• Enterprise Application Architecture
• Domain Driven Design
• pragmatic Project Automation

Most recent books that I read as a ThoughtWorker:

• Tipping Point
• Blink
• Build to Last
  
• Blue Ocean Strategy
• Crystal Clear

Things I enjoyed doing in ThoughtWorks:

• Talk to Roy (don't worry about subjects)
• Meet old timers, they will show you why the idea that "employee managed company" has a chance in ThoughtWorks. (If someone was surprised that you were not one, it is a compliment)
• Meet at least one ThoughtWorker from each office from other country, especially the noisy ones.
• Request traveling to other country for a project.
  

Things that would be fun to do in ThoughtWorks:

• Tell Obie that ruby won't fly because it does not have an IDE like IntelliJ and it is not type safe. (well, maybe not anymore since he has been fighting that battle for a while and getting very good at it)
• Tell Hammant that you love Spring
• Buy beer around the table and ask for airplane takeoff and landing stories

Things that I did not enjoy doing:

• On beach for more than two weeks
• Travel weekly with connection flights each way.
• Stuck in an airport for a delayed flight for 7 hours
• Stuck in an airplane on the ground for 3 hours
• Saying goodbyes
• On your last day, be notified that you won't be covered by medical insurance anymore (So much for chilling out before the next job), even though you have accumulated over four weeks of vacation, and you could technically take your vacation before you send in the two weeks' notice, getting covered and accumulate additional vacation at the same time. The reason? You are not worth risking the liability anymore, now that you are not going to make money for the company and all. (Not in that word but I get the message)
  

I would have put boring period of the project in there but the following are what I have done each time I got bored:

• Learned Dvorak
• Created open source projects: http://www.shaneduan.com/opensource.html
• Read books
  

Things You CANNOT Get Certified For

So...

I was chatting through IM with a friend of mine about something that I read recently (Crystal Clear). And the conversation, as hard as I tried, just went downhill from there. I got really agitated in the end, even after I realizing that half the time it is one of those flag words that got on my nerve.

Looking back at the log, I realized that I thought he knew what I have been doing at ThoughtWorks for the last two and half years, and he probably thought I had no idea about the topic in this book. During the whole conversation, we were on different depth level of the topic. I guess this is one more case proving that you should watch out for "Barriers for Effective Listening" and ask "Why do you ask".

With all that behind, I was still stunned at the fact that no matter how well so many people try to protect a good idea, there are still people out there trying to profit from it by coming up with bogus stuff in its name. And one very good example is "certification". Because apparently they succeeded in making my friend think that is what it is all about, another certification.

So here is a list of things that you cannot get certified for. Instead, only your peers who work with you day in and day out can rate you, subjectively.

• Doing TDD even when you are under the pressure to delivery. Everyone can pass a test and do a little practice in their own pleasure.
• Have the courage to speak up when there are things that you think is wrong. "Do you think you should speak up when you see something wrong?" "Yes." "Good! You are certified!"
• Sit-Together. How can you certify that, just by sitting together for a week? It is one thing to say "yeah, it is a good idea", it is another thing to go to the extreme length of getting the tools yourself and start taking cubicles apart. (Yes I have met someone who really did it).
• Code Co-ownership. I am sure everyone can check that checkbox to get certified. But how can you certify a person's willingness to learn as much as the codebase, make sure that the design is as clear as it can be, and pass on the knowledge to others as soon as possible?

Sorry but "certification" is such a binary rating system, that it just does not fit into what it is in the real world.

Then of course, it is always good to get some education on the topic and have a proof that you have finished them successfully.

But that hardly gives anyone the authorization to say that "It is nice, but it is really really hard to do a business-driven situation".

Not when the purpose of the whole thing is "Deliver the business value in whatever the best possible way".

(BTW, did you realize that if you mistype blogspot.com as blogpsot.com, it is an actual website selling ads?)

BuildMaster Surgery - Moving to Cotta

I have been struggling with the File operations for a while, before I fully understand what is going on.

After I released Cotta project, the way File operations works in Ruby is like a hideous stain on top of a shining gem, always bugged me, especially when there are these nice classes like Pathname and StringIO already exist in Ruby.

Finally, my curiosity of making it better got the best of me, and I ported BuildMaster over to Cotta Style.

I have decided to call the files CottaFile and CottaDirectory so that I don't have to worried about the name space issue. Since BuildMaster does quite a bit of shell operations, I added method "shell" to the cotta instance so that the in memory version can log them for tests.

All things are happening behind the scene. The following is a code snippet demonstrate the new API:

   #system implementation is injected here
   cotta = BuildMaster::Cotta.new(BuildMaster::InMemorySystem.new)
   file2 = cotta.file('dir2/file.txt')
   file2.exists?.should_equal false
   # parent directories are created automatically
   @file.save('my content')
   @file.copy_to(file2)
   file2.exists?.should_equal true
   file2.load.should_equal 'my content'
   file2.read {|file| puts file.gets}

The API still need to be fine tuned but they provide all the operations that BuildMaster needs for the moment. Hopefully, someday something like this can be part of the ruby library.

BuildMaster Surgery - Moving to rSpec

This is the first part of the BuildMaster surgery that I have just finished (well, 95%) that have been keeping me up late every day. I will write the second part as a separate blog.

Moving BuildMaster to rSpec turned out to be not as smoothly as I hoped. Hopefully this blog will help anyone who is interested on similar task. This is what I have figured out so far so if there is a better way of doing it do let me know.

Syntax Change

If you have tons of test/unit scripts already, changing the code to use the rSpec syntax could be boring. They do have "Test::Unit Migration" support. However, I didn't get to take full advantage. It just generated "migration error" with no detail information for me. So I had to use some regular expressions to get the most of the change done, which I have blogged in "Going rSpec".

Specification Sharing

The "Context API" document on rSpec teaches you how to call helper methods in your specification code. However, it does not show you how you can make two context share the specifications, which is very useful if you want to make sure that two different implementation of the same interface shares the exact same behaviors.

Brain from Pivotal showed me that you can achieve this by using 'extend' (instead of include). So you can write the common behavior specifications in one file:

module CottaSpecifications

def register_cotta_file_specifications
  setup do
  # this one uses the @system that will be set up by each implementation spec setup
  @file = BuildMaster::CottaFile.new(@system, Pathname.new('dir/file.txt'))
end

specify 'file can be created with system and pathname' do
  @file.name.should_equal 'file.txt'
end
end

Then you can include those common behavior in your specification file for the implementation:

require 'cotta_file_specifications'
module BuildMaster
context 'Cotta file' do
  # extending the module so that you can call the methods
  extend CottaSpecifications
  setup do
  # setup the implementation for specifications
  @system = InMemorySystem.new
end

# this call registers all the spefications
register_cotta_file_specifications
end
end

Hopefully the above code makes sense to you. If not, let me know.


Running All Specifications

Again, thanks Brain to explain the reason and provided me with a solution.

Somehow, all the registered contexts are run at the end of each context registration. So my code for loading all the tests will now make the same context run again and again:

If you have context A and B in tc_a.rb and tc_b.rb and you require each file one by one. After "require 'tc_a'", the specifications in Context A will run. After "require 'tc_b'", the specifications in Context B will run, this time ALONG with all the specification in Context A. If you have C, D, etc., then specifications of Context A will run again and again at the end of each following require.

So Brain wrote his own specification runner, which will hold off the execution until all the contexts are loaded:

require 'rubygems'
require 'spec'
#require 'diff/lcs'
dir = File.dirname(__FILE__)
#require "#{dir}/../test/common_test_case"
require 'test/unit'
Test::Unit.run = true

args = ARGV.dup
unless args.include?("-f") || args.include?("--format")
  args << "--format"
  args << "specdoc"
end
#args << "--diff"
args << $0
$context_runner  = ::Spec::Runner::OptionParser.create_context_runner(args, false, STDERR, STDOUT)

def run_context_runner_if_necessary(system_exit, has_run)
  return if system_exit && !(system_exit.respond_to?(:success?) && system_exit.success?)
  return if has_run
  exit context_runner.run(true)
end

at_exit do
  has_run = !context_runner.instance_eval {@reporter}.instance_eval {@start_time}.nil?
  run_context_runner_if_necessary($!, has_run)
end

To use this, all I needed to do was to require this file at the beginning of my "ts_buildmaster.rb" file.

And it generates nice output too:

...
Directory object with cotta for in memory system
...
- dir should return sub directory
- dir should return a directory from a relative pathname
- should get file in current directory
- should create dir and its parent
- should delete dir and its children
- should do nothing if dir already exists
- should list dirs

Directory object with cotta for physical system
...
- dir should return sub directory
- dir should return a directory from a relative pathname
- should get file in current directory
- should create dir and its parent
- should delete dir and its children
- should do nothing if dir already exists
- should list dirs
...

Tricky Business of Being a ThoughtWorks Consultant

I have been thinking about this for quite a while and the topic came up again recently. So I figure I might as well write it down. Keep in mind that these are my personal opinion as ONE ThoughtWorker based on my situations.

Blog or not to blog

Historically, my blog thoughts come through several ways, all of which is under the condition that requires me to blog at airport or late at night (like now).

1. Feeling Pain

As the book "Project Retrospective" has pointed out, when you feel the pain of doing something, it is very important to stop for long enough period of time to reflect about what you are doing. As true as it is, which makes me a true believer of retrospective, I still have the same amount, if not more than usual, of billable work that I need to do. When I do come up with an idea of improving the process, I only create more work for me to talk with others to sell the idea, one more thing on my coaching list, and a task to implement the fix.

So at the same time that I have learned a new aspect on software development, I am also burned out more from my day's work, which makes it a lot less appealing to write a blog about it. Sometimes an idea has to go through several round to mature, so I hold off the blog. Then when it is mature, I have got used to it and don't feel the excitement and urge writing it down anymore.

Yes probably it would be ideal if I can slow down my other work so that I can keep my energy up everyday, which brings back to the title "consultant", as in, like it or not, I am being paid to live up to my previous promises as much as possible.

2. Learning something new

When I got to know about something new, I always find a way to apply it so that I can learn more. What that means is my afterhour will pretty much be occupied by those activities rather than blog about my incomplete thoughts about it. To name a few:

Behaviour-driven development: Liz from UK office did a presentation on jBehave in the China office. After that, I start visiting Behaviour-driven website to learn more about it. When we created Cotta project, we decided to give it a try and it turned out to be very well. But the IDE integration sucked very much and jBehave is a bit rough around the edges (like loading all behaviors), so we joined jBehave project, created Eclipse and IntelliJ plugin and fine-tuned the API, during which process the site building part of BuildMaster became mature.

Can you imagine me keep a log of all these activities while I am working on them in my afterhour? I didn't think so. Although now I realize that I have different take on BDD from Dan North and Dave Astels, maybe I should write something about it, well, after I finish this thing about build and release that I learned from last project, and my thoughts on how BuildMaster can rock the world.

3. Getting Into Interesting Internal Discussion

From time to time I met people, ThoughtWorkers or not, who are not afraid of expressing their opinion and discuss about it. A lot of the time the outcome of the discussion is that "We now know what we don't know" (in a good way, kind of like your feeling after finish reading "the World is Flat" from what I have been told). So is there a point of blogging that? Maybe, maybe not. How to blog it, what is a good balance to strike, I think this is the time when I feel that technical writers are being paid for a reason.

Discuss or Not to Discuss

A good project is a project that is under "Total Quality Control", i.e., every single detail of the process is constantly under review to see if there are any room for improvement. Since ThoughtWorks has got itself out of the staffing business, ThoughtWorkers are generally taking the lead on the projects.

Whenever a new ThoughtWorker joins the project, it is more than welcome, as part of ThoughtWorks culture, for him or her to question every detail of how this project is run. Sometimes debates will rise during the day so that everyone can understand not only the way it is but also the history behind it.

On the other hand, ThoughtWorks is brought in as "consulting firm", being paid to say what the right way is to run a process. So when the new member joins and debates start occurring, it can be potentially perceived by the client as the incompetence of the existing project lead, incompetence of the new member, mismatch of the personality between the parties, or weird management style of ThoughtWorks.

This happened to me several times, and luckily, my clients have been open enough to raise the question thus giving me a chance to explain. It happens a lot less now because the first impression that I am trying to make is that "there is generally no silver bullet". At the same time I am giving out suggestions, sometimes strong ones, I also tell the reason behind it and that those reasons are revalidated constantly.

Then that makes me look like the kind of consultants that I have had contact in the past, with who are all talks and never tells the client anything concrete, the exact kind that I loath and want to stay way from.

Hence the "Tricky Business".

There are a few more thoughts on this but they are not as well thought. Plus it is 12:30am now and I did plan on have a full night sleep for a change, well, after I finish ironing my shirt for tomorrow.

Going rSpec

After played with rSpec for a while, I have decided to convert BuildMaster on top of it. However, the test2spec script did not work for the project, so after a few experiment, the following regular expression helped me converting the majority of the syntax. I still had to convert the class definition to context definition manually, though.

1. Converting assert_equal(a, b) to b.should_equal a

replace "assert_equal\(([^,]+),\s*([^\n]+)\)" with "$2.should_equal $1"

2. Converting "test_..." methods to "specify ..."

replace "def test_([^\s]+)" to "specify '$1' do"

Speed of Unit Test

I have heard more than once from people saying that their unit tests are long because they have thousands, or tens of thousands of them.

Some would blame it on the complicated application architecture, or the language (like Java).

But if these guys can use a script language to run 132 tests, 219 assertions in just over half a second, maybe there is something to the idea of faster tests after all.

I am not saying it is easy, just that it is a goal worth pushing for.

One Week in SLC

The past four days just flew by and I am now sitting at the airport waiting for my flight back. I was hoping to fly my wife over so that we can visit the mountains around here during the weekend but it turned out to be much more expensive. That is OK, I still need to scout out the beach park where I am going to host the BBQ party next weekend.

I held on to a story card and paired with different developer each day. The domain that this story affects turned out to be the worst part of the application so each day I did a bit of clean-up and refactoring. The story dragged on a bit but I managed to make steady progress.

Since this is a .NET project, there are SQL server and IIS server to run on the laptop. After figuring out how to start and stop the service from the command, I wrote two Ruby driver class for them in the hotel room and checked them into BuildMaster.

So now I have a script that I can call to stop both service with:

services.rb stop

and all it does is:

services = [
BuildMaster::IisDriver.new, BuildMaster::SqlServerDriver.new
].each do |service|
service.send ARGV[0]
end

Back on the Road

After almost three months of local project, which I intend to log sometime soon, I am back on the road again.

This time I have my Verizon national access card that allows me get connected to do things like writing this blog at the Oakland airport. I could request the extra battery but with only 3-hour travel time I doubt the weight is worth the benefit.

Next project is an ASP.NET application and have quite a few ThoughtWorkers on it. I am afraid that is as much as I am allowed to say. I would like to whine about the painful download/install process I have been through in order to run the application, but I don't think anyone, like .NET or not, would care.

This project is on a 4-10 schedule which means once I get myself comfortable with the code and my role, I will be able to spend only 3 nights in the hotel and sleep at home 4 nights per week. Not that I am tired of traveling but there is something to "sleeping in your own bed".

A Good Read: Eight Barriers to Effective Listening

http://www.sklatch.net/thoughtlets/listen.html

Richard Watt hosted an OpenSpace session in the last HOD. I was thinking about checking it out at the time. But other things came up and I forgot about it.

I cam in the office today preapring for the next project and saw the printout at the printer. Figure I better grab a copy and blog it before that happens again. (Also need to buy "Getting Things Done").

Tying Up Loose End - Cotta

So I have rolled off another project and I am going on the next project right away next week.

Typing up a few the loose end at the moment, since I won't be able to have a couple of weeks beach time like I did last time.

I have started with Cotta because this is most close to being ready. Based on the feedback and my experience with Ruby file handling, I have finalized the core API. Release candidate is now available. Even though I still prefer setting all the API right on the first release, my busy schedule just won't allow it. I think I'll release 1.0 and leave it like that, until I get a chance using it or get some feedback from the field.

Of course, SourceForge is done, so in the meanwhile, only the website is available.

More blog coming up with last project, and other loose ends.

Why Lotus WebAccess is not My Favorite Webmail Client

I am on a machine that has 1.99GHZ dual CPU, 2GB of RAM, and 75GB of harddrive. Yet for every scroll up and down action in my inbox view, it sends a separate request. So even when I know the exact date an email was sent, it still takes forever to locate it.

And I noticed this is on the log out screen today:

* Secure: Delete all traces of my personal use of Domino Web Access and Web pages from this computer but keep program elements (boosts performance when next person logs on).

* More secure: Delete all traces of Domino Web Access and all other Web pages in the Temporary Internet Files folder.

Right... I understand what they are trying to do. But what's up with the "Secure" and "More Secure" notion? Apparently, the "Secure" is not so "Secure" after all when there is a "More Secure", isn't it?

Benevolent Dictator

This came up during a chat with someone while I was visiting the office.

This is a term that has its own history (http://en.wikipedia.org/wiki/Benevolent_dictator), but is now applied to a kind of management style for software development. In opensource development, Codehaus is probably the most famous for this management style (project despot). You should read "Codehaus Manifesto" if you want to understand it fully.

In Agile software development, the developers have (or should have) the absolute power on deciding how and when a business value can be delivered. While this brings lots of benefits, which we won't get into here, it does bring certain risks at the same time.

Different developers might have different opinion on design of certain part of the system, or no one have strong opinion over several sound solutions for the same problem. This is when benevolent dictator steps in, makes a decision in the team's place, and move on.

Of course, this is not nearly as easy as it sound. I did a little dig (very little, mind you) on my previous projects and pulled together the following criteria:

* The power should be used when necessary but ONLY when necessary.
* Any decision is a team's decision. Everyone takes the credit, everyone accepts the responsibility.
* Decision is not final in the sense that objections are noted and tracked, so that the wrong ones can be amended as early as possible.
* The title of "benevolent Dictator" is better granted.

At last, here is a PDF file that I have on my bookmark, which I think is related: "Agile Project Management.pdf"

Ask "Why Do You Ask"

As a traveling consultant working for a company pushing for agile, I keep on being asked for similar question repeatedly. "What do you do in stand-up?" "Why is TDD good enough to spend time on?" "What do you mean the velocity is 20 this week, 25 last week, and 18 the week before???"

Naturally, I trid to come up with the best answer based on the experience I had had, the books and articles I had read, the other conversation that I had had with other people, and the understanding that I had gained about the one who asked the question. Slowly, my answer to the above questions became bigger and bigger, with more details, more angles, as if I was getting closer and closer to the truth.

WRONG...

What I have realized in the past year, was that the best way to answer these questions, actually, are:

Give a 30-second brief answer. Stop, and ask: "Why Do You Ask?"

Different people have different situation, and are seeking the solution to different problem when asking the same question. In this case, giving a by-the-book answer, no matter how pragmatic or complete it is, does not necessarily solve that problem.

Here is a true case.

When I was traveling in China, I stayed late in the office one day to do some market research and spike on BuildMaster project. Another ThoughtWorker, who was doing some of his own thing, turned to me and asked "So what have you been working on lately?"

Since he expressed interest in DbFixture before, and that I knew he was on beach at the moment, I thought he was looking for something interesting to work on. So immediately I went on talking about the problem that I was trying to solve and the "grand" vision that I had with BuildMaster, Cotta and jBehave. I went on for 5 minutes (if not more) and realized none of them gained any interest. Going out on a limb, and out of a bit frustration, I finally asked the million-dollar question.

Turned out that he had an idea of starting an open-source project to help WebServices testing and would like to get me involved because of all the xFixture projects that I had been working on.

Granted, it was not exactly a straightforward question. Yet more and more I start paying attention to the motivations behind the questions rather than the questions themselves, and even sometimes observing the conversations others have.

And the rest is history.

I can now control myself much better nowadays instead of "blah blah..." for minutes.

Otherwise, I might as well say "42"

Why Eclipse is not My Favorite IDE

Need to find a way to exclude certain folders in my Eclipse Ruby project so that it does not show up in the Open Resource dialog. Searched on line and find this in Eclipse bug database:

There is now technology in place to support exclusion filters in the workspace.
You can create an exclusion filter in I20060110 by creating a linked resource
to the "null" file system.  The linked resource will hide any file system
resource with the same name, thus acting as an exclusion mechanism.

Maybe it is now a good time to reconsider the investment in Komodo

---

Update: Thanks for Robert for pointing this one out and I stopped pounding my head (a lot less at least):

apparently you can right click on the directory, select "Properties", and check the "Derived" check box, which will do the trick.

Loading All Test Cases in Ruby

Now that I finally understand that '$:' in '$:.unshift' is a variable reference, and that 'require ...' is literally a method call loading the file, I realized why there are no utilities in ruby to load all the test cases for you:

create a file 'ts_all.rb' at the root directory of the test with the following content:

# test/unit module that will pick up all loaded testcass automatically
require 'test/unit'

# find where the root directory of the tests is
root = File.dirname(__FILE__)

# do a name matching and iterate through all tc_*.rb files
Dir.glob("#{root}/**/tc_*.rb") do |file|
# load the file (assuming naming convention)
require "#{file}"
end


That is it! Executing 'ruby ts_all.rb' will just run all the tests.

BuildMaster 0.7.0 Released

Since it has already helped me a lot on setting up my personal website (http://www.shaneduan.com), I have decided to burn some midnight oil to release 0.7.0.

Now the whole BuildMaster site can be downloaded from here http://rubyforge.org/frs/download.php/11919/buildmaster-0.7.0-docs.zip. Source is also included so that it can serve as an example itself.

Still need to work on a better way to convert absolute path reference (for link, css and images references). And need to support a site descriptor like Maven or XSite so that the XHTML writing for the skin won't be so intimidating.

Geek Joke

Heard this one yesterday from Miles:

"C++ should actually be called ++C..."

Cotta Power

So I got one feedback already for Cotta project, neat!

So the million-dollar question is: "How is Cotta different from other files system project like Apache commos VFS, other than support a lot less file system types?"

My first response is, "Ouch, that hurt!"

Second response would be, "Hmm... I think we should think through how we structure our website so that people would not try to compare it with the wrong kind of library"

From the beginning, the main reason behind Cotta project is to make file operations easy to write and test (or verify the behaviours, in jBehave term). So the main focus, which is also the power, of Cotta, is as below in two parts. We are still bouncing our thoughts around it, and the final documentation should be published on cotta project website.

Domain Driven

In my humble opinion, the current File class in Java, or the FileObject in Apache common VFS, or the Url in the VFS system of Borland JBuilder OpenAPI (BTW, it is nicely designed as well), all has one fundamental problem. For this or that reason, they still don't match fully to the domain, which is, in this case, NORMAL file system operation.

The number one problem I have is: "A file is NOT a directory". Namely, you want to list the children of a DIRECTORY, and the parent of a FILE should be a DIRECTORY. You should be able read/write only the content of a FILE, not a DIRECTORY. When your code has a path string, it should already know that it is a DIRECTORY or a FILE.

Also, when I delete a DIRECTORY, it should be GONE, instead of telling me that it is not empty. If I want a safety measure, my filesystem implementation should do that for me. My list can go on for a while if you let me...

Cotta was implemented in that way. As it was written, we have already got a project that does file operations, so we laid out the expected behaviours of the classes one by one, and implemented the code one by one. When it comes down to physical file system operations, there has been no guesses. The resulting API is compact, which is exactly what we intended.

Test Driven

Or Behaviour-Driven to be exact. When I write a class that operates on filesystem, I'd like to be able to verify the behaviour of that class. Even being a seasoned BDD developer, I still like to spend less time writing more functionality. And cutting the time spend on setting up and tearing down environment is always a good choice, hence the birth of XmlFixture (part of XPE, but you'll have to download the package to see the document), DbFixture, and now Cotta.

When it comes to behaviour verification, a common pattern is to go through "Dependency Injection". With Cotta, the domain class just need to work on a "FileSystem" instance being passed in, or one of the Cotta objects (TFileFactory, TFile, TDirectory) that can be created from the FileSystem. During the verification phase, the filesystem is replaced with a "InMemoryFileSystem".

Cotta also has FileSystem decorators that can be used to verify the behaviour on file system like read only (ControlledFileSystem), or file system with disk problems (CatastrophicFileSystem).

What Next?

The idea of making file system implementation for JAR file, FTP etc., was an after thought. I forgot to check with Apache common VFS again to see if I am re-inventing the wheels (as it turns out, I am, more or less). I think the next thing for us to do is to focus more on the two aspects that I have laid out above, and make it clear on the website about the real market value of this project.

Why dive into the Red Ocean when you have already discovered a Blue one, right?

Cotta Project in Business

Finally we are in bunnies, well, sort of.

A project to ease the pain of using file/directory related operations. See detail at the front page that is still under construction: http://cotta.sourceforge.net

A Cool Story

I thought something like this might happen when I started BuildMaster, and I sure enjoyed seeing it unfold. Thank Nat for following up on each one of us, and thank Joe for putting it all together.

You can read it here: http://buildmaster.rubyforge.org/history

BuildMaster 0.6.0 Released

Finally! I get to write this blog about this little project that I have been working on. It is amazing how much you can get done when you are commuting by BART instead of driving.

It is a Ruby project that helps on the aspect of project releasing management, and website management.

The URL is http://buildmaster.rubyforge.org

Architectural Central Approach?

Another post that I didn't post.

"... Picking story using an architecture central approach ..."

My head is hurting...

Power of Whiteboards

Just finished a kick-off of a project in Beijing, before I came back to my base office, San Francisco, for the next assignment.

The project is a short one, with one week as QuickStart + Iteration Zero, four weeks of development, and one week of bug fixing + final release.

Needless to say, the time was short and there were lots of things to do in four days, gathering the requirements, breaking them down to stories, and coming up with a release plan.

As it turned out, the whiteboards became very useful. We found two of them available, and pulled up the blinds to make the glass wall a third one.

One board at the end of the room, half filled with goal of the week, and the other half as the parking area. With the goals listed, we were able to keep ourselves focused. And we were also free to write topics or thoughts down on the parking area as we discuss, so that we don't get side tracked. From time to time, we check these items to cross out the things that have been taken care of.

The other whiteboard stands right by the door, with a little space in front of it, for requirements discussion. We use it to write down the role and goals, and draw process diagram on it again and again until we get it right. Then we divide it into stories with numbers mapped to different part of the diagram and took digital photos, so that we can easily see how the stories fit into the whole pictures.

The glass window turned into a over-all design board, with cards indicating internal components, external components, screens, and running list for the notes on each of the above. For example, there are WebServices that we need to call as part of the application. So we had a card with the WebServices name on it. During the requirement discussion, whenever we realized that we need to use the WebServices to retrieve or update some information, we will pull the card over, put it as part of the process diagram on the discussion board, and start writing down the kind of interface we need from the WebServices. In the end, we can just take a look at this card and come up with a list of dependencies that this application has on the WebServices.

At the end of the week, looking at the three boards full of arrows, blocks, notes, I just cannot imagine how we can do without them.

A Sample of Practices Interdependencies

More than once, I have got the kind of inquiry as "I'd like to adopt agile(XP) practices, which ones do you think that I should try first. It is always a hard question.

One main reason is that I very much intend to say "every team member should have the XP book by hand, the principle up on the wall, be open-minded and try to adopt the whole practices as a whole". It might sound intimidating but here is the reason:

All These Practices are Related
All these practices are supporting each other. I know it has been spelled out since day one that XP came out, but I still think it is a point that gets forgotten sometimes, especially when the team is under pressure.

In one of the training engagement that I was involved in, after some training sessions of several practices that we think the client needed, in an effort of making sure that the student understand that these practices are much better working together, we created a little dependency diagram using Graphviz.

We did it by creating the graphics file as following (detail omitted):

digraph g {
"Collaboration" [label="Collaboration"];
"Collocation" -> "Time-Boxed Iterations and Releases";
"Collocation" -> "Continuous Improvement";
...
"Test First" -> "Collocation";
"Test First" -> "Collaboration";
"Test First" -> "Automated Testing";
...
"Time-Boxed Iterations and Releases" ->"Collocation";
"Time-Boxed Iterations and Releases" ->"Collaboration";
...
}

The result was pretty staightforward:



Of course, in the presentation that we gave it looks a lot nicer with bi-directional arrows and stuff. But the idea is the same, you cannot pull one out without affecting the whole picture.

On the other hand, if you want to adopt the practices, trying them one by one will probably only make things slower and harder.

A Geek's Version of a Joke

If you have Ruby and Watir installed, here is my "Geek's Version of 'How far do you want to go for a deep joke'". Enjoy

require 'watir'

ie = Watir::IE.start("http://www.sbpoet.com/2006/01/how_far_would_y.html")
while link = ie.link(:text, /(blond)|(joke)|(this one is great)|(>CLICK<)/)
  link.focus
  link.click
end

Personally I don't find it THAT great, but like Steve Boswell always says, "I digress".

Less Leads to More?

Just found out that I never published this...

Sometimes, less does leads to more.

After I came to Xi'an, I started working on a distributed project. The development was splitted between Xi'an China and Bangalore India office.

Based on the observations, I decided to cut the development team in China down as well so that we can be more focused during the communication between India and China teams. So rather than 7-8 developers, we only have 4 developers, including me and Jake to work on the project. It worked out pretty well. We have got to know with each other and built a relationship that is good enough that we are free to use phone/IM/skype/iChat/campfire whenever any one of us needed something.

During the weekend, I had a chance to chat with another ThoughtWorker who is working as PM on another project and he gave me another example (I don't remember the example now...)

Building China Office

Flew in China on Feb. 22nd, I got myself busy right away. The experience is still very exciting and sometimes the tension is killing me.

So these are what we have been doing in China office:

Building the China Website

We were finally able to translate the ThoughtWorks website into Chinese and started hosting it on our China domain (http://www.thoughtworks.com.cn/index.html). Arguing about how to translate a sentence or even just a term has been very much fun and educational at the same time.

On the principle of "Fixing the Windows", we improved the staging environment by reducing the downtime from 30 minutes every hour to 5-6 minutes every hour. Tests were also written for two bugs that we have found during the translation test.

Building the Awareness

ThoughtWorkers in China have been pretty active in getting in touch with the local communities. We have made arrangement with several universities in Xi'an to give the student some exposure of agile software development.

Other ThoughtWorkers working on projects in the other cities are also presenting to the local universities and communities.

As for me, I am going to give presentation to BEA user group meeting this month at Shanghai (http://dev2dev.bea.com.cn/usergroup/20060460.html), titled "Agile in Practice" (originally "XP in Practice").

I think this is an interesting topic to give presentation to. Whenever after someone giving the presentation about agile development, there was always a long session of questions and answers. This presentation can serve as a follow-up by targeting those questions.

With the help from other ThoughtWorkers, I have collected feedback from the projects that we have worked on in China, grouped and categoried them, so that we can show what agile is from yet another perspective. The more I work on it, the more it reminds me of the book "XP Installed".

So I am concentrate more on the photo shots and project samples. It already has over 30 slides and over 10 megabytes in size. Maybe I will have to cut it a bit to fit into the 60 minutes that I have for this presentation.

Distributed Project

Of course, the one and only activity that leads directly to revenue in ThoughtWorks is working on a billable project. Since the first day that I entered Xi'an office, I have been assigned to a distributed project.

The client's headquarter is in UK, and this project involves their offices in Spain and France. So we have two BAs working in Europe gathering requirement. The development team was originally in our Bangalore office. Due to the lack of resource, another small development team was formed in Xi'an office and some of the work has been shifted here.

As you can imagine, the communication has been identified as a risk from day one. We have taken various measures to make sure that the developers understand the code base as well as the requirement, and that everyone is update to date on the project status. This includes Yahoo! IM configuration so that everyone can tell which story each other is working on, daily standup meeting through iChat between Xi'an and Bangalore, and daily BA meeting conference, etc. Of course, Campfire made by 37 signals helped a lot on this issues, gaining more of my already established respect for them.

Just within a month, there has already been a change of direction from the customer, which proved one more time that change is constant and planning is an on-going activity. The expats in the China team left, except me, Jake who knows the code base, and Sagar who knows the business requirements.

Performance Testing Project

This is the project that we engaged for the last two weeks. Even though it did not happen there are things that I have learned through out the process.

Business

The project came to us as a returned business. Basically, when party B did a contract for party A to build a system, they are required to find a third party to do a performance testing for the system so that they can prove that this system will work with the estimated workload.

This came to us as a returned business. It sounded like a routine contract that will just benefit all parties. Party B gets to finish its contract and get paid, party A will get to know better about the non-functional behavior of the system. As I have learned, the rate for performance testing contract is normally higher than other contracts. So it would have been a good short contract for me (working with a long time ThoughtWorker, Matthew Short) before I head back to China. And ThoughtWorks doesn't have to put me on beach.

For reasons that I cannot publicly post, the contract fell through even though technically we have a pretty good idea of who we need to work with, what we need to produce and what to do to get the data.

So now everyone lose.

This is my first fall-through project. Not totally surprised since I cannot imagine that all engagements that we have turn into a project, but still I am still a bit uneasy to the fact that despite all efforts, we just cannot influence the other parties to prevent them from taking actions that jeopardizes the project.

Or can we, I wonder?

Technically

I have done some performance testing for various projects, but always as part of the project iterative tuning process. It mainly focused on the profiling part rather than the performance benchmarking as a whole.

In the last project in China, the client who hired us had a contract with the customer, which put the specific performance requirement as part of it. It looked pretty vague to be, when reading through that part of the contract, that they are just numbers that can be easily met as long as you are free to choose the hardware, or could be almost impossible if the hardware is fixed.

In preparation for project by referring to the one that we did before, I have learned about performance testing methodology, Transaction Processing Performance Council and their benchmarks (http://www.tpc.org.)

Because the application is going to be deployed on a Linux system (while the last one was on a Windows system), with a little research and the great help from Barrow Kwan, we were able to have a Linux at our disposal, run resource monitoring programs, process the data into csv files, port them over to Microsoft Excel and generate the performance chart.

I guess I didn't totally lose on this after all.

Three Announcements from ThoughtWorks

Internally, ThoughtWorks has lots of interesting emails about announcements and discussions. Here are three announcements that I got this week that I think worth logging.

Selenium on OpenQA
Selenium has moved to OpenQA (http://www.openqa.org/selenium/). For those who don't know, Selenium is a web testing technology written to various degree by a couple of ThoughtWorkers and friends. It turns your browser into a web testing tool so that you can test your web application in any browser on any platforms. I have used to in some projects and I strongly suggest you check it out.

CruiseControl Better and Greater
I have been using CruiseControl on every single project that I have worked on in ThoughtWorks. And I have to say that they have come a long way. Now the latest version of CruiseControl launches web server along with the CruiseControl server, and has a "build" button on the reporting page!

Event Announcement: CI and Testing
Greetings Continuous Integration Testers,

This email is being sent to announce an upcoming event for everyone interested in continuous integration and the type of automated testing associated with it.

Jeffrey Fredrick and Paul Julius are cohosting an event that will focus on these topics. The event will use Open Spaces to structure conversation, understanding and innovation.

What: Open Space event discussing all aspects of CI and Testing, together
Where: Chicago, IL
When: Early April (final date tbd)
Who: Everyone interested in CI and Testing
Cost: Free

We'll be inviting people for all manner of projects and places. In fact, feel free to pass this invitation along to anyone that you think will be interested.

For us to finalize the details of time and place we need to get a feel for how many people are likely to attend. If you are interested in attending please join the CITCON mailing list at:

http://groups.yahoo.com/group/citcon

and post an introductory message. In your message it would be useful if you could indicate any topics of special interest and also how likely you are to attend.


Sincerely,
Jeffrey Fredrick (jtf@agitar.com)Paul Julius (pj@thoughtworks.com)

Hey Look! A Delete Button!

This just proves that it sometimes takes a while for even the best to see the light.

Finally GMail has a "Delete" button! Now I just need to un-train myself to click it instead of that drop-down list.

Finally Yahoo Mail logs in using SSL by default! Now I just need to stop clicking on that link on the login page.

曲突徙薪无恩泽，焦头烂额为上客 -- An IT Phenomena

This is a thought that I had during my two week vacation.

Sitting at in-law's home, I grabbed a book about Chinese idioms for my nephew to kill some time, and a story caught my attention.

The story started when a guy built a beautiful house, and invited his friends and neighbors over for a house warming party. As everyone else admiring the house and congratulating the host, one guest, however, made a comment, "The chimney is too straight up and the fire woods are too close to the stove. If you don't make the chimney into an S shape and move the logs away from the fire, chances are that the house will catch fire before long."

The host, however, was not very happy to hear these, thinking this guest is saying words of such bad luck on such a good day.

As the luck went, the house did catch fire a month later. All the neighbors came to help. Eventually the fire was put out, leaving everyone with burns and bruises.

The host was very grateful. He opened the kitchen immediately. He had everyone sitting in the best seat, and treated them with the best food and drinks.

During the meal, another guest, who also came to the house-warming party, brought up the topic, "A month ago during your house warming party, there was a guest that predicted that this would happen. Had you listen to him at that time, chances are that your house will not catch fire and we would not be sitting here enjoying the feast. Don't you find it backwards, when the one who recommended the modification on the chimney and moving away the log was not being appreciated, while the rest of us with burns and bruises are being treated as the best guests (曲突徙薪无恩泽，焦头烂额为上客)?"

The host came to realize the mistake that he has made. He immediately sent for that guest, and treated him with the best seat in the house.

As I work through projects and in different organizations, in ThoughtWorks or before ThoughtWorks, I noticed that the person who is good at putting out the fires are always the one that gets the limelight. The person could be a manager who knows how to talk to the customer to cut the scope in the last minute, or find ways to persuade the developers to work overtime, or he/she could be a developer who is good at digging through a bunch of messy code and come up with a hacking patch for a last minute bug.

On the other hand, there don't seem to have enough attention to the sources of all these crises. People tend to pay most attention to how to resolve these crises rather than how to prevent these crises. So advice was not given any thought and measures were not taken because "if it aint broke, don't fix it".

And there are more crises.

And it is more important to keep the ones who knows how to put out fires with burns and bruises.