Things You CANNOT Get Certified For

So...

I was chatting through IM with a friend of mine about something that I read recently (Crystal Clear). And the conversation, as hard as I tried, just went downhill from there. I got really agitated in the end, even after I realizing that half the time it is one of those flag words that got on my nerve.

Looking back at the log, I realized that I thought he knew what I have been doing at ThoughtWorks for the last two and half years, and he probably thought I had no idea about the topic in this book. During the whole conversation, we were on different depth level of the topic. I guess this is one more case proving that you should watch out for "Barriers for Effective Listening" and ask "Why do you ask".

With all that behind, I was still stunned at the fact that no matter how well so many people try to protect a good idea, there are still people out there trying to profit from it by coming up with bogus stuff in its name. And one very good example is "certification". Because apparently they succeeded in making my friend think that is what it is all about, another certification.

So here is a list of things that you cannot get certified for. Instead, only your peers who work with you day in and day out can rate you, subjectively.

• Doing TDD even when you are under the pressure to delivery. Everyone can pass a test and do a little practice in their own pleasure.
• Have the courage to speak up when there are things that you think is wrong. "Do you think you should speak up when you see something wrong?" "Yes." "Good! You are certified!"
• Sit-Together. How can you certify that, just by sitting together for a week? It is one thing to say "yeah, it is a good idea", it is another thing to go to the extreme length of getting the tools yourself and start taking cubicles apart. (Yes I have met someone who really did it).
• Code Co-ownership. I am sure everyone can check that checkbox to get certified. But how can you certify a person's willingness to learn as much as the codebase, make sure that the design is as clear as it can be, and pass on the knowledge to others as soon as possible?

Sorry but "certification" is such a binary rating system, that it just does not fit into what it is in the real world.

Then of course, it is always good to get some education on the topic and have a proof that you have finished them successfully.

But that hardly gives anyone the authorization to say that "It is nice, but it is really really hard to do a business-driven situation".

Not when the purpose of the whole thing is "Deliver the business value in whatever the best possible way".

(BTW, did you realize that if you mistype blogspot.com as blogpsot.com, it is an actual website selling ads?)

BuildMaster Surgery - Moving to Cotta

I have been struggling with the File operations for a while, before I fully understand what is going on.

After I released Cotta project, the way File operations works in Ruby is like a hideous stain on top of a shining gem, always bugged me, especially when there are these nice classes like Pathname and StringIO already exist in Ruby.

Finally, my curiosity of making it better got the best of me, and I ported BuildMaster over to Cotta Style.

I have decided to call the files CottaFile and CottaDirectory so that I don't have to worried about the name space issue. Since BuildMaster does quite a bit of shell operations, I added method "shell" to the cotta instance so that the in memory version can log them for tests.

All things are happening behind the scene. The following is a code snippet demonstrate the new API:

   #system implementation is injected here
   cotta = BuildMaster::Cotta.new(BuildMaster::InMemorySystem.new)
   file2 = cotta.file('dir2/file.txt')
   file2.exists?.should_equal false
   # parent directories are created automatically
   @file.save('my content')
   @file.copy_to(file2)
   file2.exists?.should_equal true
   file2.load.should_equal 'my content'
   file2.read {|file| puts file.gets}

The API still need to be fine tuned but they provide all the operations that BuildMaster needs for the moment. Hopefully, someday something like this can be part of the ruby library.

BuildMaster Surgery - Moving to rSpec

This is the first part of the BuildMaster surgery that I have just finished (well, 95%) that have been keeping me up late every day. I will write the second part as a separate blog.

Moving BuildMaster to rSpec turned out to be not as smoothly as I hoped. Hopefully this blog will help anyone who is interested on similar task. This is what I have figured out so far so if there is a better way of doing it do let me know.

Syntax Change

If you have tons of test/unit scripts already, changing the code to use the rSpec syntax could be boring. They do have "Test::Unit Migration" support. However, I didn't get to take full advantage. It just generated "migration error" with no detail information for me. So I had to use some regular expressions to get the most of the change done, which I have blogged in "Going rSpec".

Specification Sharing

The "Context API" document on rSpec teaches you how to call helper methods in your specification code. However, it does not show you how you can make two context share the specifications, which is very useful if you want to make sure that two different implementation of the same interface shares the exact same behaviors.

Brain from Pivotal showed me that you can achieve this by using 'extend' (instead of include). So you can write the common behavior specifications in one file:

module CottaSpecifications

def register_cotta_file_specifications
  setup do
  # this one uses the @system that will be set up by each implementation spec setup
  @file = BuildMaster::CottaFile.new(@system, Pathname.new('dir/file.txt'))
end

specify 'file can be created with system and pathname' do
  @file.name.should_equal 'file.txt'
end
end

Then you can include those common behavior in your specification file for the implementation:

require 'cotta_file_specifications'
module BuildMaster
context 'Cotta file' do
  # extending the module so that you can call the methods
  extend CottaSpecifications
  setup do
  # setup the implementation for specifications
  @system = InMemorySystem.new
end

# this call registers all the spefications
register_cotta_file_specifications
end
end

Hopefully the above code makes sense to you. If not, let me know.


Running All Specifications

Again, thanks Brain to explain the reason and provided me with a solution.

Somehow, all the registered contexts are run at the end of each context registration. So my code for loading all the tests will now make the same context run again and again:

If you have context A and B in tc_a.rb and tc_b.rb and you require each file one by one. After "require 'tc_a'", the specifications in Context A will run. After "require 'tc_b'", the specifications in Context B will run, this time ALONG with all the specification in Context A. If you have C, D, etc., then specifications of Context A will run again and again at the end of each following require.

So Brain wrote his own specification runner, which will hold off the execution until all the contexts are loaded:

require 'rubygems'
require 'spec'
#require 'diff/lcs'
dir = File.dirname(__FILE__)
#require "#{dir}/../test/common_test_case"
require 'test/unit'
Test::Unit.run = true

args = ARGV.dup
unless args.include?("-f") || args.include?("--format")
  args << "--format"
  args << "specdoc"
end
#args << "--diff"
args << $0
$context_runner  = ::Spec::Runner::OptionParser.create_context_runner(args, false, STDERR, STDOUT)

def run_context_runner_if_necessary(system_exit, has_run)
  return if system_exit && !(system_exit.respond_to?(:success?) && system_exit.success?)
  return if has_run
  exit context_runner.run(true)
end

at_exit do
  has_run = !context_runner.instance_eval {@reporter}.instance_eval {@start_time}.nil?
  run_context_runner_if_necessary($!, has_run)
end

To use this, all I needed to do was to require this file at the beginning of my "ts_buildmaster.rb" file.

And it generates nice output too:

...
Directory object with cotta for in memory system
...
- dir should return sub directory
- dir should return a directory from a relative pathname
- should get file in current directory
- should create dir and its parent
- should delete dir and its children
- should do nothing if dir already exists
- should list dirs

Directory object with cotta for physical system
...
- dir should return sub directory
- dir should return a directory from a relative pathname
- should get file in current directory
- should create dir and its parent
- should delete dir and its children
- should do nothing if dir already exists
- should list dirs
...

Tricky Business of Being a ThoughtWorks Consultant

I have been thinking about this for quite a while and the topic came up again recently. So I figure I might as well write it down. Keep in mind that these are my personal opinion as ONE ThoughtWorker based on my situations.

Blog or not to blog

Historically, my blog thoughts come through several ways, all of which is under the condition that requires me to blog at airport or late at night (like now).

1. Feeling Pain

As the book "Project Retrospective" has pointed out, when you feel the pain of doing something, it is very important to stop for long enough period of time to reflect about what you are doing. As true as it is, which makes me a true believer of retrospective, I still have the same amount, if not more than usual, of billable work that I need to do. When I do come up with an idea of improving the process, I only create more work for me to talk with others to sell the idea, one more thing on my coaching list, and a task to implement the fix.

So at the same time that I have learned a new aspect on software development, I am also burned out more from my day's work, which makes it a lot less appealing to write a blog about it. Sometimes an idea has to go through several round to mature, so I hold off the blog. Then when it is mature, I have got used to it and don't feel the excitement and urge writing it down anymore.

Yes probably it would be ideal if I can slow down my other work so that I can keep my energy up everyday, which brings back to the title "consultant", as in, like it or not, I am being paid to live up to my previous promises as much as possible.

2. Learning something new

When I got to know about something new, I always find a way to apply it so that I can learn more. What that means is my afterhour will pretty much be occupied by those activities rather than blog about my incomplete thoughts about it. To name a few:

Behaviour-driven development: Liz from UK office did a presentation on jBehave in the China office. After that, I start visiting Behaviour-driven website to learn more about it. When we created Cotta project, we decided to give it a try and it turned out to be very well. But the IDE integration sucked very much and jBehave is a bit rough around the edges (like loading all behaviors), so we joined jBehave project, created Eclipse and IntelliJ plugin and fine-tuned the API, during which process the site building part of BuildMaster became mature.

Can you imagine me keep a log of all these activities while I am working on them in my afterhour? I didn't think so. Although now I realize that I have different take on BDD from Dan North and Dave Astels, maybe I should write something about it, well, after I finish this thing about build and release that I learned from last project, and my thoughts on how BuildMaster can rock the world.

3. Getting Into Interesting Internal Discussion

From time to time I met people, ThoughtWorkers or not, who are not afraid of expressing their opinion and discuss about it. A lot of the time the outcome of the discussion is that "We now know what we don't know" (in a good way, kind of like your feeling after finish reading "the World is Flat" from what I have been told). So is there a point of blogging that? Maybe, maybe not. How to blog it, what is a good balance to strike, I think this is the time when I feel that technical writers are being paid for a reason.

Discuss or Not to Discuss

A good project is a project that is under "Total Quality Control", i.e., every single detail of the process is constantly under review to see if there are any room for improvement. Since ThoughtWorks has got itself out of the staffing business, ThoughtWorkers are generally taking the lead on the projects.

Whenever a new ThoughtWorker joins the project, it is more than welcome, as part of ThoughtWorks culture, for him or her to question every detail of how this project is run. Sometimes debates will rise during the day so that everyone can understand not only the way it is but also the history behind it.

On the other hand, ThoughtWorks is brought in as "consulting firm", being paid to say what the right way is to run a process. So when the new member joins and debates start occurring, it can be potentially perceived by the client as the incompetence of the existing project lead, incompetence of the new member, mismatch of the personality between the parties, or weird management style of ThoughtWorks.

This happened to me several times, and luckily, my clients have been open enough to raise the question thus giving me a chance to explain. It happens a lot less now because the first impression that I am trying to make is that "there is generally no silver bullet". At the same time I am giving out suggestions, sometimes strong ones, I also tell the reason behind it and that those reasons are revalidated constantly.

Then that makes me look like the kind of consultants that I have had contact in the past, with who are all talks and never tells the client anything concrete, the exact kind that I loath and want to stay way from.

Hence the "Tricky Business".

There are a few more thoughts on this but they are not as well thought. Plus it is 12:30am now and I did plan on have a full night sleep for a change, well, after I finish ironing my shirt for tomorrow.

Going rSpec

After played with rSpec for a while, I have decided to convert BuildMaster on top of it. However, the test2spec script did not work for the project, so after a few experiment, the following regular expression helped me converting the majority of the syntax. I still had to convert the class definition to context definition manually, though.

1. Converting assert_equal(a, b) to b.should_equal a

replace "assert_equal\(([^,]+),\s*([^\n]+)\)" with "$2.should_equal $1"

2. Converting "test_..." methods to "specify ..."

replace "def test_([^\s]+)" to "specify '$1' do"

Speed of Unit Test

I have heard more than once from people saying that their unit tests are long because they have thousands, or tens of thousands of them.

Some would blame it on the complicated application architecture, or the language (like Java).

But if these guys can use a script language to run 132 tests, 219 assertions in just over half a second, maybe there is something to the idea of faster tests after all.

I am not saying it is easy, just that it is a goal worth pushing for.

One Week in SLC

The past four days just flew by and I am now sitting at the airport waiting for my flight back. I was hoping to fly my wife over so that we can visit the mountains around here during the weekend but it turned out to be much more expensive. That is OK, I still need to scout out the beach park where I am going to host the BBQ party next weekend.

I held on to a story card and paired with different developer each day. The domain that this story affects turned out to be the worst part of the application so each day I did a bit of clean-up and refactoring. The story dragged on a bit but I managed to make steady progress.

Since this is a .NET project, there are SQL server and IIS server to run on the laptop. After figuring out how to start and stop the service from the command, I wrote two Ruby driver class for them in the hotel room and checked them into BuildMaster.

So now I have a script that I can call to stop both service with:

services.rb stop

and all it does is:

services = [
BuildMaster::IisDriver.new, BuildMaster::SqlServerDriver.new
].each do |service|
service.send ARGV[0]
end