Label Blog Post!

If you are a blogger user, did you know that you can put labels on your posts???!!!

I have been jealous of WordPress users for a long time but I am just too busy/lasy to set up a server of my own.

Until the other day I noticed a blog post with labels hosted on blogspot. Well, I guess that says how sloppy I have been on writing this blog.

Again, thank you Google!

So Close

Working on several things at the same time and they are all getting very close but just not quite. I have to say this puts a bit of stress on me.

At guidewire, I am working on a feature that involves analyzing the domain graph so that we can do things like record purging, archiving, data relationship analysis, etc. I think I have the main graph finally worked out (even have a primitive UI to show for it). I have the tests running and one of them can generate a Wiki content so that the document is always up-to-date. However, at the edge of the graph where the domain objects inside the graph has references to or from objects outside the graph, it gets a little complicated. I am keeping down a list on the wiki and it is getting more and more complete..... Just not yet.

I have arranged next sprint (two-week period) to start on next week so I was really hoping that I could finish this before I kick it off, because it will make a lot of stories easier to write. But for various reason I just couldn't quite just make it. I think I'll have to spend next Tuesday (Monday being a holiday) to make sure that the stories for the next sprint is good to go and that QA is on board with what I want to do. Adam has got me an account with Pivotal's tracker. Things are all good, just that being one-man team is not easy because you still have to do the same amount tracking if you want to still get the benefits for agile development.

At home, I have been busy with mainly two projects, BulidMaster and Selenium-Ruby.

For BuildMaster, I finally figured out how to use RubyGem to archive the file, and why reading image file with Cotta cannot read the full content (you have to call io.binmode even if you open it with 'wb' argement). I am getting very close to have Cotta support archive and zip so that we can just do 'dir.archive_to(target_file).zip' and get a zip file. I need to redesign the BuildMaster website so that project releasing, website building and cotta are presented as three separate pieces.

For Selenium, that is something that I am still not ready to share but I think it will be interesting as well. I will need to modify the selenium server and again I am getting very close.

Yet Another Keyboard Layout?

Interesting... Although switching between QWERTY and Dvorak, and telling people about Dvorak is already hard enough. If anything, I would be glad to see the day that Dvorak prevail.

Nevertheless, the comparison applet is still something very interesting.

This Made My Day

Enough said...

Bye bye Eclipse! I know I am supposed to say "thank you and miss you" and stuff, but I really don't want to lie.

Domain Based Web Testing

This has been in my mind for some time and with the creation of Selenium-Ruby I finally have someplace to put it:

http://selenium.rubyforge.org/doc/domain-based-web-testing.html

The idea of domain driven is also behind the BuildMaster and Cotta project.

fix svn tagging in sourceforge

I have been having tagging problem with Subversion in Sourceforge. It took me several days to realize that this is not going to be resolved by itself. Once again, Google helped me found the solution:

You need to point your working directory to a different repository URL, through command like:

svn switch --relocate https://svn.sourceforge.net/svnroot/cotta/trunk https://cotta.svn.sourceforge.net/svnroot/cotta/trunk

You just need to replace 'cotta' to the unix name of your project.

So Much for Java Generics

I always thought that Java generics is supported through compiler is a bit weird, and after 5 hours of email and spike, I finally know why.

Don't get me wrong, I love generics. Even though with TDD I can catch my mistakes most of the time, but with generics I can see my mistakes even before I run the tests. I won't say no to that.

Anyway, it all started with me starting a prove-of-concept project with another friend. So I am setting up libraries and figured I'll use jMock's constraint framework, after I convert it to support generics. Hence started my journey.

I checked out jmock code and was surprised to find out that there is a jmock2 directory. By the document that has been written, it is a way to stay with jMock style but don't lose the type safety. Cool! What is more, the constraint framework has been extracted into something called hamcrest.

Through email, Nat pointed me to the project in sourceforge (duh): http://sourceforge.net/projects/hamcrest. It looks very promising, even have the adapter for JUnit3, JUnit 4, and TestNG. It is apparent the intention of this project.

After I calmed down with all the excitement, I noticed one thing strange: The Matcher interface, the one interface that matching framework is evaluating matching logic, takes the "Object" type, rather than the generic type. What this means is that all the matching instance will have to do a type cast.

"That was silly", thought I, "Surely they know better than this". So I modified the interface in hamcrest, ran all the tests and proudly submitted my patch to Joe.

Turns out that the issue is not hamcrest itself, but jMock.

If you have two methods of the same name that you want to mock, say
public void test(int value);
public void test(String value);

And you provide the expectation
mock.expect("test").with(eq(1))

Upon the invocation of a.test("arg"), it will try to find the match by checking the matcher returned by eq(1) as well as that returned by eq("arg"). When comparing to eq(1) it will fail because from the instance there is no way you can figure out what the generic type is, and invoking by force will cause a class cast exception because there is a class cast injected by the compiler.

So in the above invocation, instead of having something like "unexpected invocation error", you end up with a weird ClassCastException error.

Bloody hell.

SCRUM at Guidewire

A month ago, I have joined guidwire, a local start-up building insurance products.

At guidwire, we are using SCRUM. So every new hire will get a book of "Agile Software Development with SCRUM". Somehow I missed it but I found it soon enough and got a copy right away.

I don't know why I never got a chance to read this book and I am really glad that I did. SCRUM is very much like the project management part of what I have been doing for the last several years. This includes the details like format of SCRUM meeting (daily stand-ups), feature backlog (master story list), goal driven team development.

On google video, there is a presentation done by Ken himself: http://video.google.com/videoplay?docid=-7230144396191025011&q=ken+scrum.
In it, he talked about SCRUM and XP complements each other. I didn't understand it before, and now I think it is really true.

Typing Contest : Dvorak vs. QWERT

After switching to Dvorak for almost a year, I decided to switch back to QERTY, for two reasons.

• I was pairing at least 6 hours a day and was driving everyone mad with the layout switching
• I wanted to see how fast I was with QWERTY.

Before I switched back, I timed myself in typing out the page "Dvorak Follow-Up". Now that I am fluent with QWERTY again (I should have done that earlier but I was too lazy), I just typed the page again using QWERTY.

And the result is...(drum...)...

DVORAK!

Time used with Dvorak: 9:28minutes.
Time used with QERWT: 9:55 minutes

I cannot say that I am not surprised. After all, I have been typing QWERTY almost my entire life and I only used Dvorak for a year. But here you go.

On the other side, 30 second is very small. I could just be tired tonight.

I also finished the "Joy and Pain with Fitnesse" in 29:30 minutes using QWERTY. If I ever switch to Dvorak again, I might do another contest.

Scrum et al. at Google Vedio

http://video.google.com/videoplay?docid=-7230144396191025011

I thought it was interesting on the part that SCRUM works with XP. I always thought SCRUM was too much on the management side, and now I think I know why.

Too bad most of the guys on agilechina cannot see it. Copyright issue I suppose.

Code Snippet, with Color!

I did it!

Now build master can grab code snippet from another file and apply syntax highlighting using syntax.

Next step is to figure out a way to specify the snippet validations.

I am dead tired right now and I have a 9am meeting tomorrow. So I just updated two (actually one and half) pages to show it off.

• http://buildmaster.rubyforge.org/getting-started.html
• http://buildmaster.rubyforge.org/docs/tag-references.html

I also fixed the document source and history link at the end of each page so you should be able to see the source file now.

Last Day as ThoughtWorker

Last day as ThoughtWorker, writing this blog that I have been putting off until the last minute, I find myself lost in thoughts in all the things that I have experienced since I joined ThoughtWorks 30 months ago.

It has been a nothing short of life-changing experience. As a matter of fact, it is with these changes that I have decided that in order for me to push even further, I will have to go out and look for good opportunities that can help me, rather than waiting for them to happen. As I going through the projects, my expectation on what I can do on the project has changed dramatically.

When I first joined, I was just happy if I can merrily write my tests, be happy about the 40-hour week and that the CEO is not trying to cash in one million dollars every quarter when the company stock is not going anywhere (like some company that I have worked for). Now on my last day, I have found myself comfortable coaching agile software development and influencing project management. I have experienced all the things a senior developer can do in a project, and even have had a short taste of being an IM and PM. I have been to China, twice, to lead projects. I have been on more and more discussion and presentations that involve more than "TDD" or "patterns". I am even pulling an effort with several others right now to write a book that we hope can be published.

Like my fellow Borlanders showing me the way to be a good developer, my fellow ThoughtWorkers have helped me understand what it takes to have a success project, besides technical execellence. I created this blog for my experience in ThoughtWorks. I have grown fond of it over the time and I am going to keep it up for my experience in applying agile practices.

So to wrap it up:

First several books that I read as a ThoughtWorker:

• Enterprise Application Architecture
• Domain Driven Design
• pragmatic Project Automation

Most recent books that I read as a ThoughtWorker:

• Tipping Point
• Blink
• Build to Last
  
• Blue Ocean Strategy
• Crystal Clear

Things I enjoyed doing in ThoughtWorks:

• Talk to Roy (don't worry about subjects)
• Meet old timers, they will show you why the idea that "employee managed company" has a chance in ThoughtWorks. (If someone was surprised that you were not one, it is a compliment)
• Meet at least one ThoughtWorker from each office from other country, especially the noisy ones.
• Request traveling to other country for a project.
  

Things that would be fun to do in ThoughtWorks:

• Tell Obie that ruby won't fly because it does not have an IDE like IntelliJ and it is not type safe. (well, maybe not anymore since he has been fighting that battle for a while and getting very good at it)
• Tell Hammant that you love Spring
• Buy beer around the table and ask for airplane takeoff and landing stories

Things that I did not enjoy doing:

• On beach for more than two weeks
• Travel weekly with connection flights each way.
• Stuck in an airport for a delayed flight for 7 hours
• Stuck in an airplane on the ground for 3 hours
• Saying goodbyes
• On your last day, be notified that you won't be covered by medical insurance anymore (So much for chilling out before the next job), even though you have accumulated over four weeks of vacation, and you could technically take your vacation before you send in the two weeks' notice, getting covered and accumulate additional vacation at the same time. The reason? You are not worth risking the liability anymore, now that you are not going to make money for the company and all. (Not in that word but I get the message)
  

I would have put boring period of the project in there but the following are what I have done each time I got bored:

• Learned Dvorak
• Created open source projects: http://www.shaneduan.com/opensource.html
• Read books
  

Things You CANNOT Get Certified For

So...

I was chatting through IM with a friend of mine about something that I read recently (Crystal Clear). And the conversation, as hard as I tried, just went downhill from there. I got really agitated in the end, even after I realizing that half the time it is one of those flag words that got on my nerve.

Looking back at the log, I realized that I thought he knew what I have been doing at ThoughtWorks for the last two and half years, and he probably thought I had no idea about the topic in this book. During the whole conversation, we were on different depth level of the topic. I guess this is one more case proving that you should watch out for "Barriers for Effective Listening" and ask "Why do you ask".

With all that behind, I was still stunned at the fact that no matter how well so many people try to protect a good idea, there are still people out there trying to profit from it by coming up with bogus stuff in its name. And one very good example is "certification". Because apparently they succeeded in making my friend think that is what it is all about, another certification.

So here is a list of things that you cannot get certified for. Instead, only your peers who work with you day in and day out can rate you, subjectively.

• Doing TDD even when you are under the pressure to delivery. Everyone can pass a test and do a little practice in their own pleasure.
• Have the courage to speak up when there are things that you think is wrong. "Do you think you should speak up when you see something wrong?" "Yes." "Good! You are certified!"
• Sit-Together. How can you certify that, just by sitting together for a week? It is one thing to say "yeah, it is a good idea", it is another thing to go to the extreme length of getting the tools yourself and start taking cubicles apart. (Yes I have met someone who really did it).
• Code Co-ownership. I am sure everyone can check that checkbox to get certified. But how can you certify a person's willingness to learn as much as the codebase, make sure that the design is as clear as it can be, and pass on the knowledge to others as soon as possible?

Sorry but "certification" is such a binary rating system, that it just does not fit into what it is in the real world.

Then of course, it is always good to get some education on the topic and have a proof that you have finished them successfully.

But that hardly gives anyone the authorization to say that "It is nice, but it is really really hard to do a business-driven situation".

Not when the purpose of the whole thing is "Deliver the business value in whatever the best possible way".

(BTW, did you realize that if you mistype blogspot.com as blogpsot.com, it is an actual website selling ads?)

BuildMaster Surgery - Moving to Cotta

I have been struggling with the File operations for a while, before I fully understand what is going on.

After I released Cotta project, the way File operations works in Ruby is like a hideous stain on top of a shining gem, always bugged me, especially when there are these nice classes like Pathname and StringIO already exist in Ruby.

Finally, my curiosity of making it better got the best of me, and I ported BuildMaster over to Cotta Style.

I have decided to call the files CottaFile and CottaDirectory so that I don't have to worried about the name space issue. Since BuildMaster does quite a bit of shell operations, I added method "shell" to the cotta instance so that the in memory version can log them for tests.

All things are happening behind the scene. The following is a code snippet demonstrate the new API:

   #system implementation is injected here
   cotta = BuildMaster::Cotta.new(BuildMaster::InMemorySystem.new)
   file2 = cotta.file('dir2/file.txt')
   file2.exists?.should_equal false
   # parent directories are created automatically
   @file.save('my content')
   @file.copy_to(file2)
   file2.exists?.should_equal true
   file2.load.should_equal 'my content'
   file2.read {|file| puts file.gets}

The API still need to be fine tuned but they provide all the operations that BuildMaster needs for the moment. Hopefully, someday something like this can be part of the ruby library.

BuildMaster Surgery - Moving to rSpec

This is the first part of the BuildMaster surgery that I have just finished (well, 95%) that have been keeping me up late every day. I will write the second part as a separate blog.

Moving BuildMaster to rSpec turned out to be not as smoothly as I hoped. Hopefully this blog will help anyone who is interested on similar task. This is what I have figured out so far so if there is a better way of doing it do let me know.

Syntax Change

If you have tons of test/unit scripts already, changing the code to use the rSpec syntax could be boring. They do have "Test::Unit Migration" support. However, I didn't get to take full advantage. It just generated "migration error" with no detail information for me. So I had to use some regular expressions to get the most of the change done, which I have blogged in "Going rSpec".

Specification Sharing

The "Context API" document on rSpec teaches you how to call helper methods in your specification code. However, it does not show you how you can make two context share the specifications, which is very useful if you want to make sure that two different implementation of the same interface shares the exact same behaviors.

Brain from Pivotal showed me that you can achieve this by using 'extend' (instead of include). So you can write the common behavior specifications in one file:

module CottaSpecifications

def register_cotta_file_specifications
  setup do
  # this one uses the @system that will be set up by each implementation spec setup
  @file = BuildMaster::CottaFile.new(@system, Pathname.new('dir/file.txt'))
end

specify 'file can be created with system and pathname' do
  @file.name.should_equal 'file.txt'
end
end

Then you can include those common behavior in your specification file for the implementation:

require 'cotta_file_specifications'
module BuildMaster
context 'Cotta file' do
  # extending the module so that you can call the methods
  extend CottaSpecifications
  setup do
  # setup the implementation for specifications
  @system = InMemorySystem.new
end

# this call registers all the spefications
register_cotta_file_specifications
end
end

Hopefully the above code makes sense to you. If not, let me know.


Running All Specifications

Again, thanks Brain to explain the reason and provided me with a solution.

Somehow, all the registered contexts are run at the end of each context registration. So my code for loading all the tests will now make the same context run again and again:

If you have context A and B in tc_a.rb and tc_b.rb and you require each file one by one. After "require 'tc_a'", the specifications in Context A will run. After "require 'tc_b'", the specifications in Context B will run, this time ALONG with all the specification in Context A. If you have C, D, etc., then specifications of Context A will run again and again at the end of each following require.

So Brain wrote his own specification runner, which will hold off the execution until all the contexts are loaded:

require 'rubygems'
require 'spec'
#require 'diff/lcs'
dir = File.dirname(__FILE__)
#require "#{dir}/../test/common_test_case"
require 'test/unit'
Test::Unit.run = true

args = ARGV.dup
unless args.include?("-f") || args.include?("--format")
  args << "--format"
  args << "specdoc"
end
#args << "--diff"
args << $0
$context_runner  = ::Spec::Runner::OptionParser.create_context_runner(args, false, STDERR, STDOUT)

def run_context_runner_if_necessary(system_exit, has_run)
  return if system_exit && !(system_exit.respond_to?(:success?) && system_exit.success?)
  return if has_run
  exit context_runner.run(true)
end

at_exit do
  has_run = !context_runner.instance_eval {@reporter}.instance_eval {@start_time}.nil?
  run_context_runner_if_necessary($!, has_run)
end

To use this, all I needed to do was to require this file at the beginning of my "ts_buildmaster.rb" file.

And it generates nice output too:

...
Directory object with cotta for in memory system
...
- dir should return sub directory
- dir should return a directory from a relative pathname
- should get file in current directory
- should create dir and its parent
- should delete dir and its children
- should do nothing if dir already exists
- should list dirs

Directory object with cotta for physical system
...
- dir should return sub directory
- dir should return a directory from a relative pathname
- should get file in current directory
- should create dir and its parent
- should delete dir and its children
- should do nothing if dir already exists
- should list dirs
...

Tricky Business of Being a ThoughtWorks Consultant

I have been thinking about this for quite a while and the topic came up again recently. So I figure I might as well write it down. Keep in mind that these are my personal opinion as ONE ThoughtWorker based on my situations.

Blog or not to blog

Historically, my blog thoughts come through several ways, all of which is under the condition that requires me to blog at airport or late at night (like now).

1. Feeling Pain

As the book "Project Retrospective" has pointed out, when you feel the pain of doing something, it is very important to stop for long enough period of time to reflect about what you are doing. As true as it is, which makes me a true believer of retrospective, I still have the same amount, if not more than usual, of billable work that I need to do. When I do come up with an idea of improving the process, I only create more work for me to talk with others to sell the idea, one more thing on my coaching list, and a task to implement the fix.

So at the same time that I have learned a new aspect on software development, I am also burned out more from my day's work, which makes it a lot less appealing to write a blog about it. Sometimes an idea has to go through several round to mature, so I hold off the blog. Then when it is mature, I have got used to it and don't feel the excitement and urge writing it down anymore.

Yes probably it would be ideal if I can slow down my other work so that I can keep my energy up everyday, which brings back to the title "consultant", as in, like it or not, I am being paid to live up to my previous promises as much as possible.

2. Learning something new

When I got to know about something new, I always find a way to apply it so that I can learn more. What that means is my afterhour will pretty much be occupied by those activities rather than blog about my incomplete thoughts about it. To name a few:

Behaviour-driven development: Liz from UK office did a presentation on jBehave in the China office. After that, I start visiting Behaviour-driven website to learn more about it. When we created Cotta project, we decided to give it a try and it turned out to be very well. But the IDE integration sucked very much and jBehave is a bit rough around the edges (like loading all behaviors), so we joined jBehave project, created Eclipse and IntelliJ plugin and fine-tuned the API, during which process the site building part of BuildMaster became mature.

Can you imagine me keep a log of all these activities while I am working on them in my afterhour? I didn't think so. Although now I realize that I have different take on BDD from Dan North and Dave Astels, maybe I should write something about it, well, after I finish this thing about build and release that I learned from last project, and my thoughts on how BuildMaster can rock the world.

3. Getting Into Interesting Internal Discussion

From time to time I met people, ThoughtWorkers or not, who are not afraid of expressing their opinion and discuss about it. A lot of the time the outcome of the discussion is that "We now know what we don't know" (in a good way, kind of like your feeling after finish reading "the World is Flat" from what I have been told). So is there a point of blogging that? Maybe, maybe not. How to blog it, what is a good balance to strike, I think this is the time when I feel that technical writers are being paid for a reason.

Discuss or Not to Discuss

A good project is a project that is under "Total Quality Control", i.e., every single detail of the process is constantly under review to see if there are any room for improvement. Since ThoughtWorks has got itself out of the staffing business, ThoughtWorkers are generally taking the lead on the projects.

Whenever a new ThoughtWorker joins the project, it is more than welcome, as part of ThoughtWorks culture, for him or her to question every detail of how this project is run. Sometimes debates will rise during the day so that everyone can understand not only the way it is but also the history behind it.

On the other hand, ThoughtWorks is brought in as "consulting firm", being paid to say what the right way is to run a process. So when the new member joins and debates start occurring, it can be potentially perceived by the client as the incompetence of the existing project lead, incompetence of the new member, mismatch of the personality between the parties, or weird management style of ThoughtWorks.

This happened to me several times, and luckily, my clients have been open enough to raise the question thus giving me a chance to explain. It happens a lot less now because the first impression that I am trying to make is that "there is generally no silver bullet". At the same time I am giving out suggestions, sometimes strong ones, I also tell the reason behind it and that those reasons are revalidated constantly.

Then that makes me look like the kind of consultants that I have had contact in the past, with who are all talks and never tells the client anything concrete, the exact kind that I loath and want to stay way from.

Hence the "Tricky Business".

There are a few more thoughts on this but they are not as well thought. Plus it is 12:30am now and I did plan on have a full night sleep for a change, well, after I finish ironing my shirt for tomorrow.

Going rSpec

After played with rSpec for a while, I have decided to convert BuildMaster on top of it. However, the test2spec script did not work for the project, so after a few experiment, the following regular expression helped me converting the majority of the syntax. I still had to convert the class definition to context definition manually, though.

1. Converting assert_equal(a, b) to b.should_equal a

replace "assert_equal\(([^,]+),\s*([^\n]+)\)" with "$2.should_equal $1"

2. Converting "test_..." methods to "specify ..."

replace "def test_([^\s]+)" to "specify '$1' do"

Speed of Unit Test

I have heard more than once from people saying that their unit tests are long because they have thousands, or tens of thousands of them.

Some would blame it on the complicated application architecture, or the language (like Java).

But if these guys can use a script language to run 132 tests, 219 assertions in just over half a second, maybe there is something to the idea of faster tests after all.

I am not saying it is easy, just that it is a goal worth pushing for.

One Week in SLC

The past four days just flew by and I am now sitting at the airport waiting for my flight back. I was hoping to fly my wife over so that we can visit the mountains around here during the weekend but it turned out to be much more expensive. That is OK, I still need to scout out the beach park where I am going to host the BBQ party next weekend.

I held on to a story card and paired with different developer each day. The domain that this story affects turned out to be the worst part of the application so each day I did a bit of clean-up and refactoring. The story dragged on a bit but I managed to make steady progress.

Since this is a .NET project, there are SQL server and IIS server to run on the laptop. After figuring out how to start and stop the service from the command, I wrote two Ruby driver class for them in the hotel room and checked them into BuildMaster.

So now I have a script that I can call to stop both service with:

services.rb stop

and all it does is:

services = [
BuildMaster::IisDriver.new, BuildMaster::SqlServerDriver.new
].each do |service|
service.send ARGV[0]
end

Back on the Road

After almost three months of local project, which I intend to log sometime soon, I am back on the road again.

This time I have my Verizon national access card that allows me get connected to do things like writing this blog at the Oakland airport. I could request the extra battery but with only 3-hour travel time I doubt the weight is worth the benefit.

Next project is an ASP.NET application and have quite a few ThoughtWorkers on it. I am afraid that is as much as I am allowed to say. I would like to whine about the painful download/install process I have been through in order to run the application, but I don't think anyone, like .NET or not, would care.

This project is on a 4-10 schedule which means once I get myself comfortable with the code and my role, I will be able to spend only 3 nights in the hotel and sleep at home 4 nights per week. Not that I am tired of traveling but there is something to "sleeping in your own bed".