Wednesday, December 14, 2005

The Job of Project Manager

I always believe a real good project manager is not one who always resolves crises, the same reason that a real good developer is not one who knows how to fix his own bugs. The reason is simple: it takes more skill to prevent problem than that to resolve problems.

I think a good project manager is one that make sure everything runs smoothly. For example, all the necessary preparations are done before each meeting and all the action items come out of a meeting are being followed in time and properly. Each of the team member knows what the project vision is, knows how to do his or her own job, and is motivated to do the job.

Sounds too vague huh? I agree. And the reason for this bolg is that I finally have something concrete.

For various reason, we are now having an agile development team with a project manager who is not experienced in an agile project. In China, it is very common to have a project manager with technical background, and it is very common for the project manager to help out the development work rather than managing the project.

In order to help the new PM managing this project (rather than checking out the source code and database schema), I asked for Renee, another ThoughtWorker who were the manager of this team to write down a list of responsibilities. It turns out to be very helpful for everybody. She also directed me to this on-line document (Agile Project Management), which is a good read as well.

The following is the list:

Here is a summary of the key responsibilities of a PM on an Agile project:
  • VISION - Establishing a guiding vision for the project and continuously reinforcing it through works and actions.
  • Ensure the team and client understand the vision.
  • Clear up any questions about the vision.
  • Includes timeline, release planning, scope.
  • TEAMWORK & COLLABORATION - Facilitate collaboration and teamwork through relationships and community.
  • Establish a good relationship with the team and client.
  • The team and client will come to you with issues if they trust you.
  • SIMPLE RULES - Establish and support the teamÂ’s set of guiding practices.
  • Examples include TDD, standups, pair programming, etc..
  • OPEN INFORMATION - Provide open access to information
  • Release plan
  • Story list
  • Progress, velocity
  • Staffing changes
  • LIGHT TOUCH - Apply just enough control to foster emergent order.
  • Demanding control will only cause the team to lose order.
  • Empower team members to do their jobs
  • AGILE VIGILANCE - Constantly monitor and adjust.
  • Hold retrospectives on team progress
An Agile PM needs to be full time on the project. A part-time PM cannot remove obstacles, encourage collaboration, or provide vision. Here are some addition day to day responsibilities:
  • Lead stand-ups
  • Track daily progress
  • Manage release plan
  • Communicate with customer leads
  • Supporting the team and helping the customer to understand project decisions, e.g., why we want to do J2EE without EJB
  • Listening to the developers and team, helping to remove obstacles
  • Risk identification and mitigation

Thursday, November 10, 2005

The Job of Architect

In the ThoughtWorks project that I have been working on, there are just one lead and several developers. Even though the lead is normally called "Architect" because that is the term that we use externally, he or she is much difference from what people normally think of Architects.

Rather than drawing all kinds of fancy diagrams and leave to the developers to implement the design (More than once I have heard of this theory that the developers don't have to be highly skilled as long as they do what is told).

In my previous projects, I have been very impressed with the work of our project lead. Highly technical and very experienced, the lead pairs with the team and working out the design decisions with the team. I felt that this is the best way for each member of the team to learn and grow and this is the best way to come to the right decision for the project.

On this project, being an architect turns out to be much more than just leading the development work. The current project involves three different servers posting their interfaces through WebServices and communicate to each other through these interfaces. WebMethods is the data exchange platform so that more services can be added, and there is a CA server for certificate verifications. Because of this, there are a lot of coordination required among the development groups. In addition, the requirement is not clear so I need to help the BA on the story list creation.

So between the development groups technical discussions and requirements gathering, I have not been too active on programming. The worst day was last Tuesday, where I had meeting back to back from the morning to the evening. I have been trying to play catch-up, looking through the code after work. But now I realize that with the code changes everyday, I will end up killing myself and still not able to keep myself up-to-date. So I am now delegating the design decisions to the team and pay more attention to their development practices like pair programming, test driven development, rotation, etc.

The good thing is that with the BA on track with the requirements gathering and story list creation, I get to concentrate more and more on the development.

Thursday, November 03, 2005

Two More about China Environment

So here are two more things that I thought worth mentioning.

All the three hotel rooms that I have been staying at have a central control panel by the bed. So to go to sleep you can just press one button, which will turn off all the lights and sets up the sign "Don't disturb" at the door. Pretty cool eh?

Although they don't have an alarm clock.

The connection in China to US is BAAAAAAAAAAD. I did a download speed test the other day and here are the data:
  • ThoughtWorks gforge 0.6kb
  • sourceforge 1.9kb
  • rubyforge 2.3kb
  • 2.7kb
  • codehaus 2.3kb
Doesn't that remind you the good-old days of dial-up? :)

BTW, the connection within China is pretty fast.

Sunday, October 30, 2005

Things to be Prepared for in China

First of all I think I need to say that I am still greatly enjoying working with great people in China and enjoying great food here.

Originally I put the title as "Things that I don't like in China". But later I thought about all these stuff I realized that they probably won't change in a short period of time. So the only solution is to watch out and try not to get bothered. (Granted, the city we are staying is not as a big city like Beijing or Shanghai).

Private Space

I know it always happens in a place where there are lots of people and limited spaces. However, from time to time, I still get surprised.

For example, now I have learned that when the elevator reaches your floor, you need to step right in from of the door and get ready to block the door so that you can push your way out easily. Otherwise, there will be people pushing their way in so quickly that you might have a hard time getting out.

Another case is that when talking to the front desk don't get offended if someone else just budge in and start asking question to the person who you are talking to at the moment. And don't get surprised if that person will switch attention to him or her right away and let you stand there waiting for the answer.

Limited Services

Traveling in United States, I have gotten used to stepping in a hotel and have iron board, steaming iron ready in the room. When I travel for over a week, I know I can always find a place to wash my clothes, most of the time inside the hotel that I am staying at. However, that is not the case here. Both hotels I stayed at in Shijiazhuang don't have iron and board. And I had to buy an iron because the second hotel has only the old style ones that burn shirts easily. Laundry has been a problem because I cannot find any place to wash them myself.

The other guys found two apartments to stay in. I was hoping to be able to do laundry at their places. However, the apartment services here is hardly any good. The washing machine in one apartment does not work, and the other one is missing a water pipe. What's more, there has been some repairing or construction going on and they have been out of hot water for over a week. Calling the owner or the property management has been in vain. The guys have been pretty cool about it by taking the shower at the shower room of the swimming pool downstairs. Still, I don't find it satisfying.

Task Management

In our working environment, there seems to exist a tendency of setting the task without setting the date. At the beginning the reply we here very often was "ok, let me discuss with them about this". In a project that has only 3 months and still working on requirements at the third iteration, this kind of got me nervous. However, all we needed to do was to ask for a date, either for the answer or a follow-up date about the date.

Thursday, October 13, 2005

One Week in China

This moment exactly one week ago, I checked in the Great Wall Sheraton Hotel in Beijing, and began my life as a consultant traveling oversea.

One week is not a long time, but it has already been a blast. The ThoughtWorkers in China get along very well (not that the ones in US didn't) and there is always constant conversation during the day as we cranking off the story cards. Because of this tight relationship (the other four guys actually live together in two apartments of the same complex), conversations are very open. For example, the CruiseControl server is in the same room, on a Windows machine, so the guys set it up so that every time there is a broken build, it will play a 20 second clip saying in Chinese something like
"What the !@#$% are you doing that for... Would you ever stop? I have already told you that I cannot handle it anymore, but you keep on doing it, doing it, doing it. You just don't care if I can stand it or not. If you do it one more time I probably will have to stab you..."
The original text can be found here and it is definitely much funnier to hear it in person. Because it is long and loud, it easily gets annoying, and with the good team relationship whoever broke the build definitely gets a lot peer pressure to fix the build.

There is also a bonus for me on this project. That is I get to get involved in a lot of project management stuff. With the client speaking Chinese, I am helping the manager whenever we need to talk to the client or whenever they need to talk to us about project plan and arrangement. The current crucial task is the story list. Because there is not a full-time BA, and that we are only developing one part of the system, and because that the time is limited, we are trying to finalize the story list, so that we can track the project progress with higher accuracy. So I am also going to play the role of BA for a little bit whenever there is a translation needed.

On Monday, we had a kick-off meeting. Because this is the first time that the customers were involved in iteration planning, and that this is the second time that they were presented with Agile Software Development, we decided to give them a second presentation of agile software development, focusing on the benefit, role and responsibility of the customers, and we were able to invite the deputy director. I believe the presentation was well received, because the deputy directory got excited and spoke at the end that we should receive the full support from their side. The full-time customers that have been assigned to us also turned out to be good natured and knowledgeable to the domain.

One more reason that I love this team is also because that the guys are very comfortable with me adding the Dvorak keymap to each laptop that I touched. We set Alt+Shift+8 for QWERT and Alt+Shift+9 for Dvorak and it works out pretty good. Someone even showed interest in learning Dvorak. Hehee... Maybe sometime we can finally say bye-bye to QWERT in the China office.

It is also interesting to learn the Chines meaning of the terminologies that I have been working with for the last 6 years. The following is a list that is on top of my mind:
  • Development: 开发
  • Iteration: 迭带
  • Document: 文档
  • Release: 发布
  • Deployment: 部署
  • Class: 类型
  • Method: 方法
  • Agile Development: 敏捷开发
  • Project Management: 项目管理
  • Test: 测试
  • Run Test: 跑测试
  • Constructor: 构造函数
And learning Wubi (五笔), a Chinese input method, is fun as well.

Friday, October 07, 2005

Arrived at ShiJiaZhuang

After a 12-hour flight, half hour Customs Service (The 'Paul and Jamie Buckman' behind me sure made it seem like 2 hours), one night hotel, 2-hour train ride with half hour delay, I finally arrived the city of the project, ShiJiaZhuang (石家庄).

I have finally met three of the four ThoughtWorkers in China office, Li Mo, Li Chaoqun and Li Xiao. (Yes, Li is a common name in China), and I got a chance to talk with Li Mo on the train. Sid's comment of ThoughtWorkers was "young and super smart" (sounds like the early days of ThoughtWorkers?) and talking with them sure confirmed it.

What I have gathered so far has already shown that projects in China is definitely different from the states, unfortunately I cannot reveal much. As much as I want to keep this blog on my experience in ThoughtWorks, the special condition in China might force me to log lots of them on an internal server. I guess we shall see.

Saturday, September 24, 2005

Walking off, and Going to China Project

This is one of the times that I am proud to be a ThoughtWorker.

We have been working with the current client for over a half year (I joined three months ago). The agreement has always been that this is a transformation (formerly known as enablement) project, that, given the clients' background of waterfall approach and ThoughtWorks' experience in agile development, should rely highly on ThoughtWorks' input. That is, for a start, during the scoping phase form a small group (6) and have one on one ratio of ThoughtWorker and the employee from the client.

But apparently someone thinks that if they pay us they own us. Before the pilot project even starts (hence gets the requirement), there is already order from high up to use one of those so called "J2EE framework server". And during the initial phase, there are going to be 9 people instead, with only 3 from ThoughtWorks. All these decisions are made without even consult ThoughtWoks (you would think that should be the reason to hire a "consultant"). And no reason has been given for these decision.

So as the current finishes this month. The manager decided not to renew it and walk off from a project that is not built on a solid ground of trust and understanding, even thought the rate on this project is fair and we might have a good account. This decision got the support from the headquarter and the four of us are already on the list of available consultants for October.

As for me, I am really glad of the decision being made, because that shows that we are serious about being good at what we do, and the we care about have happy consultants on the project.

Another good reason for me to be happy is that the moment my name come up, there is already a good project for me to go -- China. Because of this and last transformation project that I worked on, and the fact that I speak Madrine, I turn out to be a good fit for this project that needs someone to come in and help out. Time is short so I am busy working on the visa and travel plans.

At this moment, my goal of China trip is the following, I am still updating it:
  • Get to know the ThoughtWorkers in China: I have heard of great things about the people that we hire there. I need to spend as much time as I can to get to know them and exchange our experiences to learn from each other.
  • Get to know the market in China: Despite the great potential in Xi'an, I still think there should be a lot of opportunities in Beijing and Shanghai. Agile software development in China is getting a lot of attention, if we don't grasp the market and build our good names, there will be someone else very soon.
  • Get to know the IT environment: There might be a lot of things that I am taking for granted here that is not as available. Programming in a non-English speaking environment might bring different dynamics.

Wednesday, September 21, 2005

Deployment Script with Watir (Take THAT, Websphere!)

So, after some struggling with Websphere on deploying an EAR file (Don't ask me question about why we still bother, please... ) and not able to get a hand on a simple deployment script (I think there should be one but I just didn't have the time to look further), I have decided to automate the web process that I have been doing, using WATIR.

Call me innocent, but I really thought that depolying an EAR file on an application server would be easy, given the 'comprehensive' file structure and all the deployment descriptors.

But no..., you have to click through this long 'wizard' (ooohhh...) process every single time. Not only you have to do it manually, it also takes quite a lot of time. And I happen to be one of the impatient guys who thinks that it is a machine's job, then a human should not suffer.

In case you haven't heard of it, Watir ( is a ruby library used for web testing. I am still not sure how exactly it works, but the result is that you can control an internet explorer instance using Ruby script. So basically almost anything you do with IE, you can write a script and automate it.

So long story short, I was finally able to write a script that uploads the file to the server, click through the steps, and save the configurations. Needless to say, I am very impressed the progess that Watir has made, functionality as well as the documentations.

The script is actually not that long. Here it is (with comments):

require 'watir'

class IeLauncher
result = ie = Watir::IE.start(*args)
if block_given?
result = yield ie
return result

# Ignore the above, they are just basic set up.

# Launch IE, go to the specified URL'http://localhost:9060/ibm/console/') do |ie|
# Log in, my local server doesn't have a password
ie.text_field(:name, 'username').set('admin')
ie.button(:value, 'Log in').click

# Only one login is allowed so you will have to log out last one
if ie.contains_text('Login Conflict'), 'action', 'force').set
ie.button(:value, 'OK').click

# Apprantly when you deploy an application, you need to 'save' it.
# This recovers it
if ie.contains_text('Recover prior changes'), 'action', 'recover').set
ie.button(:value, /OK/).click

# This get the frame that has the navigation tree and look for the right node
navigation = ie.frame('navigation_tree'), 'Applications').click, 'Enterprise Applications').click
detail_frame = ie.frame('detail')

# Uninstall the existing deployment
if detail_frame.contains_text('DukesBank')
dukesbank_checkbox = detail_frame.checkbox(:value, /DukesBank/)
detail_frame.button(:value, 'Uninstall').click
detail_frame.button(:value, 'OK').click

# Install
detail_frame.button(:value, 'Install').click, 'radioButton', 'local').click

# Upload the file (Note, file path cannot have space)
detail_frame.file_field(:name, 'localFilepath').set('c:\Work\Dukes\DukesBankEAR\build\artifacts\DukesBankEAR.ear')

# Clicking through the wizards
button = detail_frame.button(:value, 'Next')
while button.exists?
button = detail_frame.button(:value, 'Next')
detail_frame.button(:value, 'Finish').click, 'Save to Master Configuration').click
detail_frame.button(:value, 'Save').click, 'Enterprise Applications').click

Friday, September 09, 2005

Feedback from the Client

I have meant to post this a while ago, but with the traveling my daily schedule is still not settled. So I never found a good time that I can just quiet down and write some blog.

This week is a short week due to the past long weekend. So we were able to convince the client that it is better for us to work from our home office so that we don't need to spend two days traveling for a four-day week. And once again, I am able to continue my routine of blogging.

After my last enablement project (now the term is "transforming" because "enablement" has been used along with "training". I will explain that later), the client team had a project retrospective, discussing what they have learned during the project.

With their still fresh memory, they were able to extrapolate the following, with which they gave the presentation to the rest of the organization. They are nice enough to let me publish this. I hope this can be useful for some of you who are still wondering about agile software engineering, or at least some of the "weird practices" that I have been defending from time to time. This is the first-handed note. Hopefully I can help them revise the format for more formal presentations.

Agile: One Developer's Perspective

What methodology did we use in the past?
  • It doesn't have a name, as far as I know, but here are some features of it
  • big upfront requirements-gathering, big upfront design, with some formal and some informal documentation
  • one developer per application, or a couple of developers working in different tiers of an application
  • integration & testing in a big bang at the end
  • manual testing

Some problems we had with this approach

Big upfront...
  • late-breaking requirements changes combined with inflexible deadlines led to periodic frenzies of coding that eroded the integrity of the big upfront design
  • sometimes we specified & designed more than we could actually implement with the time and resources we had
  • integration late in the development cycle led to periodic frenzies of coding that eroded the integrity of the big upfront design
  • disagreements about how open we should be to changes

One developer, or a couple of developers working in different tiers of an application...
  • we didn't always take advantage of possibilities for using some common code or common approaches across applications
  • difficulty scaling up for larger efforts
  • the culture of code ownership by individuals was sometimes at odds with our "esprit de corps"
Integration & testing in a big bang at the end; manual testing...
  • huge burden placed upon QA (when we had QA at all)
  • difficulties detecting the ramifications of maintenance changes, so we became timid about upgrading applications and we ended up with multiple versions of the same module in production
  • we often had as many opinions about how close we were to being finished as we had team members
  • we did not know how to thoroughly test componentry; we could only test built apps
For me personally, the biggest problem was that I was unhappy. I felt that our team could be great, but we just couldn't get there, and I didn't know how to be "part of the solution".

Enter Thoughtworks, with a philosophy and a toolbox

Some things I was immediately enthusiastic about...
  • Eclipse IDE (Great no matter what methodology you embrace)
  • Automated testing
  • Daily standup meeting
  • Frequent discussions as a group about what is working and what isn't, so that we can make incremental corrections in our direction
  • Realistic attitude towards estimating: as the project progresses, we can get more accurate in our predictions of a delivery date, but we can't expect perfect accuracy at the beginning.
  • steady pace, avoiding individual heroics
  • Simplicity--the art of maximizing the amount of work not done--is essential. (From "Principles behind the agile manifesto"
Some things I was skeptical but open-minded about...
  • TDD
  • Test-driven development requires that a unit test be written for each tiny piece of functionality before you write any application code
  • The unit tests are written by the developers, not QA
  • Although "test" appears in its name, test-driven development is not so much about testing as it is about enforcing a development discipline that makes refactoring possible
  • The unit tests serve as documentation of the developer's understanding of requirements
  • "radical co-location"
  • continuous integration
  • fully automated build and test process that allows a team to build and test their software many times a day
  • everybody using the same code base, checking in and updating their code very frequently
  • at first, I was incredulous that checking in and building as frequently as once per day would even be possible; TWers viewed once a day as a minimum
Some things I was deeply skeptical about...
  • paired programming
  • code ownership is to be avoided
  • evolutionary design through refactoring

Martin Fowler's definition

What is Refactoring?
Refactoring is a disciplined technique for restructuring an existing body of code, altering its internal structure without changing its external behavior. Its heart is a series of small behavior preserving transformations. Each transformation (called a 'refactoring') does little, but a sequence of transformations can produce a significant restructuring. Since each refactoring is small, it's less likely to go wrong. The system is also kept fully working after each small refactoring, reducing the chances that a system can get seriously broken during the restructuring.
  • very short iterations
  • less emphasis on formal documentation
  • "embrace change"

A year later
Life on our team is completely different than before. From my perspective, it is much better.
Did the philosophy and the toolbox make a difference?
Yes, but not all by themselves. We had to have...
  • the right attitude
  • one-on-one mentoring from experienced agile developers
  • effective leadership
  • discipline

We could not have made changes as extensive as this without
  • positive attitudes
  • team spirit
  • perseverance

one-on-one mentoring from experienced agile developers
  • from the agile manifesto: "value individuals and interactions over processes and tools"
  • TDD feels initially so unnatural, that it is essential to have a patient, experienced person at your elbow
  • without discipline, the agile philosophy can degenerate into "a license to hack", so it is important to have constant reminders about the discipline until it is second nature

effective leadership
  • the project manager may be an exception to the general principle that individual heroics should not be required
  • protected the group from distractions
  • keeps both the big picture and myriad details in mind
  • inspires confidence

  • do not write even the smallest method without writing a test first; see the test fail, then write the minimum amount of code to make it pass
  • update from the repository frequently, run the entire test suite before each check-in of your own
  • builds & test suites run continuously; if your check-in breaks the build, STOP and fix
  • to maintain design and code integrity, coding decisions are made in pairs...if the pair can agree, go forward
If the discipline is not observed, then we can't "embrace change" and "evolve the design through refactoring" because we can't be confident about extensive changes.


Below is a summary of BTM/70, a methodology in use at a job I had 20 years ago at a company. It is probably an example of what the "value individuals and interactions over processes and tools" principle is a reaction against. All that I remember about BTM/70 is that there were big binders of sample forms, and every step of the way, one had to photocopy the appropriate forms and fill them out. I do not recall any specific benefits.
The life cycle of an BTM/70 project consists of eight (8) phases ranging from Systems Requirements to System Implementation. Coupled with BTM/70 is ET/70, a Project Control tool set that is used for project planning and tracking. ET/70 assists in milestone planning, helps establish project target dates, supports activity tracking, and detail time reporting of each staff member down to the task level, within each activity, of each phase of the project

Monday, July 11, 2005

Start Traveling

Starting traveling last week. I realize that since Cysive I have forgotten how tiring it can get sometimes. Fortunately, I have got a pile of book that I have been wanting to read but never "find" the time between work, hiking and gaming. Since I can do none of the above on the 8-9 road trip twice a week, a book surely comes in handy.

BTW, Robin told me about this interesting Dvorak post on slashdot (

Monday, June 06, 2005

Dvorak Follow-Up - Why I Think It Is Great

Two weeks into trying out Dvorak hardly makes me into an expert on the keyboard layout, and chances are that my opinion about it will change as the time go by. So I was trying to focus more on how I managed to learn it rather than any conclusion that I have drawn.

However, I realized that since I already made the comment that "It is great", I'd better explain it in the whole aspect, even though it is just a snapshot of what I am thinking and it might very well change.

I think it is great because I am amazed at how easy it was to learn. With vows at the left hand home row, it somehow makes it very easy to remember. Most of the common key combinations stay on the home row, which makes very easy to type. Here is a good example (he calls it Dvorak poetry) that Alex sent me:
Shane Duan nodded to his sis as he situated his sedan in the shade

If you want to know what it feels like, here is the key this sentence mapped to on a QWERTY input:
:jald Hfal lshhdh ks jg; ;g; a; jd ;gkfakdh jg; ;dhal gl kjd ;jahd

I think it is great because in Dvorak, you can type the whole sentence without your hands leaving the home row. And I believe that is the case with Dvorak that you stay on the home row most of the time. I think that is the main reason why it is so easy to remember. You just need to learn the home row and you can pick up the rest as you go.

No I am not ready to commit my life to Dvorak. The website that I have referred to ( has very though references on both side of the story, especially here ( And this link ( referred form this link in the comment, is making a very compelling case. However, just like the author said: you don't have to believe either side of the argument. I am hoping through a first hand experience, I can make a convincing case either way.

Of course, the speed will always be the hot topic here. However, from what I have concluded so far, I am afraid there is no easy answer. As a fast typer, I felt that the speed comes from not only the ability to map a letter to a key without thinking, but also that to map the key combinations. What this means is that probably there is no much difference between the inputs as far as a fast typist concern because he or she will simply just remember the same key combos. If that is true, I might be the wrong person to try Dvorak, since I type 8 hours a day for living.

With that being said, I think that a casual typer probably would benefit more from Dvorak input. However, since all the keyboards you can see during everyday life are labeled with QWERTY, one tip to learn is not to look at the keyboard. Ironically, if you can do that, you are probably already a good typer.

I also have to be honest with you. I am a geek, and I love trying different stuff. Just the fact that I am able to say "I tried it for a year and I think..." rather than "I read it on this website one day that it seems..." is already something worth doing for me.

The last thing is really rather subjective. I never had any good feeling for QWERTY typing. I simply do it because it was the only choice. However, I somehow enjoyed Dvorak. It could be the combination of the fact of typing mostly on home row (With kjd as the keys for "the" just makes more sense) and the fun of trying something new.

Sunday, June 05, 2005

Two Weeks Into Dvorak

It has been two weeks since I have switched to Dvorak and I really think it is something worth writing about.

The topic of Dvorak just came out of a lunch conversation. I was talking about the fact how sometimes we keep doing what used to useful without realizing that the circumstances have changed. A small example was how Java IDEs forget that maybe the traditional windows mapping doesnot apply anymore (who opens a file through file chooser, or print the code anymore). And IntelliJ's found of F keys making its useful features much less accessible, and that by some miracle Eclipse made the right choice of having all ctrl+alt+key reserved to refactoring related actions.

Then the topic switched to even the key board layout tha comes with every computer was not designed to make your typing fast and easy and how I heard of another kind of keyboard that is supposed to do just that. As it turned out, Adam knows exactly what it is because he has tried it before.

With Adam's reference, I was able to enable my laptop (window XP) with Dvorak keymap and find the related sites. After reading through it, I figure I'll give it a try.

I am a fast typer. I got my hands on a type writer and a typing training book when I was a kid. So I literally learned typing before I even knew all the words that I was typing. Then in graduate school for a year I got addicted to IRC and another year on text-based MUD game, both of which require intense typing in that the faster you type the more fun you can get out of it.

So you should believe when I tell you this: Dvorak is great!

I started with a online tutorial ( I do about three sessions per day, 15 - 30 minutes per session (when I am getting tired or bored and staring mixing keys). At work I still have to use the QWERTY because it would be too slow otherwise.

After a week, I felt that I can finish the lessons of the home row without problem, but my training on the other rows is not making good progress. I felt that it is because my old habbit and this half Dvorak is not working togethir. Since one major advantage of Dvorak is supposed to be the fact that your fingers stay on home row most of the time, I have decided to swithch to it full time, with my Fitnesse blog as the first target.

Well, that weekend was not too easy. It is as if I am learing to walk again. For a while I was even wondering if it is really worth the effort. Two-day is hardly a long enough period to give up, so I just lowered my target on how much I can write in a hour so that I won't be too frustrated. I also noticed that most of the time I am having trouble because I am too used to the old way that for key combination like "ing", "tion", or even "th" my figer will go without even my brain had a chance to process them in Dvorak context. So I concentrated on several key binding that is giving my trouble.

Now after two weeks, I would recommend anyone who wants to learn typing to try it out, AFTER you have read the pro's and con's on the websit and make your decision.

Tuesday, May 31, 2005

Memorial Weekend - A Sample of Quality of Life with XP

First of all, the criteria of the quality of life are rather subjective. If you do enjoy working on your project at home, then all I can say is that you are lucky. For me, there are so many other things besides work and software engineering that I am just too interested in to give up.

One good thing that comes with XP is that you have the control of your time. Because you always know how far you are to finish the project, your project management is fairly predictable. As a result, you can deliver the project on time, and your personal life is not affected.

So, even though my work is still busy as hell, I was still able to put it behind and enjoy my long weekend. For some person it might be really obvious, but my old buddies at JBuilder team would only know too well how obsessed I can get.

Without further a due, here it is:

Stinson Beach Hiking
This is by far the best hiking place that I have been to around bay area. The reason is simple: This is the only place that you can hike on the beach, on the grass field, and the woods in one afternoon.

Brentwood Cherry Picking
Eat all that you can pick, and buy home the ones that you cannot. Just make sure that you need to pace yourself otherwise your stomache might not be able to take it.

Ninja Gaiden
I don't understand why they want to associate this game with Dead or Alive 3, because it is totally different kind of game. Besides the normal similar quest that is quite similar to Final Fintasy X, it requires the player to analyze the enemy's attacking patterns and come up with counter measures. The last big boss was quite disappointing, though, because I just chased him around like a mad man and that killed him.

Texas Hoedown
Yes I also do not believe that there is luck in poker. But, I do believe they should put the number on those chips so that I don't have to remember them.

Monday, May 23, 2005

Joy and Pain with Fitnesse

(This blog is entered using Dvorak input, which I am going to write a blog about after I either master it or give it up)

Last Wednesday (May 18th.) I gave a presentation to the BayXP ( titled "Joy and Pain with Fitness". I am putting the presentation material here, and try to explain the examples.

Joy and Pain with Fitnesse

Presentation of what we have learned during our usage with Fitnesse, an acceptance test framework, and how we have benefited from it. This presentation is intended on purpose to avoid raising philosophical issues, software engineering methodology, or what the correct usage of the Fitnesse should be. Rather, it concentrates on the retrospective of our experience with Fitnesse. It is more important to understane the reason behind

Fitness' own description can be found at its website "OneMinuteDescripiton", with the following in particular in our project:
  1. Fitnesse is an acceptance testing framework. Its targeting audience is not only the developer but also the user who is going to write and run acceptance tests, like domain experts, QA, manager, etc. (And it has done a great job to achieve this goal).
  2. Fitnesse is written in Java, which makes it easy for developer to assist the acceptance test writing.
What We Have Achieved
Just like an application product, we slowly developed a set of Fitnesse fixtures that allow everyone on the team, including the QAs and BAs, to set up the system state, simulate the events processing, and assert the end state of the system by writing XML document assertions and database assertions.

In order to document how to use the fixtures, we also created a "Fixture Tutorial" page, which contains all the fixtures that are available, with detailed descriptions. Each page is a test by itself, which means that people can change them and run them. This is very similar to the ones on Fitnesse page, e.g. JdbcSample, except ours are related directly to our domain.

How Fitnesse Works
The following is the outline of what we have learned in our project. It is quite possible that there are useful features that we have not discovered so you should make sure to check out Fitnesse website.
  1. Fixtures are loaded through reflection by very simple naming convention. For example, if the header of the table is "Insert Person Data", then the class "InsertPersonData" will be loaded and instantiated. The packages in which Fitnesse should look for the classes can be specified through "Import" fixtures.
  2. A page called "SuiteSetup" will run at the beginning of the suite and will not run for a single test. A page called "SetUp" will run at the beginning of each test all the time. The same rule applies to "SuiteTearDown" and "TearDown".
  3. The following are the four most useful features to us in this project:
    1. Column Fixture: The column name without the question mark is a public field in the fixture that Fitnesse will set value to. The column name with the question mark is a public method in the fixture that Fitnesse will call and compare to the value in the table.
    2. Row Fixture: Great for doing assertion on a set of data. For example, if you want to assert that the person has three specific phone numbers (555-2222, 555-4444, 555-9999), you just need write the Fitnesse page with these numbers and implement a row fixture that simply returns all the phone numbers this person has. Fitnesse will do a fantastic job of comparing these two sets and show you the difference.
    3. Action Fixture: The first column of each row is mapped to a public method. This is especially useful to run a sequence of actions and check the result between the actions. This fixture is being utilized by jWebFit.
    4. If you use standard fixtures, all data type conversions are handled seamlessly. As explained in "DataTypesInFixtures".
Why We Love Fitnesse
We believe that tests are good and that we as XP and Agile developers should do everything we can to encourage developers to write unittests and QA/BA to write acceptance tests. In this project, we would never have achieved this level of tests without Fitnesse. To quote from our QA:
It is great to have a tool that allows me writing tests for our system. I have been quite uncomfortable with matching logic. With these many tests and assertions (over 23,000) I am feeling a lot better now.
Because it is easy and fun to write test, many things just follow naturally. Stories are written with test in mind, making them easy to verify. Acceptance tests are also written early, sometimes even ahead of the iteration. Consider this is the team that didn't know what a story card is and didn't have any automated tests, we are really satisfied with the result.

For BA and QA, or any non-technical member of the team, Fitnesse is great because it has very clean UI with neat user interaction design. It is much less itimidating than a programming language. You just feel that you are writing a Wiki page with tables that are runnable, instead of a program with limited way of writing comments. The pages are much more readable than the JUnit tests.

For developers, there are already fixtures helping them writing unit tests. With Fitnesse written in Java and loading fixtures through simple reflections, it is quite easy to hook up the Fitnesse with the existing testing fixtures, thus greatly reduces the code duplications athe cost of the Fitnesse fixtures.

Why We Hate Fitnesse
I believe it is as important to explain the struggle we had with Fitnesse as well as the success, because this way the reader can learn to use this tool effectively. It is also possible that in the limited amount of time and with project development to bring business value as our ultimate goal, we might not have learned the way that Fitnesse was originally designed for. And I'd very much like to be directed to the right direction.

No matter how many times that I say it is very easy to write Fitnesse, you still have to spend time writing it. Also because it is not the developers who write and run the Fitnesse, you get the same issues that you have for any project. What this means is that developers are taking requirement request. It is only natural that you want to help, and the better a developer you are, the more likely you are going to put down your story and write fixture instead. Before you know it, your velocity is down and you are only overwhelmed with more and more fixture requests. There is another member of BayXP who encountered the similar problem.

The target audience of Fitnesse are the none-technical member of the team so it is doing its best job to achieve that. This achievement, however, comes with the cost of the usability for developers. A developer who has been infected by TDD is a developer who has been spoiled by JUnit. With Fitnesse we have to again and again wait patiently for the suite to finish. While waiting, we have no idea how many have been run, how many there are in total, how long it is going to take. When there is a failure, we have to wait for all the tests to finish before we can see the report. (You can copy the URL and open it in another browser window but then you have to worry about running two acceptance test processes at the same time on the same machine, which can give you some trouble sometimes.) Even with the report, you still cannot open that line of code with one click. And it is a lot harder to put breakpoint at the place that you want in debug a problem.

In the spirit of Continuous Integration, we run some of our Fitnesse tests through CruiseControl. The best way we were able to figure out is to launch the server, then use the Fitnesse runner connecting to it. With the way Fitnesse is implemented, we ended up with a server thread serving the HTML, a client thread reading it, and another thread monitor the communication between these two. Somewhere it is not managed correctly and it is causing the process to hang. We spent a lot of time trying to figure out the reason but this problem only occurs on one of the Solaris machine (which happened to be our release machine), occurs 10% of the chance on our Linux machine (which is our build machine) and never on the windows machine. The closest thing that we found online is this post about how an earlier similar bug was fixed. We moved to an Linux machine for the release and bounce build machine every now and then, and moved on.

Fitnesse also tries too hard, in my humble opinion, to be everything. Rather than a simple webapp, it is a stand alone web server. This means that you have another server to manage, another account to manage and another port to open. It also has its own Wiki engine so it is another set of rule to remember (after played with 5 different rules among Wiki, Twiki, Confluence, Rubywiki, one really gets tired.) Fitnesse dose not extend JUnit framework. The whole suite is actually run as one test, which means memory will increase as the test is running because of the HTML page.

As we work with Fitnesse, we have developed the following patterns that helped us to use it successfully.

Just like any product feature request, every Fitnesse fixture request needs to be considered and discussed. We include them in our iteration planning meeting, we estimate them, and we pair up to implement them.

As we develop the fixtures, we write the tutorial pages first, which serves as our tests for the fixtures themselves.

Fitnesse pages are saved in their original format in TXT files. We launch the server with the correct arguments so that it does not manage the history of the pages. Then we check the files into our version control system.

For each project, we use one singleton class to manage the data that we want the fixtures to pass to each other, and make sure that all fixtures are stateless. In this way, the fixtures can stay simple and we were able to optimize the tests to make all the test run under 5 minutes.

For each project, we designated one suite as our acceptance tests area. Once a tests pass, we move it in there so that it never fails. In this way, the QA can feel free to write any tests they want and still can check in.

Always remember, it is nice and cool to do everything, but it is worth the effort only when the benefit outweighs the cost. We didn't have Fitnesse tests for a project that involve Peoplesoft because it uses views that are oracle specific. We resort to jWebUnit tests for another webapp because it was not as easy to use jWebFit (However, with the helper classes that we developed later on for web tests, we are taking another look at it to see how much it would cost now. The QAs is getting a hang of automated testing and starts feeling bad about not able to write them for Web apps).


Friday, May 13, 2005


Adewale Oshineye sent me an email because there was a link from my profile page on ThoughtWorks linking to this blog, to ask me if I want to add mine to the ThoughtBlog. The reason that I have not done it is because I want to start publishing this blog only when I really know what kind of blog I am going to keep here, or even if I think it is worth it to keep a blog about my experience in ThoughtWorks.

But I figure, help is already here it really does not make sense to turn it down so I replied yes. Then I figure it is time to subscribe to the blog feed to see what my fellow ThoughtWorkers are writing about, especially with ThunderBird it is so easy.

So already I have learned something new today! I thought I have known all that is to know about writing assertions (assertEquals, assertNotEquals, assertTrue, assertFalse, simple, eh?). Boy was I wrong. Joe's blog "Flexible JUnit assertions with assertThat()" sure opened up my eye.

When I was working for JBuilder, I wrote a test framework for unit-testing some part of JBuilder modules. When a test fail, it is always desirable to see as much information as possible in the error message regarding information like project configuration, run configuration, etc. However, JUnit requires you to provide a message before the assertion happens, which means that all the tests will need to spend a lot of time constructing the messages, even though they might not be needed. So I wrote this LazyAssert that only constructs the message if the assertion fails, and ended up duplicating a lot of Assert APIs. I was quite bugged by that result but concluded that there was nothing I can do.

And now I know the right way.

My hats off to Joe for sharing it with us. And again to my fellow ThoughtWorkers.

Update of an example:

Check this out:

...(Click for full stack trace)...
at org.jmock.core.VerifyingTestCase.runBare(Unknown Source)

Expected: timestamp is after 2005-05-16 15:21:34.871
but got : Mon May 16 15:21:34 PDT 2005

...(Click for full stack trace)...
at org.jmock.core.VerifyingTestCase.runBare(Unknown Source)

Sunday, May 01, 2005

Fellow ThoughtWorkers

One thing that I found different working at ThoughtWorks, are the fellow ThoughtWorkrs. Here are three examples:

I am the kind of person that cannot remember people's name unless I get to know them personally. So even though I greatly enjoyed the book Patterns of Enterprise Application Architecture, I only know the name of the main author Martin Fowler. Well, after working at Yabe project, I picked it up one day again to look up a pattern. Guess how surprised for me to find out that of the people that I worked with, three are on the cover (David Rice, Edward Hieatt, Robert Mee). Even though only Dave works for ThoughtWorks, I think this still shows that this company is well connected.

ThoughtWorkers are all opinionated and love a good argument. Someone told me this the first week that I joined, and I have seen it again and again. I think this is good because it will bring out the best from all sides so that we can always think of the way we look at things or do things. Of course, I can understand that it is not for everybody, even though I enjoyed every argument.

Here is a good example. During a discussion about the practical use of loosely coupled architecture like web services, EJB, or CORBA, Dan North made the following comment that cracked me up:
You mean you don't use CORBA?? Are you mad??

How on earth do you instantiate an object without first looking up a Factory Factory using IIOP to register as a Listener for a Factory to call a Factory Method via reflection, if you don't have CORBA?

Bloody amateurs.

Wednesday, April 20, 2005

What is Wrong with This Code

Managing and assisting a project development is more and more interesting to me nowadays. However, from time to time, I will run into some code that I think worth documenting.

What is wrong with this code in a test case:

public void testShowAddressInvalidCountryCode() {
DomainError e= ErrorMsg.AddressInvalidCountryCode;
String msg = ErrorMsg.showErrorMsg(e,new DomainErrorValue(ErrorMsg.PERMANENT_ADDRESS));
assertTrue(msg!="No error msg");
If you don't see why, let me explain:
  • Don't use negative assertions: Asserting two things not equal is most error-prone. If you assert that a return value NOT equals 0, for example, chances are that this test will still pass, even if a bug is to be introduced in the method making it return 6 instead of 5.
  • Don't use instance comparison: If the method decide to create a new copy for return object, then it is impossible to test. For the example here, if the method innocently construct the message "No error msg" instead of returning the constant, the instance check will render useless.
  • Don't do System.out during test: This will pollute the output in the console and make it useless. If it is anything need to be verified, verify it with assertions. ASH project went a little to the extreme but it has very sound reason.
  • Do test-driven development: Code like this is most likely a symptom of lacking test-driven development.

Friday, April 01, 2005

Happy Apile Fool's Day

On this day, there are two things that I want to mention. Normally I stick this blog to my working experience, but I figure everything has an exception...

GMail bumped up everyone's storage! This is one company that keeps reminding me that you can make things better by making things right. They keep on surprising me in a good way and I just love this company more and more. As a matter of fact, I would have worked for them, if their HR were not so slow on setting up interviews (that was an interesting experience last year, switching my Google HR contact three times in two months), that I ended up running into ThoughtWorks, a company that I should have been working for since year 2000. Even though my job in JBuilder was a great experience, I am sure I can do much better in a company like ThoughtWorks.

ThoughtWorks name was mentioned in a April Fool's webpage (PaiOn Chair). I asked around and didn't hear anyone knows the folks in Cenqua. So if that is the fact, this is a good sign that we are being regarded as THE XP shop, a name that everyone is building for.

Friday, March 25, 2005

So many things to do, so little time

My blog is really falling behind, yet I have little time to do catch up. There are so many things on my plate right now that I think I should just pick one, focus on it. I am seriouse. When I cannot even find time to play "Enemy Territory" in a week, I think I am trying to juggling too many balls.

Work at the client is busy as well. We have extended our work here until the end of May, which I think is very nice. It was sort of a hard deadline that we will only be here until the end of March, and we have been working hard on the transition. With our work extended here, we can finally see the product gets release and deployed rather than having to just put faith in ourselves about the successfull of our enablement.

As part of the XPE project release, and following the book Pragmatic Project Automation, I am starting building a release script in Ruby. (Ruby is a language that I have grown more and more fond of, BTW). I figure I might as well make it generic enough so that I can use it for other projects. So I have created a rubybuilder (or Build Master) on ruby forge ( Need to do some inital check-in's of the source code and the website setup.

Another project that I need to work on is the jbuilder-opentool project. That is because I have found Eclipse more and more annoying with each and every day. I don't care what people say, this is one product that just never failed to surprise me in a bad way (instead of like IntelliJ, which always surprises you in a good way. I am too familiar with JBuilder to be surprised either way). If I can just create an opentool that allows me read the Eclipse workspace and project, I will be able to do something useful things with JBuilder, starting with profiling.

Talking about profiling, another item in my list was the performance of the webapp that we were building. Somehow it just hangs whenever trying to load from database. Finally with the DBA's help, I find out that if you use "setLong" instead of "setBigDecimal" in prepared statement for a numeric column that is used for search, the index will not be used, rather, the query will simply scan the table. Converting a long to a BigDecimal is a bit hassel, so I settled for changing the query to convert the parameter using "convert" function in sybase.

On the last note, I just noticed (Thank you WebPastie, great job Bill!), that if you search on google with "Shane Duan", the first page is all mine, different stuff I submitted dating all the way back to my start-up days. I supposed an English name "Shane" with a Chinese "Duan" is an unusual combination for a name.

Anyhow, this post barely scratch the surface of what I have been doing presently, more coming up. Not this weekend though, because the snow in Tahoe is too good to miss!!!

Saturday, March 05, 2005

XP Practices in Use

So we have an opensource page built. I don't have something that I am very proud of ( yet, but this is still a good start. Adam, Paul and I have some good idea from our current project about writing some library to help writing testing about database related library. We still need to sort out the detail but the idea is to follow the pattern that we have so that we can run acceptance test locally against HSQLDB, and have cruisecontrol run them against the real database.

It seems that it helps by writing down my thoughts and experiences as I encounter different issues. So even though I originally meant this blog to be what I have seen and heard among fellow ThoughtWorkers, I am going to writing down what I see and hear in the projects that I work in as well.

Ping Pong Pattern in Pair Programming
Adam mentioned it before and we tried, my experience was not exactly great, because sometimes you just need to grab the keyboard because either suddenly you are in an area that you know better, you see a pattern that you want to apply to some code you wrote earlier, or you simply want to drag your partner out of the hole he or she starts digging. Either you break the pattern start coding yourself, or try to direct your partner what to write (which can be really frustrating for you and annoying for him/her). Or sometimes while fixing a test, you notice that you want to touch another class. So do you write a failing test for that class and fix yourself, or let your partner fix it, and come back fix the test you are trying to fix? My experience has always been, if something does not feel natural, stop it, if it is something that you have to keep reminding yourself, figure out why and see if it is necessary. (Yes, test-driven development can feel that way but now I cannot imagin not doing that. Well, JBuilder opentools can be an exception, but I am working on that).

However, it worked surpringly well yesterday.

It is a new area that two developers worked on. One left for vocation so one developer knows the code. Friday morning, we have three stories that we want to start on, all related to the same area. So instead of pairing, we did 'triple-paring' for a couple of hours on one story (Luckily, mine. :D). Since it was not clear how to do it, we decided to ping-pong (or rather cutthroat). And it worked very well. I noticed that we are more focused, and I don't have to keep pulling the other people's leg by asking them to write test first. And we finished rather smoothly.

So my conclusion is, pair-programming works best of two developers are on the same level, and are both disciplined on test-driven development and agressive refactoring. Ping-poing pairing works best when at least one of the developers are still yet to be fully converted. Now that I have written it, it is a little like staing the obvious.

Stand-up Token
When we started this enablment project, we introduced stand-up meeting. So sometimes people are too shy to speak up, or the conversation can drag. Roben brought in a toy ball as a stand up token. A person can speak only when he/she has the ball, and tosses it to the next person once he/she is done. It is another thing that I have heard and never figured out how it helps.

It turns out that having a ball tossed to you does have a psychological effect on you. People speak louder and clearer. Even though we didn't impose the rule that you have to ask for the ball in order to speak, people do feel a bit guilty to strike a conversation without it. Instead, we write topics on the white board and get to it right after the stand-up, which is exactly what we should do.

Role Hats
No I don't have a good experience with this one, yet. But I wonder if it is worth it to introduce a role hats. Lacking business analyst is starting to taking a toll, I think. More and more often we ask around what is the whole picture of a certain feature, because developers implemented them by implementing one story after another, and the QA tested them by writing one acceptance after another. We are using Confluence to put support document, however no one has any incentive to keep them correct (that is right, those documents we have sometimes do not match the old code we see).

Monday, February 28, 2005

Early Deployment

I read about early deployment in XP, I believe in early deployment in XP, still I didn't push it and now I am a little stressed out.

This is a project that we have decided to rewrite part of the functionalities, because of the crazy dependencies I mentioned in last several posts (which is, like, half a year ago). We figure, we are just writing new Java files. We are still going to create one jar file for which the deployment and install script will use. So at the beginning of the project (one month after it started), we did a manual deployment as a proof of concept, rather than focusing on making sure that the existing script works well with it, and then forgot about it.

Now it is time to deploy to testing server and making sure that we can do it easily (one click deployment). I started look into the existing deployment script. Then I realize that it makes some assumption and has to be changed to accomendate the changes we have made.

For one thing, the development environment here used to be Unix box with VI editors. So the CVS modules are really small, of less than 20 files. Each module has a build file that will update library (similar to Maven process), compile, archive, and upload the resulting jar (with build number at the end of the file name) into a central AFS directory. This leads to a very messy building process. For example, if module A depends on module B. At a point of time, a-1.4b100.jar is built using b-1.0b22.jar. And now you want to debug why the product does work (for release fix, for example), you will have to check out module A by its tag "ant-v1.4-b100" and check out module B by its tag "ant-v1.0-b22". Try some fix in B, run the build, upload the jar with the newer build number. Then you need to change the build number for b.jar in the build.xml of A so that the right jar file can be pulled during the build. Then run the build of project A.

Still with me? Now imagine module A depends on not only b-1.0b22.jar, but also c-2.5b23.jar, d-1.0b2.jar, e-1.5b2.jar... If that is not bad enough for you, c-2.5b23.jar can be built upon b-1.0b08.jar, in turn. Now what do you think of that?

Now that we want to encourage refactoring and usage of a real Java IDE (which is more than I can say about Eclipse). We introduced several main projects, each one contains the source directories of the existing modules so that we don't need to move files around (mainly because of CVS). However, having observed how bad it can get, we decided to build everything off the head. So we have one project called "unified" which is where the common library classes go, and several other projects depend on it (3 so far). So when we build, say, project document processor, we want the build script to call to unified build file to test and build the library jar first, then use that jar the compile the current project and run the test.

However, changing the deployment process is out of scope for this process, so we still want the current release script to work. But now that we need to upload two jar files (library jar and the project jar) rather than one, the exiting script has to be modified. And you know how tedious it gets writing scripts. So before I know it, it is already the end of iteration.

So, early deployment, get it out of the way, even if you think it is simple.