Office Friday

I did not realize the importance of office Friday until we did it last week. Last time we did it, I was working in the office so it was no much different for me, well, plus that I was crunching on the deadline to fix something so I did not find enough time just hang around in the afternoon.

This time I worked extra hours on Thursday and went to work early on Friday, so that I can take the afternoon off and went back to the office. Meeting other people in the office that have been on the road was surely fun. And it was defenitely good to chat with Dave and Scott again. I think enablement work does wear you out slowly because you have to keep teaching other people. Besides patience and money you don't really gain much. I guess one can also learn how to deal with badly designed code and old code but it defenitely is not as pleasure as working with another person who is familiar with XP and have a stimulating conversation. I am not saying the people at Drofnats are dull, it is just that repeating and explain the things that I have taken for grantetd for the last three years is really not all that exciting (seeing the delight in the deveopers' eyes was rewarding, though).

What's up with CruiseControl

The tests cannot run in IDE anymore, again!! The only thing that I can think of is that people just don't use IDE to run tests. Honestly I think this is very wrong. I'll need to send out an email to see how they are using the IDE.

Sent out email and there were no response, go figure. Although the status text is fixed and now it really shows the real status. Just a 'rebuild' button then I am all set.

A bold move and a change in direction

Everybody (well, I was singled out so I had to move along) took a bold move. Rather than spend all the time to sort out the different versions of the jar files and figure out if it is safe to upgrade the dependencies, we have decided to just put everything under one project.

The pro: Now we have the project under the parent directory of the source, rather than using links in Eclipse, we don't have to explain to others why sometimes you need to commit "api" project and sometimes "person" project. A simple set up defenitely save us a lot of time for communication. We now can refactor/rewrite the code using IDE, rather than dealing with each unit test for each class.

The con: The risk. Nobody has any idea what this blind upgrade would bring. However, since this project does not look too complicated by its own business logic, rather its complexity comes from the complicated build/deployment process, we are hoping that what we did won't come back and bite us down the road. Actually, I think we are betting on it.

On the other hand, this is not a decision that made by ThoughtWorkers, rather Wendy is quite on board with this. So if the customer is aware of the decision making and the risk taking, I don't think it will get out of hand.

A change in direction

We were going to slowly refactor the code to be cleaner design. However, after Clinton taking a look at the code and from his previous DAL experience, he suggested that it is actually easier to write another framework, and start moving the existing DAOs over. So rather than change the code in registry or newregistry, we are going to move code over to the new registry3 package that he is writing at the moment.

It is interesting to see how their developers are getting excited over this "framework writing". Clinton is pretty good at test-driven devleopment so I heop in time they will learn that the real value is in the way that he develoes this code, rather that the code or the diagram that comes out of it. Diagrams are nice only when they are not taken literally.

Adam and I are focusing on the tracer tests. I am really glad that he is also doing this because I think this is a safty net that will help us understand how the registry is supposed to work. Rewriting implies that we need to understand all the requirements that the current registry meets, and we DON'T actually have those. They are in the head of multiple person here. We are getting more and mroe information and we need to track them with story cards, however there are still a lot of unknowns.

Bringing in Business Value?

We have been working with the developers there on writing more and more unittesting coverage. However, now I am thinking that we probably should turn around and start writing acceptance test.

Talked to the QA on Friday and boy was she streessed out. They don't have any automated test. What's worse, they don't even have a testing script. She has an Excel spreadsheet that contains a list of "objectives". Whenever there is a new release, someone will deploy it on the testing server. She gets the notification and goes through her objectives. For each item, she has to access the database to find out a suitable testing data, then go to the site do the manual test. Because the database is synchronized with production periodically, there is no testing data setup. And all the steps are in the QA's head. Even the most simple test, for example, testing validating of the datefield for a user who wants to set up absence email, is not a trivial task, and it DOES get broken from time to time.

Since agile process is always about bringing business value to the customer at an early stage, I think we should write automated web tests to relief the QA of the burden, rather than keep doing exercise with the developers. It is fun and all but after all it is just writing unit test for existing features, most of which none of us (including their developers) understand.

HsqlDb and Database Design Issue

Got in memory database working, thank you HSQLDB! With this in place, we will be able to write some unittest or functional tests without worrying about stub out the database. We acquired a set of SQL from Sybase that can help us create the tables and views. However, because Sybase is not using standard data type and SQL, it was a pain to convert all the script. We had to comment out all the ones that we are not going to use.

Here is another good example of why some people should be directed instead of enabled. Because you can make table/view names case sensitive, someone here decided to create views as the same name as the table, except in different case. Thank God that the view is exactly the same as table (used only for READ/WRITE control), I was able to get by without creating it. Also that someone decided to create different view of the same name for different type of user. So it will be interesting to run into that scenario down the road.

This is what you get for not using a nice IDE

Three Drofnats developers all use terminals, vi and cvs command line. So there was no easy way to check references or do the simplest refactoring. So they came up with a few dozens of cvs moduels that depends on each other. Jar files are built through these modules and used by each other as library, in a Maven kind of way. Since everything is done manually, it is impossible to check for references or update calls when a method is changed, or even make sure no cyclic dependencies have been introduced.

For example, Module A can have three version a-1.0b1.jar, a-1.1b3.jar, and 1.2b3.jar, and requires a c-2.3b3.jar. Module B can have two versions, b-2.0b3.jar and b-2.3b3.jar, and has to depend on a-1.1b3.jar and c-2.3b3 to build the latest jar file. Module C has one version c-2.3b3 and has to depend on a-1.0b3.jar

Eh...If you have not been confused, I hope you have seen the big problem here. They have done a VERY good job on making sure the building system can handle all these versions, in a Maven like way. But I don't suppose this can be kept up for a long time.

With the introduction of Eclipse (Not my favorite IDE, not even the second most favorite by the way), we have been able to show the problem to them. However, without any automatic tests, we don't dare just upgrade all modules to depend on latest version of any other modules. It seems that we will have to find out the change between each versions, and manually update them for the simple ones. Then for the not-so-simple ones, we'll have to write functional tests first. The database is not decoupled, however, so we are still researching on how to write test without having to have a full-blown server.

To be continued...

New Project : Drofnats

Started the next project in, let's call it, Drofnats. Compared to the SFD that I visited two weeks ago, this one is just the oppsite. The IT department, at least that ones that we have been talking to, are very much open-minded, after being bitten by a bad project management.

It is quite interesting that according to the switches of scope, time, cost and quality, they are very willing to achieve high qualty, less likely to cut scope, and flexible on time and cost. I am really very happy that they picked us than the other two companies, one of which is IBM, because they could be used if not careful. It is such a good candidate for Agile development, and I really hope we can have a great success.

Today is mostly spent on preparations. The pilot inception deck power point is the most interesting. It reminds me of the design document that we had at Cysive. It is very powerful but yet not to be abused. However, I especially like the slide of "Is and Is-not". It clearly specifiies what is to be covered and what is not, in a simple two-column table. This means that it did not take too long to produce, it will be easy to change, and yet it clearly defines the scope so that everyone will be on the same page.

It is also my first agile project from beginning. In Yabe, they have already planned it by the time I came in, and it was not by a ThoughtWorker. Even though this is not from the very beginning, I still get to see the iteration zero, or the equivalence. It is hectic at certain angle, but it is being handled very well.

The project estimate chart is the next thing I should check out. I also like espeically the way priority/rick/scope is being presented by a two dimensional chart with dots whose size match the scope.

Another Day at the Beach

Another day at the beach. It does give me a lot of time to check out the CruiseControl and its competitors AntHill and DamageControl. CuirseControl is a great open source project that can play a big role on agile development, however it still needs quite a bit work on usability and features.

However, the tests cannot run together because they are affecting the state of some singleton. Interesting, I would think that all ThoughtWorkers, at least those involved in the OpenSoruce project would have an IDE and run tests through IDE. The whole tests take only 1 minute to run using Ant even with fork turned, though. I fixed the tests but too bad that the sourceforge is down, I cannot submit any changes.

Once I submit it I'll get the .Net CD from Dave and take a look into notifications in DamageControl. BTW, the MX4J and JMX are somehow more complicated to understand than I thought they should be.

More notes on notification. Ran into these two open source projects that seems to make IM very easy.
GAIM: A IM just like trillian (http://gaim.sourceforge.net/)
IMTask: A IM library (http://imtask.sourceforge.net/)
We can make CruiseControl server sending out messages through IM about building status, and we can also make a CruiseControl bot that start the build, etc. (On the bot part, maybe should start with an IRC bot)

Still looking at China IT, found quite a few IT related website. Now I just need to go through them and get a feeling on the general knowledge and conception.

Finished Two Projects

Just finished two projects of a client, let's call this client Yabe for potential legal issue.

Yabe is a big client, brought to us by friend of ThoughtWorks. They use ClearCase and Eclipse. I have to say, the set up of the ClearCase and Eclipse is very anti-agile. (I'd like to say it is the products themselves but I have met some other developers that argued that it is the setup instead of the tools themsevles). Everytime I work on this environment, time just fly by while I am staring at the screen waiting for something stupidly long to finish, and before I know it, it is already past 9pm. I cannot imagin how in-hourse developers live their lives, but I sure hope their stock options are well worth it.

(Ok. Here is a good example of how screwed up these tools, or their settings, are. I renamed a private static method that takes no argument and returns void, and it tooke Eclipse 2 minutes to come back. Auuuuuuuuugggggggggggggggggggggggggghhhhhhhhhhhhh)

Here is another testimony of ThoughtWorkers' ability. Dave and Alex (An independent consultant) pulled the subsystem and its library over to a CVS repository, and use IntelliJ to build upon it. CruiseControl is set up. Periodically we sync. up to the Yabe world using a script that Alex wrote. So with CVS and IntelliJ, we were able to happily write programs using test-driven development. Yabe has a document that supposely contains all the requirement of the new application, similar to the use case document. We divided it up into story cards, gave them numbers and came up with our estimates.

So from our team's point of view, we are doing pure XP. We pair, we track, we test, we refactor. The resulting application stays healthy wihle we continue adding more and more compilcated features (BTW, writing web application without using session tracking is not a trivial task). It is my first XP project since the one for Novell in year 2001. And it just felt like an place I belong.

One problem is that as far as the customer concerns, this is just like any other project that they have done. Throwing the document into a blackbox and hoping something good will come out of it by the time of the contract. We didn't realize that ANYTHING different from the document, we were supposed to get approval from the customer. Of course being an XP team, we simply shot an email to the client project manager, asked the question and did the change. At the end, we did more than what the document said, and yet the customer thought we were two-week behind.

Let's Get Started

Ok.

After four months settling in, I finally start writing my blogs.

The reason for this blog is because I have a good vibe about this new company that I recently joined: ThoughtWorks.

There are three main reasons that made me want to leave my rewarding JBuilder career and start a career that requires traveling, talking, dealing with new environment all the time.

1. Extreme Programming and Agile Development: I believe this is the future of the software development. Or at least this will be a very good alternative of RUP. However, I also see frustration during adaptation of XP methodology, to a point that I almost think this is a ideal goal that can never be achieved. However, ThoughtWorkers seem to have done it very well and are up to the challenge of making it into a medium to large size team. If this methodology can be applied to a bigger size project, I defenitely want to be part of it.
2. Career growth: Again and again, Borland management has disappointed me making me realize that there is nothing left for me in Borland. I could talk about the reason for a whole day but I am really tired of it and I'll save my breath here. ThoughtWorks honor developers very much and even though time still has to tell how much I can grow my career here, I feel much more confident in working here. I could learn the art of project management of agile development, and understand what it takes to deliver a good project. There are OpenSource projects launched by ThoughtWorks that I can get involved into. China is also a very good market for software development.
3. Martin Fowler. I have read most of his books and have been up to date on his websites. I believe if he chose ThoughtWorks out of all the possible companies, it must has done something right for him.