Thursday, February 09, 2006

Performance Testing Project

This is the project that we engaged for the last two weeks. Even though it did not happen there are things that I have learned through out the process.

Business

The project came to us as a returned business. Basically, when party B did a contract for party A to build a system, they are required to find a third party to do a performance testing for the system so that they can prove that this system will work with the estimated workload.

This came to us as a returned business. It sounded like a routine contract that will just benefit all parties. Party B gets to finish its contract and get paid, party A will get to know better about the non-functional behavior of the system. As I have learned, the rate for performance testing contract is normally higher than other contracts. So it would have been a good short contract for me (working with a long time ThoughtWorker, Matthew Short) before I head back to China. And ThoughtWorks doesn't have to put me on beach.

For reasons that I cannot publicly post, the contract fell through even though technically we have a pretty good idea of who we need to work with, what we need to produce and what to do to get the data.

So now everyone lose.

This is my first fall-through project. Not totally surprised since I cannot imagine that all engagements that we have turn into a project, but still I am still a bit uneasy to the fact that despite all efforts, we just cannot influence the other parties to prevent them from taking actions that jeopardizes the project.

Or can we, I wonder?

Technically

I have done some performance testing for various projects, but always as part of the project iterative tuning process. It mainly focused on the profiling part rather than the performance benchmarking as a whole.

In the last project in China, the client who hired us had a contract with the customer, which put the specific performance requirement as part of it. It looked pretty vague to be, when reading through that part of the contract, that they are just numbers that can be easily met as long as you are free to choose the hardware, or could be almost impossible if the hardware is fixed.

In preparation for project by referring to the one that we did before, I have learned about performance testing methodology, Transaction Processing Performance Council and their benchmarks (http://www.tpc.org.)

Because the application is going to be deployed on a Linux system (while the last one was on a Windows system), with a little research and the great help from Barrow Kwan, we were able to have a Linux at our disposal, run resource monitoring programs, process the data into csv files, port them over to Microsoft Excel and generate the performance chart.

I guess I didn't totally lose on this after all.

Tuesday, February 07, 2006

Three Announcements from ThoughtWorks

Internally, ThoughtWorks has lots of interesting emails about announcements and discussions. Here are three announcements that I got this week that I think worth logging.

Selenium on OpenQA
Selenium has moved to OpenQA (http://www.openqa.org/selenium/). For those who don't know, Selenium is a web testing technology written to various degree by a couple of ThoughtWorkers and friends. It turns your browser into a web testing tool so that you can test your web application in any browser on any platforms. I have used to in some projects and I strongly suggest you check it out.

CruiseControl Better and Greater
I have been using CruiseControl on every single project that I have worked on in ThoughtWorks. And I have to say that they have come a long way. Now the latest version of CruiseControl launches web server along with the CruiseControl server, and has a "build" button on the reporting page!

Event Announcement: CI and Testing
Greetings Continuous Integration Testers,

This email is being sent to announce an upcoming event for everyone interested in continuous integration and the type of automated testing associated with it.

Jeffrey Fredrick and Paul Julius are cohosting an event that will focus on these topics. The event will use Open Spaces to structure conversation, understanding and innovation.

What: Open Space event discussing all aspects of CI and Testing, together
Where: Chicago, IL
When: Early April (final date tbd)
Who: Everyone interested in CI and Testing
Cost: Free

We'll be inviting people for all manner of projects and places. In fact, feel free to pass this invitation along to anyone that you think will be interested.

For us to finalize the details of time and place we need to get a feel for how many people are likely to attend. If you are interested in attending please join the CITCON mailing list at:

http://groups.yahoo.com/group/citcon

and post an introductory message. In your message it would be useful if you could indicate any topics of special interest and also how likely you are to attend.


Sincerely,
Jeffrey Fredrick (jtf@agitar.com)Paul Julius (pj@thoughtworks.com)