Friday, September 09, 2005

Feedback from the Client

I have meant to post this a while ago, but with the traveling my daily schedule is still not settled. So I never found a good time that I can just quiet down and write some blog.

This week is a short week due to the past long weekend. So we were able to convince the client that it is better for us to work from our home office so that we don't need to spend two days traveling for a four-day week. And once again, I am able to continue my routine of blogging.

After my last enablement project (now the term is "transforming" because "enablement" has been used along with "training". I will explain that later), the client team had a project retrospective, discussing what they have learned during the project.

With their still fresh memory, they were able to extrapolate the following, with which they gave the presentation to the rest of the organization. They are nice enough to let me publish this. I hope this can be useful for some of you who are still wondering about agile software engineering, or at least some of the "weird practices" that I have been defending from time to time. This is the first-handed note. Hopefully I can help them revise the format for more formal presentations.

Agile: One Developer's Perspective

What methodology did we use in the past?
  • It doesn't have a name, as far as I know, but here are some features of it
  • big upfront requirements-gathering, big upfront design, with some formal and some informal documentation
  • one developer per application, or a couple of developers working in different tiers of an application
  • integration & testing in a big bang at the end
  • manual testing

Some problems we had with this approach

Big upfront...
  • late-breaking requirements changes combined with inflexible deadlines led to periodic frenzies of coding that eroded the integrity of the big upfront design
  • sometimes we specified & designed more than we could actually implement with the time and resources we had
  • integration late in the development cycle led to periodic frenzies of coding that eroded the integrity of the big upfront design
  • disagreements about how open we should be to changes

One developer, or a couple of developers working in different tiers of an application...
  • we didn't always take advantage of possibilities for using some common code or common approaches across applications
  • difficulty scaling up for larger efforts
  • the culture of code ownership by individuals was sometimes at odds with our "esprit de corps"
Integration & testing in a big bang at the end; manual testing...
  • huge burden placed upon QA (when we had QA at all)
  • difficulties detecting the ramifications of maintenance changes, so we became timid about upgrading applications and we ended up with multiple versions of the same module in production
  • we often had as many opinions about how close we were to being finished as we had team members
  • we did not know how to thoroughly test componentry; we could only test built apps
For me personally, the biggest problem was that I was unhappy. I felt that our team could be great, but we just couldn't get there, and I didn't know how to be "part of the solution".

Enter Thoughtworks, with a philosophy and a toolbox

Some things I was immediately enthusiastic about...
  • Eclipse IDE (Great no matter what methodology you embrace)
  • Automated testing
  • Daily standup meeting
  • Frequent discussions as a group about what is working and what isn't, so that we can make incremental corrections in our direction
  • Realistic attitude towards estimating: as the project progresses, we can get more accurate in our predictions of a delivery date, but we can't expect perfect accuracy at the beginning.
  • steady pace, avoiding individual heroics
  • Simplicity--the art of maximizing the amount of work not done--is essential. (From "Principles behind the agile manifesto" http://agilemanifesto.org/principles.html)
Some things I was skeptical but open-minded about...
  • TDD
  • Test-driven development requires that a unit test be written for each tiny piece of functionality before you write any application code
  • The unit tests are written by the developers, not QA
  • Although "test" appears in its name, test-driven development is not so much about testing as it is about enforcing a development discipline that makes refactoring possible
  • The unit tests serve as documentation of the developer's understanding of requirements
  • "radical co-location"
  • continuous integration
  • fully automated build and test process that allows a team to build and test their software many times a day
  • everybody using the same code base, checking in and updating their code very frequently
  • at first, I was incredulous that checking in and building as frequently as once per day would even be possible; TWers viewed once a day as a minimum
Some things I was deeply skeptical about...
  • paired programming
  • code ownership is to be avoided
  • evolutionary design through refactoring

Martin Fowler's definition

What is Refactoring?
Refactoring is a disciplined technique for restructuring an existing body of code, altering its internal structure without changing its external behavior. Its heart is a series of small behavior preserving transformations. Each transformation (called a 'refactoring') does little, but a sequence of transformations can produce a significant restructuring. Since each refactoring is small, it's less likely to go wrong. The system is also kept fully working after each small refactoring, reducing the chances that a system can get seriously broken during the restructuring.
  • very short iterations
  • less emphasis on formal documentation
  • "embrace change"

A year later
Life on our team is completely different than before. From my perspective, it is much better.
Did the philosophy and the toolbox make a difference?
Yes, but not all by themselves. We had to have...
  • the right attitude
  • one-on-one mentoring from experienced agile developers
  • effective leadership
  • discipline

Attitude
We could not have made changes as extensive as this without
  • positive attitudes
  • team spirit
  • perseverance

one-on-one mentoring from experienced agile developers
  • from the agile manifesto: "value individuals and interactions over processes and tools"
  • TDD feels initially so unnatural, that it is essential to have a patient, experienced person at your elbow
  • without discipline, the agile philosophy can degenerate into "a license to hack", so it is important to have constant reminders about the discipline until it is second nature

effective leadership
  • the project manager may be an exception to the general principle that individual heroics should not be required
  • protected the group from distractions
  • keeps both the big picture and myriad details in mind
  • inspires confidence

discipline
  • do not write even the smallest method without writing a test first; see the test fail, then write the minimum amount of code to make it pass
  • update from the repository frequently, run the entire test suite before each check-in of your own
  • builds & test suites run continuously; if your check-in breaks the build, STOP and fix
  • to maintain design and code integrity, coding decisions are made in pairs...if the pair can agree, go forward
If the discipline is not observed, then we can't "embrace change" and "evolve the design through refactoring" because we can't be confident about extensive changes.


Notes


Below is a summary of BTM/70, a methodology in use at a job I had 20 years ago at a company. It is probably an example of what the "value individuals and interactions over processes and tools" principle is a reaction against. All that I remember about BTM/70 is that there were big binders of sample forms, and every step of the way, one had to photocopy the appropriate forms and fill them out. I do not recall any specific benefits.
The life cycle of an BTM/70 project consists of eight (8) phases ranging from Systems Requirements to System Implementation. Coupled with BTM/70 is ET/70, a Project Control tool set that is used for project planning and tracking. ET/70 assists in milestone planning, helps establish project target dates, supports activity tracking, and detail time reporting of each staff member down to the task level, within each activity, of each phase of the project

No comments: